From mwh at python.net  Sun Jun  1 13:20:56 2003
From: mwh at python.net (Michael Hudson)
Date: Sun, 01 Jun 2003 12:20:56 +0100
Subject: [pypy-dev] [gmane.comp.python.pypy] a few things
Message-ID: <2mhe7af3hj.fsf@starship.python.net>

For some reason, posting via gmane doesn't.

-------------- next part --------------
An embedded message was scrubbed...
From: Michael Hudson <mwh at python.net>
Subject: a few things
Date: Sun, 01 Jun 2003 12:01:13 +0100
Size: 1371
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030601/7019f1a3/attachment.mht>
-------------- next part --------------


-- 
31. Simplicity does not precede complexity, but follows it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From hpk at trillke.net  Sun Jun  1 13:29:47 2003
From: hpk at trillke.net (holger krekel)
Date: Sun, 1 Jun 2003 13:29:47 +0200
Subject: [pypy-dev] [gmane.comp.python.pypy] a few things
In-Reply-To: <2mhe7af3hj.fsf@starship.python.net>;
	from mwh@python.net on Sun, Jun 01, 2003 at 12:20:56PM +0100
References: <2mhe7af3hj.fsf@starship.python.net>
Message-ID: <20030601132947.U9377@prim.han.de>

[Michael Hudson Sun, Jun 01, 2003 at 12:20:56PM +0100]
> For some reason, posting via gmane doesn't.

I tried to restrict sending mail to the pypy-dev list to
members (for anti-spamming reasons) but obviously failed. 
I reenabled the old (everyone can post) scheme for now.

    holger

From mwh at python.net  Mon Jun  2 10:43:26 2003
From: mwh at python.net (Michael Hudson)
Date: 02 Jun 2003 09:43:26 +0100
Subject: [pypy-dev] [gmane.comp.python.pypy] a few things
In-Reply-To: <20030601132947.U9377@prim.han.de>
References: <2mhe7af3hj.fsf@starship.python.net>
	<20030601132947.U9377@prim.han.de>
Message-ID: <7h33cisvpht.fsf@pc150.maths.bris.ac.uk>

holger krekel <hpk at trillke.net> writes:

> [Michael Hudson Sun, Jun 01, 2003 at 12:20:56PM +0100]
> > For some reason, posting via gmane doesn't.
> 
> I tried to restrict sending mail to the pypy-dev list to
> members (for anti-spamming reasons) but obviously failed. 
> I reenabled the old (everyone can post) scheme for now.

Well, I am not a member of the list; I read it via gmane.

I'm not sure what to do about this.  I could subscribe and then
disable delviery, I guess.

Cheers,
M.

-- 
  nonono,  while we're making wild  conjectures about the behavior
  of completely irrelevant tasks, we must not also make serious
  mistakes, or the data might suddenly become statistically valid.
                                        -- Erik Naggum, comp.lang.lisp


From mwh at python.net  Mon Jun  2 10:57:21 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 02 Jun 2003 09:57:21 +0100
Subject: [pypy-dev] 
 Re: [pypy-svn] rev 745 - in pypy/trunk/src/pypy: interpreter module
 objspaceobjspace/std objspace/std/test
In-Reply-To: <20030601163000.20FA55A186@thoth.codespeak.net>
	(arigo@codespeak.net's
	message of "Sun,  1 Jun 2003 18:30:00 +0200 (MEST)")
References: <20030601163000.20FA55A186@thoth.codespeak.net>
Message-ID: <2m3cisamby.fsf@starship.python.net>

arigo at codespeak.net writes:

> classes and inheritance, still missing __init__() support

Oh, I have that :-)

I just can't check it in easily.  I maybe able to come up with some
insane solution like burning diffs to CD, scping them from this
Edinburgh computer room machine to my room, applying them and checking
in from there... or I could wait a week.

Cheers,
M.

-- 
  Programming languages should be designed not by piling feature on
  top of feature, but by removing the weaknesses and restrictions
  that make the additional features appear necessary.
               -- Revised(5) Report on the Algorithmic Language Scheme

From hpk at trillke.net  Mon Jun  2 15:56:40 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 2 Jun 2003 15:56:40 +0200
Subject: [pypy-dev] [gmane.comp.python.pypy] a few things
In-Reply-To: <7h33cisvpht.fsf@pc150.maths.bris.ac.uk>;
	from mwh@python.net on Mon, Jun 02, 2003 at 09:43:26AM +0100
References: <2mhe7af3hj.fsf@starship.python.net>
	<20030601132947.U9377@prim.han.de> <7h33cisvpht.fsf@pc150.maths.bris.ac.uk>
Message-ID: <20030602155640.V9377@prim.han.de>

[Michael Hudson Mon, Jun 02, 2003 at 09:43:26AM +0100]
> holger krekel <hpk at trillke.net> writes:
> 
> > [Michael Hudson Sun, Jun 01, 2003 at 12:20:56PM +0100]
> > > For some reason, posting via gmane doesn't.
> > 
> > I tried to restrict sending mail to the pypy-dev list to
> > members (for anti-spamming reasons) but obviously failed. 
> > I reenabled the old (everyone can post) scheme for now.
> 
> Well, I am not a member of the list; I read it via gmane.
> 
> I'm not sure what to do about this.  I could subscribe and then
> disable delviery, I guess.

Let us keep it open for the time beeing. If spam proofs
to be a problem then we find something. 

    holger

From arigo at tunes.org  Thu Jun  5 17:32:59 2003
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 5 Jun 2003 17:32:59 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
Message-ID: <20030605153259.GA14596@magma.unil.ch>

Hello everybody,

Here is a technical note for the way Samuele and I implemented objects and
types based on multimethods.

In objspace/std/ you will find for each type xxx two files: xxxobject.py and
xxxtype.py. The former defines a class W_XxxObject. Each instance of
W_XxxObject is seen by the user (i.e. the person using the language, i.e.
application-level) as an object of type xxx. The xxxtype.py file, on the other
hand, defines a class W_XxxType, which is expected to have exactly one
instance, which is what the user sees as the xxx type object (e.g. the single
instance of W_IntType is what the user sees as 'types.IntType' or 'int').

The same applies to the type 'object': it has a objectobject.py file defining
the class W_ObjectObject, whose instances are the direct instances of
'object', i.e. what the user can have by writing 'object()'. These are pretty
featureless. There is also objecttype.py with the class W_ObjectType whose
single instance is seen as 'object', the type.

The same for type objects: the file typeobject.py defines the class
W_TypeObject, whose instances are types. That's why all other W_XxxType
classes actually inherit (at interpreter-level) from W_TypeObject. And there
is W_TypeType in typetype.py whose single instance is the 'type' type.

The situation for user-defined types and instances of these types is more
problematic. Right now, all user-defined type object are instances of
W_UserType, found in usertype.py, and instances of these user-defined types
are instances of W_UserObject from userobject.py. Application-level
inheritance is not visible at all as inheritance among these interpreter-level
classes; there is some look-up mecanism in W_TypeObject that is responsible
for emulating it. (This is necessary, otherwise we would be stealing the 
underlying inheritance mecanism of CPython, which is not what we want to do 
for user-defined types with possibly multiple inheritance and method 
resolution orders that depend on the CPython version!).

There is a problem, however, which is related to how we hack to let a
W_UserObject pass for some other W_XxxObject class for some other parts of the
stdobjspace.  Try for example:

>>> class specialtype(type):
...   pass
... 
>>> specialtype('L', (list,), {})
<specialtype object at 138362900>
>>> L = _
>>> L([1,2,3])
  File ".../userobject.py", line 51, in getsinglebuiltintype
    mro = list(w_type.getmro())
AttributeError: W_UserObject instance has no attribute 'getmro'

(I'm going to try to fix this right now.) This is because w_type contains the
'L' object, which the interpreter expects to be a W_TypeObject, but which is
actually a W_UserObject because it is an instance of the W_UserType named
'specialtype'.

We'll have to sort this out, and a proper solution probably means starting
with the following: in CPython, if say C inherits from the built-in class D,
then instances of C are implemented as a structure which starts with the same
fields as a D structure. This is so that the PyObject* pointer can pass for a
pointer to the structure can pass for a pointer to a C or a D object, for the
interpreter. In PyPy, we can emulate this behavior in W_UserObject: an
instance of W_UserObject could pass for an instance of some other
dynamically-determined W_XxxObject using some __getattr__ hacks. This can be
written in such a way that it is RPython-friendly, so that the C translator
knows that it is expected to produce compatible structure layouts and do
pointer typecasts.

Interestingly, we could also later add other implementations of W_UserType 
that don't suffer from this restriction.


A bient?t,

Armin.

From hpk at trillke.net  Sat Jun  7 08:51:51 2003
From: hpk at trillke.net (holger krekel)
Date: Sat, 7 Jun 2003 08:51:51 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030605153259.GA14596@magma.unil.ch>;
	from arigo@tunes.org on Thu, Jun 05, 2003 at 05:32:59PM +0200
References: <20030605153259.GA14596@magma.unil.ch>
Message-ID: <20030607085151.O9377@prim.han.de>

Hello Armin!

[Armin Rigo Thu, Jun 05, 2003 at 05:32:59PM +0200]
> Hello everybody,
> 
> Here is a technical note for the way Samuele and I implemented objects and
> types based on multimethods.

Thanks for the info although i can't fully grok it, yet, because some other
pieces are missing (also see below).

Btw, could someone from the Sweden sprint crew post some
notes about how the current pypy-source tree can be used? What is
expected to work (or if that list is shorter: what does not work :-).

I am especially interested in 

- how the new test-machinery works so i can make sure that i 
  don't break big stuff while trying to fix/play around 

- a recap of the current StdObjsSpace registration/multimethod
  mechanisms. 

- any "entry points" other than interactive.py? 

- is loading of python-modules of the underlying python version
  supported? 

> In objspace/std/ you will find for each type xxx two files: xxxobject.py and
> xxxtype.py. The former defines a class W_XxxObject. Each instance of
> W_XxxObject is seen by the user (i.e. the person using the language, i.e.
> application-level) as an object of type xxx. The xxxtype.py file, on the other
> hand, defines a class W_XxxType, which is expected to have exactly one
> instance, which is what the user sees as the xxx type object (e.g. the single
> instance of W_IntType is what the user sees as 'types.IntType' or 'int').

Is the distinction especially between 'W_TypeObject' and 'W_TypeType' 
really "right"? I thought that these two sort of fall together. 

Anyway, for starters, i just added (what i think) a correct 'repr' to the
W_TypeObject 

    def type_repr(space, w_obj):
        return space.wrap("<type '%s'>" % w_obj.typename)

I also noticed that string representations aka 

    "with'mixed'quotes" 

always result in 

    'with'mixed'quotes'

but i wasn't sure how to fix this. Any hints? 

thanks,

    holger

From arigo at tunes.org  Sat Jun  7 15:00:50 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 7 Jun 2003 15:00:50 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030607085151.O9377@prim.han.de>
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de>
Message-ID: <20030607130050.GA19854@magma.unil.ch>

Hello Holger,

On Sat, Jun 07, 2003 at 08:51:51AM +0200, holger krekel wrote:
> Btw, could someone from the Sweden sprint crew post some
> notes about how the current pypy-source tree can be used? What is
> expected to work (or if that list is shorter: what does not work :-).

I would like to put this information in the Wiki, but I begin to feel confused
with the page names. There are quite a lot of pages with similar intent but
different names or vice-versa, and the name "sprint" generally refers to the
goals or results of one of the sprints, without any logic to know which one.  
Just browse RecentChanges to see what I mean.

We should really name the sprints, e.g. HildesheimSprint and GothenburgSprint.
Can we rename or delete pages ? (Maybe via the svn repository containing the
wiki instance ?)

> - how the new test-machinery works so i can make sure that i 
>   don't break big stuff while trying to fix/play around 

pypy/testall.py runs all the tests, using the object space specified in the
environment variable OBJSPACE -- which must be exactly
pypy.objspace.std.StdObjSpace to use the standard object space; the trivial
object space is used otherwise.

pypy/testwice.py is a hack around the previous one to run all the tests in 
both object spaces.

The individual .../test/test_xxx.py files are still runnable. There is a 
testsupport.py file in all test directories for glueing purposes; running it 
directly should execute all the tests in that directory.

> - a recap of the current StdObjsSpace registration/multimethod
>   mechanisms.

The W_XxxObject classes of the standard object space are, precisely,
*implementations* of objects. This is not the same as the *type* of the
object. It is possible to provide several implementations for the same type
(the user should not see the difference, though). Typical uses of this would
be to hide the int/long disctinction to the user altogether, or more
interestingly to provide more efficient versions of the data structures like
string, list or dict when they become large. 

For example, a complex string implementation can allow constant-time
concatenation. (This would allow algorithms that build a long string to use +=
instead of the less readable trick of storing small strings in a list and
calling ''.join(list) at the end.)  But the simple strings of CPython are much
better for the typical short strings, hence we really need both.

The register() method of a multimethod registers a function that accepts
arguments with a particular *implementation*, and not a particular *type*.  
This is clear in your type_repr() :

>     def type_repr(space, w_obj):
>         return space.wrap("<type '%s'>" % w_obj.typename)

This is the repr() for W_TypeObject's, i.e. for types that are implemented as
a W_TypeObject instance. The W_TypeObject class defines the 'typename'
attribute, so you can read it there. If we had another different
implementation for types the above type_repr() would not apply to it.

There is no way to dispatch a multimethod based on the type of an argument,
but so far it seems that in most cases you really want a specific
implementation anyway. What it seems that we may need at some places is type
checking (not dispatching), to enforce the type of an object independently of
any particular implementation. BTW the same is true for the built-ins that we
implement in Python; right now they do no type checking (they might fail with
a TypeError in the middle of the code, just like any Python function receiving
ill-typed arguments); this is unlike CPython's builtins, which are very
strict.

Let's come back to multimethods. All operators are multimethods, but there are
also type-specific multimethods (like list.append()) defined in the W_XxxType
class. This allows the type to provide several implementations of the method
(typically, if there are several list implementations, each one must define
its own implementation of append()).

As an exception to what I just said, the type of an argument of a multimethod
is actually used for the dispatch in the case where the user refers to this
multimethod as a (bound or unbound) Python method. For example, when we write
'int.__add__(x, y)', the first argument must be an int object (by Python's
unbound methods rule) and so the multimethod only dispatches to the
implementations that have as a first argument some W_XxxObject instance
implementing an int (or a subclass of int). Similarily, when we use
'list.append(lst,obj)' we create an unbound method which corresponds to the
'list_append' multimethod, with the additional condition that the first
(implicit self) argument is constrained to be of type list. The more common
idiom 'lst.append(obj)' does the same, with a bound method instead. (BTW, I
think there is no support for unbound methods in instmethobject.py right now).

In the arguments to register(), the name W_ANY (a synonym for W_Object) means
that anything at this position is fine. This can also be used to write default
implementations that will be called if the multimethod cannot find a more
specific implementation (or if the more specific implementation raises a
FailedToImplement exception). There are examples of that in the file
default.py. Another example would be to provide default type-specific method
implementations that would work for any implementation of the type, which
would be used unless a given implementation provides a more efficient version.
For example :

def any_list_extend(space, w_list, w_otherlist):
    return space.inplace_add(w_list, w_otherlist)

W_ListType.list_extend.register(any_list_extend, W_ANY, W_ANY)

Note that the first argument is not W_ListObject. It still does not mean that
all objects in the world will automatically get an extend() method that
defaults to +=, because the only way to access the list_extend multimethod is
via a bound or unbound method of the list type, restricting its usage to
objects of type list. (The four lines above should be put in listtype.py, not
in listobject.py.)

The getattr multimethod is a bad example that needs to be fixed at some time:
we generally dispatch on the second argument ('w_attr') being W_ANY, but in 
general what we want is a string, and one that we can immediately read and 
compare with other strings, so that what we want is generally a W_StringObject 
and nothing else.

The whole thing still has rough edges. I feel that the inheritance hierarchy
among the classes in objspace/std is confusing (it does not correspond in any
way to the inheritance of Python types). I would also like to investigate the
possibility to dissociate the "container overhead" (the ob_type and
ob_refcount fields in CPython) and the actual object implementation (say,
ob_ival for integers). I am not too sure but I feel that it would allow us to
do some clean-up and could result in interesting features, like an
implementation of "lists of similar objects" that could simply pack the
object's contents in a big array without the ob_type+ob_refcount overhead
(e.g. packing a list of int objects results in a list of structures containing
a single ob_ival field, i.e. an efficient packed array of ints). Yes, I'm
dreaming about obsoleting the 'array' module as well :-)


> - any "entry points" other than interactive.py?

There is main.py in the same directory. Its purpose is that 'python main.py
script-and-options' should be the same as 'pypy script-and-options' if we had
a working 'pypy' program. I guess that main.py should invoke interactive.py
when started with no argument (it doesn't right now).

> - is loading of python-modules of the underlying python version
>   supported? 

Yes, althought there are few modules from the standard library that we can
actually import. And site.py is not automatically loaded, which I guess is the
reason why (our application-level) sys.path does not contain the current
directory, but you can add it (importing sys and manipulating a list work
fine). We also have a problem with exception handling: when application-level
code raises an exception, PyPy tries to import (at application-level again)  
the types module, which fails and raises an exception again, which only
confuses things more. I think Michael has done something about it; if not this
particular point should be fixed by removing dependencies to the types module
in opcode_app.py. (The necessary type objects could be obtained differently;  
at least for now I'm fine with the idea of simply pushing them into
opcode_app.py after we load it.)

> Is the distinction especially between 'W_TypeObject' and 'W_TypeType' 
> really "right"? I thought that these two sort of fall together. 

No: the instances of W_TypeObject are types. There are a lot of them. But the 
single instance of W_TypeType is 'type'.

> I also noticed that string representations aka 
> 
>     "with'mixed'quotes" 
> 
> always result in 
> 
>     'with'mixed'quotes'
> 
> but i wasn't sure how to fix this. Any hints? 

That's in stringobject.py. str_repr() does no quoting at all, really. Here too
Michael already replaced it with another equally bad implementation which
however has the advantage of producing the correct result. The correct thing
to do is to rewrite the quoting algorithm from stringobject.c.


A bient?t,

Armin

From mwh at python.net  Sat Jun  7 21:38:32 2003
From: mwh at python.net (Michael Hudson)
Date: Sat, 07 Jun 2003 20:38:32 +0100
Subject: [pypy-dev] [gmane.comp.python.pypy] deep breakage
Message-ID: <2msmqladaf.fsf@starship.python.net>

What's going on?

-------------- next part --------------
An embedded message was scrubbed...
From: Michael Hudson <mwh at python.net>
Subject: deep breakage
Date: Sat, 07 Jun 2003 20:35:59 +0100
Size: 1906
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030607/07b1e84d/attachment.mht>
-------------- next part --------------


-- 
  The above comment may be extremely inflamatory. For your
  protection, it has been rot13'd twice.
                           -- the signature of "JWhitlock" on slashdot

From 5ba3qpg8l5ps at cs.com  Sun Jun  8 14:45:07 2003
From: 5ba3qpg8l5ps at cs.com (Josefina Giles)
Date: Sun, 08 Jun 03 12:45:07 GMT
Subject: [pypy-dev] Pypy-dev-owner Lose Weight Easy
Message-ID: <r$3--$7bj5$3se@x7n0xz2>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030608/44d807bb/attachment.html>

From arigo at tunes.org  Sun Jun  8 16:13:56 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sun,  8 Jun 2003 07:13:56 -0700 (PDT)
Subject: [pypy-dev] [gmane.comp.python.pypy] deep breakage
In-Reply-To: <2msmqladaf.fsf@starship.python.net>;
	from mwh@python.net on Sat, Jun 07, 2003 at 08:38:32PM +0100
References: <2msmqladaf.fsf@starship.python.net>
Message-ID: <20030608141356.75DCDECA1@bespin.org>

Hello Michael,

On Sat, Jun 07, 2003 at 08:38:32PM +0100, Michael Hudson wrote:
> try:
>     "x" + 1
> except TypeError, e:
>     raise e
> 
> because here 'e' gets bound to a *string* and then this is what's
> reraised.

This is the same problem as in CPython. Most failing internal operations call
PyErr_SetString(), which causes a string to be used as the exception value.
The hack is that when the user code reaches an 'except' clauses the exception
is "normalized", which translates the pair "TypeError class + string" into the
pair "TypeError class + TypeError instance".

We might either choose to mimic this behavior or clean it up a bit, e.g. by
always using internally a single object as the exception instance. String
exceptions are being deprecated, but still we could reasonably emulate them,
say by inventing a StringException class. This would isolate the hacky parts
around this specific class (like setting sys.exc_type to something else than
current_exception.__class__ for this particular case).


A bient?t,

Armin.


From roccomoretti at netscape.net  Mon Jun  9 05:10:52 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Sun, 08 Jun 2003 23:10:52 -0400
Subject: [pypy-dev] Objects and types in the stdobjspace
Message-ID: <79B10674.54E3614E.9ADE5C6A@netscape.net>

Armin Rigo <arigo at tunes.org> wrote:

>On Sat, Jun 07, 2003 at 08:51:51AM +0200, holger krekel wrote:

>> - how the new test-machinery works so i can make sure that i
>> ? don't break big stuff while trying to fix/play around
>
>pypy/testall.py runs all the tests, using the object space specified in the
>environment variable OBJSPACE -- which must be exactly
>pypy.objspace.std.StdObjSpace to use the standard object space; the >trivial
>object space is used otherwise.
>
>pypy/testwice.py is a hack around the previous one to run all the tests in
>both object spaces.

("previous" being the pypy/testall.py file, not any pre-sprint test hook.)

... and pypy/testcts.py is a rather clever piece of code to run tests on 
StdObjSpace, keep track of the results, and report any difference from run 
to run.

>The individual .../test/test_xxx.py files are still runnable. There is a
>testsupport.py file in all test directories for glueing purposes; running 
>it directly should execute all the tests in that directory.

I've been playing around with it over the weekend, and I feel things 
(testing wise) need to be clarified slightly. First, I think we should 
move all pypy-as-a-whole testing related stuff into an approriate 
subdirectory ("pypy/testing" or some such. As it is now, pieces are 
scattered in pypy and pypy/interpreter.

Secondly, the testing framework as written will only run tests at the 
interpreter level. Some tests (such as test_exceptcomp and test_exec) 
should be run at the application level. Additionally, there is no 
provision for running the CPython regression tests automatically within 
the current testing framework.

The thought occured to me if it would not be easiest to "appropriate" the 
CPython regrtest.py framework for our purposes. As best I can tell, it's 
written in a general fashion already, so "all" we would need to do is get 
it to recognize when to run tests under interpreter level and when to run 
tests under application level. (Which could potentially be accomplished by 
a naming convention.)

>> - any "entry points" other than interactive.py?
>
>There is main.py in the same directory. Its purpose is that 'python >main.py
>script-and-options' should be the same as 'pypy script-and-options' if we >had
>a working 'pypy' program. I guess that main.py should invoke >interactive.py
>when started with no argument (it doesn't right now).

Other problems, I assume (from code inspection, not from direct running) 
are that the commandline arguments are not passed to sys.argv, sys.path 
may or may not contain the directory of the script, and 
sys.modules['__main__'] is not set equal to the script module 
(only a problem, I think, when running unittest.main()). And in keeping 
with my rearranging mood, I think main.py should probably be placed in the 
pypy/ directory. -- I also think someone should give me <Dr. Evil> ONE 
BILLION DOLLARS </Dr. Evil> and melons should taste more like currants, so 
that shows you what *I* know.

>> - a recap of the current StdObjsSpace registration/multimethod
>> ? mechanisms.
>
>The W_XxxObject classes of the standard object space are, precisely,
>*implementations* of objects. This is not the same as the *type* of the
>object. It is possible to provide several implementations for the same >type
>(the user should not see the difference, though). Typical uses of this >would
>be to hide the int/long disctinction to the user altogether, or more
>interestingly to provide more efficient versions of the data structures >like
>string, list or dict when they become large.

FANTASTIC! An excellent way of looking at things.

But I'm still a little hazy. Could you walk us through how this scheme 
would work with multiple implementations of the same type and specifics as 
to how this would be different from the (CPython) current implementation 
of types being the same as the implementation?

Congrats to the spriters - it looks like quite a few good things have been 
accomplished.

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From mwh at python.net  Mon Jun  9 11:36:08 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 09 Jun 2003 10:36:08 +0100
Subject: [pypy-dev] Re: [gmane.comp.python.pypy] deep breakage
References: <2msmqladaf.fsf@starship.python.net>
	<20030608141356.75DCDECA1@bespin.org>
Message-ID: <2mvfvfiodz.fsf@starship.python.net>

I wonder if I can post via gmane yet...

Armin Rigo <arigo at tunes.org> writes:

> Hello Michael,
>
> On Sat, Jun 07, 2003 at 08:38:32PM +0100, Michael Hudson wrote:
>> try:
>>     "x" + 1
>> except TypeError, e:
>>     raise e
>> 
>> because here 'e' gets bound to a *string* and then this is what's
>> reraised.
>
> This is the same problem as in CPython. Most failing internal operations call
> PyErr_SetString(), which causes a string to be used as the exception value.
> The hack is that when the user code reaches an 'except' clauses the exception
> is "normalized", which translates the pair "TypeError class + string" into the
> pair "TypeError class + TypeError instance".

I think I now have code that does this, after modifying
decode_code_arguments to not try to catch IndexError (that one was fun
to find!).

> We might either choose to mimic this behavior or clean it up a bit, e.g. by
> always using internally a single object as the exception instance.

I couldn't get normalizing the exception at raise time (roughly what
you mean here, right?) to work... there may be a reason other than
performance that CPython does it this way.

> String exceptions are being deprecated, but still we could
> reasonably emulate them, say by inventing a StringException
> class. This would isolate the hacky parts around this specific class
> (like setting sys.exc_type to something else than
> current_exception.__class__ for this particular case).

I'm not sure I understand...

Cheers,
M>

-- 
 Two decades later, well-known hacker Henry Spencer described the 
 Perl scripting language as a "Swiss-Army chainsaw", intending to 
 convey his evaluation of the language as exceedingly powerful but 
 ugly and noisy and prone to belch noxious fumes.   -- the jargon file


From mwh at python.net  Mon Jun  9 11:46:56 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 09 Jun 2003 10:46:56 +0100
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
References: <79B10674.54E3614E.9ADE5C6A@netscape.net>
Message-ID: <2msmqjinvz.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> Armin Rigo <arigo at tunes.org> wrote:
>
>>On Sat, Jun 07, 2003 at 08:51:51AM +0200, holger krekel wrote:
>
>>> - how the new test-machinery works so i can make sure that i
>>> ? don't break big stuff while trying to fix/play around
>>
>>pypy/testall.py runs all the tests, using the object space specified in the
>>environment variable OBJSPACE -- which must be exactly
>>pypy.objspace.std.StdObjSpace to use the standard object space; the >trivial
>>object space is used otherwise.
>>
>>pypy/testwice.py is a hack around the previous one to run all the tests in
>>both object spaces.
>
> ("previous" being the pypy/testall.py file, not any pre-sprint test hook.)
>
> ... and pypy/testcts.py is a rather clever piece of code to run tests on 
> StdObjSpace, keep track of the results, and report any difference from run 
> to run.

'clever' is a strange way of spelling 'unfinished' :-) Ideally, this
would run after every checkin (or so) and hassle the checker-in (or
this list) if any tests broke.  It would be nice to get this in place
before or early on in the next sprint.

>>The individual .../test/test_xxx.py files are still runnable. There is a
>>testsupport.py file in all test directories for glueing purposes; running 
>>it directly should execute all the tests in that directory.
>
> I've been playing around with it over the weekend, and I feel things 
> (testing wise) need to be clarified slightly. First, I think we should 
> move all pypy-as-a-whole testing related stuff into an approriate 
> subdirectory ("pypy/testing" or some such. As it is now, pieces are 
> scattered in pypy and pypy/interpreter.

Probably, yes.

> Secondly, the testing framework as written will only run tests at the 
> interpreter level. Some tests (such as test_exceptcomp and test_exec) 
> should be run at the application level.

Yes.  I don't see how to easily and gracefully allow this.

> Additionally, there is no provision for running the CPython
> regression tests automatically within the current testing framework.
>
> The thought occured to me if it would not be easiest to "appropriate" the 
> CPython regrtest.py framework for our purposes. As best I can tell, it's 
> written in a general fashion already, so "all" we would need to do is get 
> it to recognize when to run tests under interpreter level and when to run 
> tests under application level. (Which could potentially be accomplished by 
> a naming convention.)

Hmm, maybe.  regrtest has become somewhat crufty over the years, I
don't think we need all of it.  In a way, I'd prefer to keep things
within the unittest framework if possible (although working closely
with it for the first time has made its Java heritage unpleasantly
clear...).

We also need a LOT more tests!

>>> - any "entry points" other than interactive.py?
>>
>>There is main.py in the same directory. Its purpose is that 'python >main.py
>>script-and-options' should be the same as 'pypy script-and-options' if we >had
>>a working 'pypy' program. I guess that main.py should invoke >interactive.py
>>when started with no argument (it doesn't right now).
>
> Other problems, I assume (from code inspection, not from direct
> running) are that the commandline arguments are not passed to
> sys.argv, sys.path may or may not contain the directory of the
> script, and sys.modules['__main__'] is not set equal to the script
> module (only a problem, I think, when running unittest.main()).

Yes.  It's very much unfinished -- and actually it would make a good
project for someone who doesn't get the ins and outs of the standard
object space to move this further to completion...

> And in keeping with my rearranging mood, I think main.py should
> probably be placed in the pypy/ directory.

Hmm.  Maybe, or maybe the executable that people actually run will
look like this:

#!/usr/bin/python
from pypy.interpreter import main
main.main()

[...]

> Congrats to the spriters - it looks like quite a few good things
> have been accomplished.

Thanks, I think we really moved things forward.  Are you able to come
to the next sprint?

Cheers,
M.

-- 
48. The best book on programming for the layman is "Alice in
    Wonderland"; but that's because it's the best book on
    anything for the layman.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From arigo at tunes.org  Mon Jun  9 13:09:58 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 9 Jun 2003 13:09:58 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <79B10674.54E3614E.9ADE5C6A@netscape.net>
References: <79B10674.54E3614E.9ADE5C6A@netscape.net>
Message-ID: <20030609110958.GC30895@magma.unil.ch>

Hello Rocco,

On Sun, Jun 08, 2003 at 11:10:52PM -0400, Rocco Moretti wrote:
> But I'm still a little hazy. Could you walk us through how this scheme 
> would work with multiple implementations of the same type and specifics as 
> to how this would be different from the (CPython) current implementation 
> of types being the same as the implementation?

Ok. In CPython the type completely defines the implementation, via the
PyTypeObject structure (tp_size and all the function pointers). All objects
have a type pointer (ob_type), which it a very fast way to retrieve both the
type of the object and its implementation (as it is the same thing).

Ultimately this PyObject structure is something that we will want to have in
*some* versions of PyPy, so that we can be compatible with CPython's extension
modules, at least. I'm confident that we'll be able to do that automatically
based on the current (different) way the stdobjspace works. This is what we
did with multimethods: during the latest sprint we could tie them back to
Python's __add__&co methods, but internally we use multimethods all the way
simply because they are more natural in our context.

The "natural" way to look at the object implementation of the stdobjspace is
that there is only a finite and reasonably small number of different
implementations defined (unlike types, of which the user can create as many as
he wishes). The translator-to-C program can thus implement a wrapped object
with a struct that has some (arbitrary) tag to distinguish between
implementations. In CPython, the tag is the ob_type field. In our case we are
free to choose whatever we want. For example it could be a small integer:  
0=W_IntObject, 1=W_ListObject, 2=W_TupleObject, and so on. It can be
completely arbitrary. It could also be a pointer to some data describing the
implementation. But using a small integer makes multimethod dispatch extremely
fast : to dispatch on objects 'a' and 'b', read 'a->tag' and 'b->tag' and use
them as indices in a (N by N) table of function pointers. It is quite simpler
and faster than CPython's way of playing around with each object's type's
tp_number->nb_add.

That's for the motivation. Now several implementations for the same type can
coexist, provided we give some heuristics to select between them. For example,
suppose we have two string implementations, a W_StringObject and a
W_ConcatenatedStringObject. Then the concatenation of two W_StringObject
should check if the resulting string would grow larger than some threshold; if
so, instead of building and returning a W_StringObject it builds and returns a
W_ConcatenatedStringObject, enabling whose new algorithms to carry on with the
manipulation of what the user still sees as a plain string.

Choice of implementation can also be done when the object is initially built.  
For example, imagine we had a W_SmallIntObject implementation that can store
an integer of up to 30 bits by abusing a pointer field (e.g. by setting the
last two bits to an odd value, and assuming real pointers are never odd).
There should be (there isn't yet) an implementation of inttype_new() in
inttype.py (for calls to 'int(obj)'). This one should examine the numeric
value it should build and depending on whether it is small enough to fit 30
bits or not, it can build a W_SmallIntObject or a W_IntObject. Actually,
'space.wrap(someinteger)' could also invoke this mechanism.


A bient?t,

Armin.

From arigo at tunes.org  Mon Jun  9 13:30:16 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 9 Jun 2003 13:30:16 +0200
Subject: [pypy-dev] Re: [gmane.comp.python.pypy] deep breakage
In-Reply-To: <2mvfvfiodz.fsf@starship.python.net>
References: <2msmqladaf.fsf@starship.python.net>
	<20030608141356.75DCDECA1@bespin.org> <2mvfvfiodz.fsf@starship.python.net>
Message-ID: <20030609113016.GA31963@magma.unil.ch>

Hello Michael,

On Mon, Jun 09, 2003 at 10:36:08AM +0100, Michael Hudson wrote:
> > We might either choose to mimic this behavior or clean it up a bit, e.g. by
> > always using internally a single object as the exception instance.
> 
> I couldn't get normalizing the exception at raise time (roughly what
> you mean here, right?) to work... there may be a reason other than
> performance that CPython does it this way.

Ah. Not sure exactly why. I am only aware of two ways CPython ever tries to
set exceptions: either by the 'raise' statement, which does the normalization
stuff immediately; or by one of the PyErr_SetXxx() calls, which as far as I am
aware is only ever used on built-in exception types (so no user code runs as a
result of normalizing these exceptions). The only reason I can see so far is
MemoryError, which (if we don't normalize it right away) can be raised without
memory allocation. However even this example is dubious, because we could have
a pre-built MemoryError instance in a corner just in case. But well, I'm not
completely aware of all the funny start-up inter-dependencies...

> > String exceptions are being deprecated, but still we could
> > reasonably emulate them, say by inventing a StringException
> > class. This would isolate the hacky parts around this specific class
> > (like setting sys.exc_type to something else than
> > current_exception.__class__ for this particular case).
> 
> I'm not sure I understand...

The point what just that if we replace the internal exc_type/exc_value pair
with a single exc_value pointer (which is the clean thing to do in my
opinion), then we cannot represent string exceptions any more and have to come
up with a new class whose instances can store both the string describing the
exception, and the associated value.


Armin

From mwh at python.net  Mon Jun  9 13:49:28 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 09 Jun 2003 12:49:28 +0100
Subject: [pypy-dev] Re: [gmane.comp.python.pypy] deep breakage
References: <2msmqladaf.fsf@starship.python.net>
	<20030608141356.75DCDECA1@bespin.org>
	<2mvfvfiodz.fsf@starship.python.net>
	<20030609113016.GA31963@magma.unil.ch>
Message-ID: <2mn0grii7r.fsf@starship.python.net>

Armin Rigo <arigo at tunes.org> writes:

> Hello Michael,
>
> On Mon, Jun 09, 2003 at 10:36:08AM +0100, Michael Hudson wrote:
>> > We might either choose to mimic this behavior or clean it up a bit, e.g. by
>> > always using internally a single object as the exception instance.
>> 
>> I couldn't get normalizing the exception at raise time (roughly what
>> you mean here, right?) to work... there may be a reason other than
>> performance that CPython does it this way.
>
> Ah. Not sure exactly why. I am only aware of two ways CPython ever tries to
> set exceptions: either by the 'raise' statement, which does the normalization
> stuff immediately; or by one of the PyErr_SetXxx() calls, which as far as I am
> aware is only ever used on built-in exception types (so no user code runs as a
> result of normalizing these exceptions). The only reason I can see so far is
> MemoryError, which (if we don't normalize it right away) can be raised without
> memory allocation. However even this example is dubious, because we could have
> a pre-built MemoryError instance in a corner just in case. But well, I'm not
> completely aware of all the funny start-up inter-dependencies...

But but but ... instantiating an Exception object involves calling a
callable, which unless you're careful will involve a call to
unpackiterable which depends on being able to raise IndexError ...

Actually, my change to decode_code_arguments may have made this
possible.  I'll check, but I'm sure there are other entertaining
infinite recursions lying in wait.

>> > String exceptions are being deprecated, but still we could
>> > reasonably emulate them, say by inventing a StringException
>> > class. This would isolate the hacky parts around this specific class
>> > (like setting sys.exc_type to something else than
>> > current_exception.__class__ for this particular case).
>> 
>> I'm not sure I understand...
>
> The point what just that if we replace the internal
> exc_type/exc_value pair with a single exc_value pointer (which is
> the clean thing to do in my opinion), then we cannot represent
> string exceptions any more and have to come up with a new class
> whose instances can store both the string describing the exception,
> and the associated value.

Oh right.

I could explain how much I care about string exceptions, but with a
fixed width font, all characters are too wide.

Cheers,
M.

-- 
  The ability to quote is a serviceable substitute for wit.
                                                -- W. Somerset Maugham


From mwh at python.net  Mon Jun  9 14:16:31 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 09 Jun 2003 13:16:31 +0100
Subject: [pypy-dev] Re: [gmane.comp.python.pypy] deep breakage
References: <2msmqladaf.fsf@starship.python.net>
	<20030608141356.75DCDECA1@bespin.org>
	<2mvfvfiodz.fsf@starship.python.net>
Message-ID: <2mu1aztpi8.fsf@starship.python.net>

Michael Hudson <mwh at python.net> writes:

>> This is the same problem as in CPython. Most failing internal operations call
>> PyErr_SetString(), which causes a string to be used as the exception value.
>> The hack is that when the user code reaches an 'except' clauses the exception
>> is "normalized", which translates the pair "TypeError class + string" into the
>> pair "TypeError class + TypeError instance".
>
> I think I now have code that does this, after modifying
> decode_code_arguments to not try to catch IndexError (that one was fun
> to find!).

I've checked my code in.  I'm not totally certain of it, but I was
able to reenable a few tests of the exception machinery in
test_interpreter.py, so I'm going to pronounce it "good enough for
now".

Incidentally, on this machine

Plain Old Python does something like 5-6000 pystones a sec
PyPy/TrivialObjSpace does ~ 7
PyPy/StdObjSpace does ~ 0.2

(this is in the battery-saving reduced processor performance mode, &
with some hacks to let pyston import time.clock)

Cheers,
M.

-- 
  Worryingly, DEFUN appears to be a function that removes all the
  fun from something: after using it all your code is converted 
  to C++.                              -- Tim Bradshaw, comp.lang.lisp


From guido at python.org  Mon Jun  9 15:54:30 2003
From: guido at python.org (Guido van Rossum)
Date: Mon, 09 Jun 2003 09:54:30 -0400
Subject: [pypy-dev] Re: deep breakage
In-Reply-To: Your message of "Mon, 09 Jun 2003 12:00:02 +0200."
             <20030609100002.26FE45BFF7@thoth.codespeak.net> 
References: <20030609100002.26FE45BFF7@thoth.codespeak.net> 
Message-ID: <200306091354.h59DsUh04062@odiug.zope.com>

> From: Armin Rigo <arigo at tunes.org>
> To: Michael Hudson <mwh at python.net>
> Subject: Re: [pypy-dev] [gmane.comp.python.pypy] deep breakage
> 
> Hello Michael,
> 
> On Sat, Jun 07, 2003 at 08:38:32PM +0100, Michael Hudson wrote:
> > try:
> >     "x" + 1
> > except TypeError, e:
> >     raise e
> > 
> > because here 'e' gets bound to a *string* and then this is what's
> > reraised.

This is of course false.  e gets a TypeError instance (or at least
that's what it does in CPython, and what it should do).

> This is the same problem as in CPython. Most failing internal
> operations call PyErr_SetString(), which causes a string to be used
> as the exception value.

No, as the argument to the exception constructor.

> The hack is that when the user code reaches an 'except' clauses the
> exception is "normalized", which translates the pair "TypeError
> class + string" into the pair "TypeError class + TypeError
> instance".

Right, and for all practical purposes,

  raise TypeError, "foo"

is the same as

  raise TypeError("foo")

> We might either choose to mimic this behavior or clean it up a bit,
> e.g. by always using internally a single object as the exception
> instance.

That's fine.  CPython uses the pair so that for certain exceptions
raised *internally* (by C code, not by raise statements) the
instantiation is never done if the exception is caught and cleared by
C code further down the stack.  For example, when iterating over a
built-in sequence, the StopIteration exception is raised but never
instantiated.

> String exceptions are being deprecated, but still we could
> reasonably emulate them, say by inventing a StringException
> class. This would isolate the hacky parts around this specific class
> (like setting sys.exc_type to something else than
> current_exception.__class__ for this particular case).

I would not even bother.  String exceptions are strongly deprecated
part of the language.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Mon Jun  9 15:59:57 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 09 Jun 2003 14:59:57 +0100
Subject: [pypy-dev] Re: deep breakage
References: <20030609100002.26FE45BFF7@thoth.codespeak.net>
	<200306091354.h59DsUh04062@odiug.zope.com>
Message-ID: <2mu1azgxlu.fsf@starship.python.net>

Guido van Rossum <guido at python.org> writes:

>> From: Armin Rigo <arigo at tunes.org>
>> To: Michael Hudson <mwh at python.net>
>> Subject: Re: [pypy-dev] [gmane.comp.python.pypy] deep breakage
>> 
>> Hello Michael,
>> 
>> On Sat, Jun 07, 2003 at 08:38:32PM +0100, Michael Hudson wrote:
>> > try:
>> >     "x" + 1
>> > except TypeError, e:
>> >     raise e
>> > 
>> > because here 'e' gets bound to a *string* and then this is what's
>> > reraised.
>
> This is of course false.  e gets a TypeError instance (or at least
> that's what it does in CPython, and what it should do).

Heh, no it *was* true for PyPy, and I was describing a bug.

I think I've fixed it now (I made a bunch of checkins an hour or so
ago).

Cheers,
M.

-- 
  It could be argued that since Suitespot is infinitely terrible,
  that anything else, by very definition of being anything else,
  is infinitely superior.                -- ".", alt.sysadmin.recovery


From guido at python.org  Mon Jun  9 16:24:36 2003
From: guido at python.org (Guido van Rossum)
Date: Mon, 09 Jun 2003 10:24:36 -0400
Subject: [pypy-dev] Re: deep breakage
In-Reply-To: Your message of "Mon, 09 Jun 2003 12:00:02 +0200."
             <20030609100002.26FE45BFF7@thoth.codespeak.net> 
References: <20030609100002.26FE45BFF7@thoth.codespeak.net> 
Message-ID: <200306091424.h59EOaW04165@odiug.zope.com>

> From: Armin Rigo <arigo at tunes.org>
> To: Michael Hudson <mwh at python.net>
> Subject: Re: [pypy-dev] [gmane.comp.python.pypy] deep breakage
> 
> Hello Michael,
> 
> On Sat, Jun 07, 2003 at 08:38:32PM +0100, Michael Hudson wrote:
> > try:
> >     "x" + 1
> > except TypeError, e:
> >     raise e
> > 
> > because here 'e' gets bound to a *string* and then this is what's
> > reraised.

This is of course false.  e gets a TypeError instance (or at least
that's what it does in CPython, and what it should do).

> This is the same problem as in CPython. Most failing internal
> operations call PyErr_SetString(), which causes a string to be used
> as the exception value.

No, as the argument to the exception constructor.

> The hack is that when the user code reaches an 'except' clauses the
> exception is "normalized", which translates the pair "TypeError
> class + string" into the pair "TypeError class + TypeError
> instance".

Right, and for all practical purposes,

  raise TypeError, "foo"

is the same as

  raise TypeError("foo")

> We might either choose to mimic this behavior or clean it up a bit,
> e.g. by always using internally a single object as the exception
> instance.

That's fine.  CPython uses the pair so that for certain exceptions
raised *internally* (by C code, not by raise statements) the
instantiation is never done if the exception is caught and cleared by
C code further down the stack.  For example, when iterating over a
built-in sequence, the StopIteration exception is raised but never
instantiated.

> String exceptions are being deprecated, but still we could
> reasonably emulate them, say by inventing a StringException
> class. This would isolate the hacky parts around this specific class
> (like setting sys.exc_type to something else than
> current_exception.__class__ for this particular case).

I would not even bother.  String exceptions are strongly deprecated
part of the language.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From arigo at tunes.org  Mon Jun  9 16:30:13 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 9 Jun 2003 16:30:13 +0200
Subject: [pypy-dev] Re: deep breakage
In-Reply-To: <200306091354.h59DsUh04062@odiug.zope.com>
References: <20030609100002.26FE45BFF7@thoth.codespeak.net>
	<200306091354.h59DsUh04062@odiug.zope.com>
Message-ID: <20030609143013.GA3745@magma.unil.ch>

Hello Guido,

On Mon, Jun 09, 2003 at 09:54:30AM -0400, Guido van Rossum wrote:
> > This is the same problem as in CPython. Most failing internal
> > operations call PyErr_SetString(), which causes a string to be used
> > as the exception value.
> 
> No, as the argument to the exception constructor.

Well, it is used as the exception value internally (stored in exc_value) until
some other part of code that actually wants to show the exception to Python
code normalizes it. This is what I was proposing to remove (and let our
equivalent of PyErr_SetString() immediately build the exception instance). I'm
not too concerned about building a StopIteration instance too much right now.  
We could even have a single pre-built instance of it for performance purposes
(later).

> I would not even bother.  String exceptions are strongly deprecated
> part of the language.

Ok. I tend to write occasionally 'raise "this-should-not-happend!"', but never
'raise X, Y' where X is a string. If it's not just me, I guess that this
limited backwards support could be quite easily hacked in by having 'raise
String' default to 'raise Exception, String', then.


A bient?t,

Armin.

From hpk at trillke.net  Tue Jun 10 00:02:55 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 10 Jun 2003 00:02:55 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030607130050.GA19854@magma.unil.ch>;
	from arigo@tunes.org on Sat, Jun 07, 2003 at 03:00:50PM +0200
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
Message-ID: <20030610000255.B21086@prim.han.de>

Hi Armin,

[Armin Rigo Sat, Jun 07, 2003 at 03:00:50PM +0200]
> Hello Holger,
> 
> On Sat, Jun 07, 2003 at 08:51:51AM +0200, holger krekel wrote:
> > Btw, could someone from the Sweden sprint crew post some
> > notes about how the current pypy-source tree can be used? What is
> > expected to work (or if that list is shorter: what does not work :-).
> 
> I would like to put this information in the Wiki, but I begin to feel confused
> with the page names. There are quite a lot of pages with similar intent but
> different names or vice-versa, and the name "sprint" generally refers to the
> goals or results of one of the sprints, without any logic to know which one.  
> Just browse RecentChanges to see what I mean.
> 
> We should really name the sprints, e.g. HildesheimSprint and GothenburgSprint.
> Can we rename or delete pages ? (Maybe via the svn repository containing the
> wiki instance ?)

I think that's a very good idea (HildesheimSprint etc.). We need to get
the wiki into a better shape, badly. In July i plan to spent more time
to work on advancing the current wiki code and content. A main goal is
to make it easy to edit via a subversion-checkout using your favorite
editor. 

> pypy/testwice.py is a hack around the previous one to run all the tests in 
> both object spaces.
> 
> The individual .../test/test_xxx.py files are still runnable. There is a 
> testsupport.py file in all test directories for glueing purposes; running it 
> directly should execute all the tests in that directory.

Btw, I don't like the duplicated testsupport.py files. I think Rocco already 
suggested to put one version into a fixed place which i agree too. 
pypy.test.support seems like a canonical place. Filename completions
on "te<tab>" should result in "test_" :-) 

> > - a recap of the current StdObjsSpace registration/multimethod
> >   mechanisms.
> 
> The W_XxxObject classes of the standard object space are, precisely,
> *implementations* of objects. This is not the same as the *type* of the
> object. It is possible to provide several implementations for the same type
> (the user should not see the difference, though). Typical uses of this would
> be to hide the int/long disctinction to the user altogether, or more
> interestingly to provide more efficient versions of the data structures like
> string, list or dict when they become large. 

If we would restrict ourself to just one possible implementation of a
type then it wouldn't be neccesary to have a W_XxxObject for each type
but the implementation methods could be directly on the Type object? 

> For example, a complex string implementation can allow constant-time
> concatenation. (This would allow algorithms that build a long string to use +=
> instead of the less readable trick of storing small strings in a list and
> calling ''.join(list) at the end.)  But the simple strings of CPython are much
> better for the typical short strings, hence we really need both.

I am wondering if this design comes from your psyco experience. IIRC
PSYCO improves the common "string+=something" cases by doing the 
list/append dance internally. Which model does PSYCO use to do this
compared to the PyPy-approach? 

> The register() method of a multimethod registers a function that accepts
> arguments with a particular *implementation*, and not a particular *type*.  
> This is clear in your type_repr() :
> 
> >     def type_repr(space, w_obj):
> >         return space.wrap("<type '%s'>" % w_obj.typename)
> 
> This is the repr() for W_TypeObject's, i.e. for types that are implemented as
> a W_TypeObject instance. The W_TypeObject class defines the 'typename'
> attribute, so you can read it there. If we had another different
> implementation for types the above type_repr() would not apply to it.

Any reason not name it "__name__" instead of "typename"? 

> There is no way to dispatch a multimethod based on the type of an argument,

But this could be emulated by registering all implementations of a type,
couldn't it? 

And if you register the signature 

    (W_StringObject, W_ANY)

and the signature 

    (W_ANY, W_StringObject)

for a "MultiMethod" which one gets selected? 
IOW, what are the exact resolving rules ... 
Left-to-right more-specific wins? 

> Let's come back to multimethods. All operators are multimethods, but there are
> also type-specific multimethods (like list.append()) defined in the W_XxxType
> class. This allows the type to provide several implementations of the method
> (typically, if there are several list implementations, each one must define
> its own implementation of append()).
> 
> As an exception to what I just said, the type of an argument of a multimethod
> is actually used for the dispatch in the case where the user refers to this
> multimethod as a (bound or unbound) Python method. For example, when we write
> 'int.__add__(x, y)', the first argument must be an int object (by Python's
> unbound methods rule) 

I'd have thought that the "unbound methods rule" is more a restriction
rooting in the dependence of a python method on its "fixed" data structure of 
the object. If so, wouldn't it be more pythonic to avoid LBYL (Look
before you leap) style of restrictions? 

> In the arguments to register(), the name W_ANY (a synonym for W_Object) means
> that anything at this position is fine. 

(trying hard to avoid to get picky about names but failing ...) why
not use plain W_Object? W_ANY sounds like CORBA to me :-)

Anyway, this all sounds good and makes sense to me even if i am asking
questions all over the place. 

> This can also be used to write default
> implementations that will be called if the multimethod cannot find a more
> specific implementation (or if the more specific implementation raises a
> FailedToImplement exception). There are examples of that in the file
> default.py. Another example would be to provide default type-specific method
> implementations that would work for any implementation of the type, which
> would be used unless a given implementation provides a more efficient version.
> For example :
> 
> def any_list_extend(space, w_list, w_otherlist):
>     return space.inplace_add(w_list, w_otherlist)
> 
> W_ListType.list_extend.register(any_list_extend, W_ANY, W_ANY)

The register methods scattered all over the source make the source
code harder to read IMHO. Any way to introduce a sufficiently strict naming 
scheme and some magic of autoregistering the methods by introspection/analysis
of the names at the end of a file? 

> ...
> > I also noticed that string representations aka 
> > 
> >     "with'mixed'quotes" 
> > 
> > always result in 
> > 
> >     'with'mixed'quotes'
> > 
> > but i wasn't sure how to fix this. Any hints? 
> 
> That's in stringobject.py. str_repr() does no quoting at all, really. Here too
> Michael already replaced it with another equally bad implementation which
> however has the advantage of producing the correct result. The correct thing
> to do is to rewrite the quoting algorithm from stringobject.c.

which requires some more string methods. Lots of ways to go and improve
stuff :-)

greetings,

    holger

From roccomoretti at netscape.net  Tue Jun 10 05:53:20 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Mon, 09 Jun 2003 23:53:20 -0400
Subject: [pypy-dev] Objects and types in the stdobjspace
Message-ID: <09881AA6.49B3E698.9ADE5C6A@netscape.net>

holger krekel <hpk at trillke.net> wrote:

>Btw, I don't like the duplicated testsupport.py files. I think Rocco already 
>suggested to put one version into a fixed place which i agree too. 
>pypy.test.support seems like a canonical place. 

The one problem with that is if you are sitting 
in /norwegian/blue/pypy/interpreter/test, and '/norwegian/blue/' isn't on 
your PYTHONPATH, you choke on trying to import pypy.test.support. One of 
the main reasons (that I see, at least) for testsupport is that by doing a 
relative 'import testsupport', it automagically massages your PYTHONPATH 
such that 'from pypy...' imports work. But I agree that it probably should 
be pared down to a stub, and it looks like someone (mwh?) has already 
started that by moving a bunch of code to testtools.py

My suggestion was to move all the other files for general test support 
(unittest_w.py, testtools.py etc) to a single directory, to leave the main 
directory and the interpreter directory relatively uncluttered.

>Filename completions
>on "te<tab>" should result in "test_" :-) 

Could we rename it 'support_tests.py'?

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From roccomoretti at netscape.net  Tue Jun 10 06:05:54 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Tue, 10 Jun 2003 00:05:54 -0400
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
Message-ID: <5467712B.6B233B39.9ADE5C6A@netscape.net>

Michael Hudson <mwh at python.net> wrote:

>roccomoretti at netscape.net (Rocco Moretti) writes:

>> Secondly, the testing framework as written will only run tests at the
>> interpreter level. Some tests (such as test_exceptcomp and test_exec)
>> should be run at the application level.
>
>Yes. ?I don't see how to easily and gracefully allow this.

Naming convention? test_excptcomp_app.py and test_exec_app.py for example?

>Hmm, maybe. ?regrtest has become somewhat crufty over the years, I
>don't think we need all of it. ?In a way, I'd prefer to keep things
>within the unittest framework if possible 

regrtest has provisions for working with unittest files, and if we want to 
work with the CPython regression tests, we need to deal with regrtest in 
some fashion anyway. (But I agree - regrtest is intimidating to work with.)

>(although working closely
>with it for the first time has made its Java heritage unpleasantly
>clear...).

How so?

>We also need a LOT more tests!

Like what? (Knowing what to test is 73.495% of the battle [+- 50%])

>> And in keeping with my rearranging mood, I think main.py should
>> probably be placed in the pypy/ directory.
>
>Hmm. ?Maybe, or maybe the executable that people actually run will
>look like this:
>
>#!/usr/bin/python
>from pypy.interpreter import main
>main.main()

I think: 

#!/usr/bin/python
from pypy import main
main.main()

is a bit cleaner, especially if we start to expand into the realm of 
multiple BytecodeSpaces/ExecutionSpaces (i.e. if interpreter becomes 
std_interpreter)

>Thanks, I think we really moved things forward. ?Are you able to come
>to the next sprint?

Unfortunately no, I'm not able to travel at this time. (Too busy.)

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From mwh at python.net  Tue Jun 10 13:30:30 2003
From: mwh at python.net (Michael Hudson)
Date: Tue, 10 Jun 2003 12:30:30 +0100
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
References: <09881AA6.49B3E698.9ADE5C6A@netscape.net>
Message-ID: <2mof16gofd.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> holger krekel <hpk at trillke.net> wrote:
>
>>Btw, I don't like the duplicated testsupport.py files. I think Rocco already 
>>suggested to put one version into a fixed place which i agree too. 
>>pypy.test.support seems like a canonical place. 
>
> The one problem with that is if you are sitting 
> in /norwegian/blue/pypy/interpreter/test, and '/norwegian/blue/' isn't on 
> your PYTHONPATH, you choke on trying to import pypy.test.support. One of 
> the main reasons (that I see, at least) for testsupport is that by doing a 
> relative 'import testsupport', it automagically massages your PYTHONPATH 
> such that 'from pypy...' imports work. But I agree that it probably should 
> be pared down to a stub, and it looks like someone (mwh?) has already 
> started that by moving a bunch of code to testtools.py

Yes.  I'm not sure how much more can easily be taken out of the
testsupport files; some more, probably, but not a huge amount.

> My suggestion was to move all the other files for general test support 
> (unittest_w.py, testtools.py etc) to a single directory, to leave the main 
> directory and the interpreter directory relatively uncluttered.

This is probably goodness.  There is (or, at least, I have) a tendency
to just dump stuff in pypy/interpreter when there's no obviously
better place for it.

>>Filename completions
>>on "te<tab>" should result in "test_" :-) 
>
> Could we rename it 'support_tests.py'?

If you care sufficiently, be my guest.  I don't.

Cheers,
M.

-- 
  Darn!  I've only got 10 minutes left to get falling-down drunk!  I
  suppose I'll have to smoke crack instead now.
                     -- Tim Peters is checking things in on 2002-12-31


From mwh at python.net  Tue Jun 10 13:44:03 2003
From: mwh at python.net (Michael Hudson)
Date: Tue, 10 Jun 2003 12:44:03 +0100
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
References: <5467712B.6B233B39.9ADE5C6A@netscape.net>
Message-ID: <2mllwagnss.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> Michael Hudson <mwh at python.net> wrote:
>
>>roccomoretti at netscape.net (Rocco Moretti) writes:
>
>>> Secondly, the testing framework as written will only run tests at the
>>> interpreter level. Some tests (such as test_exceptcomp and test_exec)
>>> should be run at the application level.
>>
>>Yes. ?I don't see how to easily and gracefully allow this.
>
> Naming convention? test_excptcomp_app.py and test_exec_app.py for example?

Could do.  But what would such a file look like?  I don't think pypy
can import unittest yet, and I doubt that's something we really want
to do.

>>Hmm, maybe. ?regrtest has become somewhat crufty over the years, I
>>don't think we need all of it. ?In a way, I'd prefer to keep things
>>within the unittest framework if possible 
>
> regrtest has provisions for working with unittest files, and if we want to 
> work with the CPython regression tests, we need to deal with regrtest in 
> some fashion anyway. (But I agree - regrtest is intimidating to work with.)

True.

>>(although working closely
>>with it for the first time has made its Java heritage unpleasantly
>>clear...).
>
> How so?

Using classes where a function would have done, basically.  I don't
know -- I don't know Java at all well -- but it just seems a bit
over-OOP-y.

>>We also need a LOT more tests!
>
> Like what? (Knowing what to test is 73.495% of the battle [+- 50%])

Think of something.  We probably don't test it <wink>.

Tests of app-defined classes seem to be particularly absent.

>>> And in keeping with my rearranging mood, I think main.py should
>>> probably be placed in the pypy/ directory.
>>
>>Hmm. ?Maybe, or maybe the executable that people actually run will
>>look like this:
>>
>>#!/usr/bin/python
>>from pypy.interpreter import main
>>main.main()
>
> I think: 
>
> #!/usr/bin/python
> from pypy import main
> main.main()
>
> is a bit cleaner, especially if we start to expand into the realm of 
> multiple BytecodeSpaces/ExecutionSpaces (i.e. if interpreter becomes 
> std_interpreter)

I say it's a wash.  But if you want to change it, please go right
ahead...

>>Thanks, I think we really moved things forward. ?Are you able to come
>>to the next sprint?
>
> Unfortunately no, I'm not able to travel at this time. (Too busy.)

Shame!

Cheers,
M.

-- 
  In short, just business as usual in the wacky world of floating
  point <wink>.                        -- Tim Peters, comp.lang.python


From arigo at tunes.org  Tue Jun 10 14:13:23 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 10 Jun 2003 14:13:23 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610000255.B21086@prim.han.de>
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de>
Message-ID: <20030610121323.GA491@magma.unil.ch>

Hello Holger,

On Tue, Jun 10, 2003 at 12:02:55AM +0200, holger krekel wrote:
> If we would restrict ourself to just one possible implementation of a
> type then it wouldn't be neccesary to have a W_XxxObject for each type
> but the implementation methods could be directly on the Type object? 

Well, the implementation can only be on the actual class implementing the 
object, and that is W_XxxObject. The W_XxxType class looks like a hack to me 
at the moment. I think I would prefer another way to declare types, e.g. using 
the module globals directly instead of a class with a single instance.

In this scheme the xxxtype module would be the "declaration" of the type,
whereas the xxxobject module(s) would contain the class(es) implementing it.  
If we really don't want to separate them then they could probably both be in a
single module, sure. Now whether you call this module xxxobject or xxxtype is
unclear. I think that systematically separate type with implementation is a
good idea anyway.

> I am wondering if this design comes from your psyco experience. IIRC
> PSYCO improves the common "string+=something" cases by doing the 
> list/append dance internally. Which model does PSYCO use to do this
> compared to the PyPy-approach? 

Yes, it is clearly related. Psyco uses alternate implementations internally,
e.g. for the "string+=string". Whenever I need a real CPython object to be
built however I must revert back to the standard implementation, of course. I
see something similar to be possible between PyPy and CPython-compatible
extension modules: PyPy can use whatever implementation it likes for the
objects, but as soon as one object can be seen by an extension module then it
must revert to the standard CPython-like PyObject.

> > >     def type_repr(space, w_obj):
> > >         return space.wrap("<type '%s'>" % w_obj.typename)
> 
> Any reason not name it "__name__" instead of "typename"?

Because "__name__" has a special meaning in Python classes; we expect it to be
"W_TypeObject", i.e. the name of w_obj.__class__. We could name it "tp_name"  
or anything else, but I guess "__name__" is just a bad choice (just like the
addition between wrapped objects is not "__add__" but "space.add").

> > There is no way to dispatch a multimethod based on the type of an argument,
> 
> But this could be emulated by registering all implementations of a type,
> couldn't it?

No piece of code should expect it knows all possible implementations of a
type. If this were needed, a more declarative way would be better.

> And if you register the signature 
> 
>     (W_StringObject, W_ANY)
> 
> and the signature 
> 
>     (W_ANY, W_StringObject)
> 
> for a "MultiMethod" which one gets selected? 

The rules are not "official" yet because we'll see in practice if that needs
refinement. In general they are left-to-right, yes, so an exact match for the
first argument always wins.

> I'd have thought that the "unbound methods rule" is more a restriction
> rooting in the dependence of a python method on its "fixed" data structure of 
> the object. If so, wouldn't it be more pythonic to avoid LBYL (Look
> before you leap) style of restrictions?

What Samuele and I had in mind would be to somehow tie the implementations to 
the object types they are expected to work with. We wanted to emulate 
Python's behavior of:

  1 .__add__(1.0)  -> NotImplemented
  1.0 .__add__(1)  -> 2.0

which is somehow reasonable, given that we have defined the following two
implementations: 'int+int' and 'float+float'. The second one should not be
available to 'int.__add__' because it requires the bound argument to be
converted ("delegated") to float, and 'int.__add__(1.0,1.0)' is certainly no
Python code that we'd like to see succeed.

> (trying hard to avoid to get picky about names but failing ...) why
> not use plain W_Object? W_ANY sounds like CORBA to me :-)

We came up with the equivalence 'W_ANY is W_Object' later on the sprint. They
used to be different things. I have a future redesign on the back of my head
that would change (and reduce) a lot of this boilerplate code, and which would
probably replace 'W_ANY' with 'object' actually. More about it in some later
e-mail.

> > def any_list_extend(space, w_list, w_otherlist):
> >     return space.inplace_add(w_list, w_otherlist)
> > 
> > W_ListType.list_extend.register(any_list_extend, W_ANY, W_ANY)

Not sure how to reduce this, however. I am not too fond of name-based magic.  
Function attributes would be handy but there is no nice way to define them
before the body of the function. Docstring-based hacks are bad bad bad. And
the following has already been (justifiably) rejected as not good-looking and
requiring quite some metaclass hacks:

class __magic__(W_ANY, W_ANY):
    def list_extend(space, w_list, w_otherlist):
        return space.inplace_add(w_list, w_otherlist)

One more (really magic too) alternative I could think of is

def any_list_extend(space, w_list, w_otherlist, __impl__=(W_ANY, W_ANY)):
    return space.inplace_add(w_list, w_otherlist)

Well, while we are at it, really magic things can happend and make this work:

def any_list_extend(space, w_list, w_otherlist):
    withinstance(w_list, W_ANY)
    withinstance(w_otherlist, W_ANY)
    return space.inplace_add(w_list, w_otherlist)

or

def any_list_extend(space, w_list, w_otherlist):
    implementing(W_ListType.list_extend, W_ANY, W_ANY)
    return space.inplace_add(w_list, w_otherlist)

Does it look like we are about to reinvent a type declaration system for
Python ?


A bient?t,

Armin.

From hpk at trillke.net  Tue Jun 10 15:26:36 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 10 Jun 2003 15:26:36 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610121323.GA491@magma.unil.ch>;
	from arigo@tunes.org on Tue, Jun 10, 2003 at 02:13:23PM +0200
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
Message-ID: <20030610152636.D21086@prim.han.de>

[Armin Rigo Tue, Jun 10, 2003 at 02:13:23PM +0200]

thanks for your answers. reducing the mail to one point ...

> > > def any_list_extend(space, w_list, w_otherlist):
> > >     return space.inplace_add(w_list, w_otherlist)
> > > 
> > > W_ListType.list_extend.register(any_list_extend, W_ANY, W_ANY)
> 
> Not sure how to reduce this, however. I am not too fond of name-based magic.  
> Function attributes would be handy but there is no nice way to define them
> before the body of the function. Docstring-based hacks are bad bad bad. And
> the following has already been (justifiably) rejected as not good-looking and
> requiring quite some metaclass hacks:
> 
> class __magic__(W_ANY, W_ANY):
>     def list_extend(space, w_list, w_otherlist):
>         return space.inplace_add(w_list, w_otherlist)

Evil. 

> One more (really magic too) alternative I could think of is
> 
> def any_list_extend(space, w_list, w_otherlist, __impl__=(W_ANY, W_ANY)):
>     return space.inplace_add(w_list, w_otherlist)

I was thinking more along these lines ...

    def list_extend__any_any(space, w_list, w_otherlist):
        return space.inplace_add(w_list, w_otherlist)

which is bulk-registered at the end of the file like this:

    for name, obj in vars().items():
        if hasattr(obj, 'func_code'):  # or some other selector 
            funcname, sig = name.split('__')
            getattr(space, funcname).register(obj, ssig=sig.split('_'))

cheers,

    holger

From arigo at tunes.org  Tue Jun 10 15:43:53 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 10 Jun 2003 15:43:53 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610152636.D21086@prim.han.de>
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de>
Message-ID: <20030610134353.GA5452@magma.unil.ch>

Hello Holger,

On Tue, Jun 10, 2003 at 03:26:36PM +0200, holger krekel wrote:
> I was thinking more along these lines ...
> 
>     def list_extend__any_any(space, w_list, w_otherlist):
>         return space.inplace_add(w_list, w_otherlist)

Yes, I know, this is the kind of naming magic which I really dislike I must
say. It's fine for 'any' which is a bit special anyway, but it starts to be
messy when substrings of the name must be converted into classes that might be
in other modules, or multimethod objects you'd like to register to, etc...

I'd very reluctantly admit that you could require that the relevant classes
and multimethods are to be present in the current module's globals (thus
forcing the use of 'from xxx import classes'), and will be searched only here.  
Still I don't like that. My moto is that forced naming conventions are bad,
and direct references to the objects you're interested in are good.


A bient?t,

Armin.

From hpk at trillke.net  Tue Jun 10 16:03:00 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 10 Jun 2003 16:03:00 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610134353.GA5452@magma.unil.ch>;
	from arigo@tunes.org on Tue, Jun 10, 2003 at 03:43:53PM +0200
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de> <20030610134353.GA5452@magma.unil.ch>
Message-ID: <20030610160300.G21086@prim.han.de>

Hello Armin,

[Armin Rigo Tue, Jun 10, 2003 at 03:43:53PM +0200]
> Hello Holger,
> 
> On Tue, Jun 10, 2003 at 03:26:36PM +0200, holger krekel wrote:
> > I was thinking more along these lines ...
> > 
> >     def list_extend__any_any(space, w_list, w_otherlist):
> >         return space.inplace_add(w_list, w_otherlist)
> 
> Yes, I know, this is the kind of naming magic which I really dislike I must
> say. It's fine for 'any' which is a bit special anyway, but it starts to be
> messy when substrings of the name must be converted into classes that might be
> in other modules, or multimethod objects you'd like to register to, etc...
> 
> I'd very reluctantly admit that you could require that the relevant classes
> and multimethods are to be present in the current module's globals (thus
> forcing the use of 'from xxx import classes'), and will be searched only here.  
> Still I don't like that. My moto is that forced naming conventions are bad,
> and direct references to the objects you're interested in are good.

Though i somewhat (*) agree i also think that intermingling function definitions 
and function executions is ugly, too.  Does it make sense to at least move
the register calls to the end of the file, then? Usually the func name 
and the argument names make it human-understandable enough which
type-implementations a function is supposed to handle. 

cheers,

    holger


(*) forgetting about all the special __ names ...
    these names serve a similar goal which is to hook 
    their associated methods into the python object machinery
    (i.e. __init__, __getattr__, __call__, ...). My proposed '__' 
    part of the function name was meant to "remind" of this. 

From arigo at tunes.org  Tue Jun 10 16:30:27 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 10 Jun 2003 16:30:27 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610160300.G21086@prim.han.de>
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de> <20030610134353.GA5452@magma.unil.ch>
	<20030610160300.G21086@prim.han.de>
Message-ID: <20030610143026.GC5452@magma.unil.ch>

Hello Holger,

On Tue, Jun 10, 2003 at 04:03:00PM +0200, holger krekel wrote:
> Though i somewhat (*) agree i also think that intermingling function definitions 
> and function executions is ugly, too.  Does it make sense to at least move
> the register calls to the end of the file, then? Usually the func name 
> and the argument names make it human-understandable enough which
> type-implementations a function is supposed to handle. 

This last point is actually another problem by itself, because we all tend to
forget the register() stuff after we wrote the function definition. If the
register() calls are to be moved at the end of the file it will not help.  
Somehow I find that the closer this information is to the function definition,
the better; even just after the body is somehow not close enough, hence the
search for header-based alternatives.

The suggestion pointed out by Paul:

def any_list_extend(space, w_list, w_otherlist):
    assert isinstance(w_list, W_ANY)
    assert isinstance(w_otherlist, W_ANY)
    return space.inplace_add(w_list, w_otherlist)

is closer to this goal, and hopefully "natural" enough to avoid type
declaration flame war. When I wrote the example this one is based on I forgot
that the multimethod we want to register to should be mentioned somewhere as
well -- but where ? If that's in the function name, we are back to naming
conventions. I'm not convinced about naming convention, but well, just for the
comparison:

def getitem__ListObject_SliceObject(space, w_list, w_slice):
    ...

def list_append__ListObject_object(space, w_list, w_obj):
    ...

Why not.


Armin

From hpk at trillke.net  Tue Jun 10 17:36:19 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 10 Jun 2003 17:36:19 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610143026.GC5452@magma.unil.ch>;
	from arigo@tunes.org on Tue, Jun 10, 2003 at 04:30:27PM +0200
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de> <20030610134353.GA5452@magma.unil.ch>
	<20030610160300.G21086@prim.han.de> <20030610143026.GC5452@magma.unil.ch>
Message-ID: <20030610173619.H21086@prim.han.de>

[Armin Rigo Tue, Jun 10, 2003 at 04:30:27PM +0200]
> Hello Holger,
> 
> On Tue, Jun 10, 2003 at 04:03:00PM +0200, holger krekel wrote:
> > Though i somewhat (*) agree i also think that intermingling function definitions 
> > and function executions is ugly, too.  Does it make sense to at least move
> > the register calls to the end of the file, then? Usually the func name 
> > and the argument names make it human-understandable enough which
> > type-implementations a function is supposed to handle. 
> 
> This last point is actually another problem by itself, because we all tend to
> forget the register() stuff after we wrote the function definition. If the
> register() calls are to be moved at the end of the file it will not help.  
> Somehow I find that the closer this information is to the function definition,
> the better; even just after the body is somehow not close enough, hence the
> search for header-based alternatives.
> 
> The suggestion pointed out by Paul:
> 
> def any_list_extend(space, w_list, w_otherlist):
>     assert isinstance(w_list, W_ANY)
>     assert isinstance(w_otherlist, W_ANY)
>     return space.inplace_add(w_list, w_otherlist)
> 
> is closer to this goal, and hopefully "natural" enough to avoid type
> declaration flame war. When I wrote the example this one is based on I forgot
> that the multimethod we want to register to should be mentioned somewhere as
> well -- but where ? If that's in the function name, we are back to naming
> conventions. I'm not convinced about naming convention, but well, just for the
> comparison:
> 
> def getitem__ListObject_SliceObject(space, w_list, w_slice):
>     ...
> 
> def list_append__ListObject_object(space, w_list, w_obj):
>     ...
> 
> Why not.

nice to hear :-)

Please find attached a modified "stringobject.py" where i 
implemented this convention. I found that the naming scheme
was confused sometimes and the new "enforcement" makes this
very regular and readable (IMHO, of course).

After all it's still possible to do "explicit" registrations if
there are special cases. 

cheers,

    holger
-------------- next part --------------
from pypy.objspace.std.objspace import *
from stringtype import W_StringType
from intobject   import W_IntObject
from sliceobject import W_SliceObject
from listobject import W_ListObject
from instmethobject import W_InstMethObject
from noneobject import W_NoneObject
from tupleobject import W_TupleObject

from rarray import CharArrayFromStr, CharArraySize


applicationfile = StdObjSpace.AppFile(__name__)

class W_StringObject(W_Object):
    statictype = W_StringType

    def __init__(w_self, space, str):
        W_Object.__init__(w_self, space)
        w_self._value = CharArrayFromStr(str)

    def __repr__(w_self):
        """ representation for debugging purposes """
        return "%s(%r)" % (w_self.__class__.__name__, w_self._value.value())


registerimplementation(W_StringObject)


def _isspace(ch):
    return ord(ch) in (9, 10, 11, 12, 13, 32)  

def _isdigit(ch):
    o = ord(ch)
    return o >= 48 and o <= 57

def _isalpha(ch):
    o = ord(ch)
    return (o>=97 and o<=122) or (o>=65 and o<=90)

def _isalnum(ch):
    o = ord(ch)
    return (o>=97 and o<=122) \
        or (o>=65 and o<=90) \
        or (o>=48 and o<=57)
def _isupper(ch):
    o = ord(ch)
    return (o>=65 and o<=90)

def _islower(ch):   
    o = ord(ch)
    return (o>=97 and o<=122)


def _is_generic(w_self, fun): 
    space = w_self.space   
    v = w_self._value
    if v.len == 0:
        return space.w_False
    if v.len == 1:
        c = v.charat(0)
        return space.newbool(fun(c))
    else:
        res = 1
        for idx in range(v.len):
            if not fun(v.charat(idx)):
                return space.w_False
        return space.w_True

def str_isspace__String(space, w_self):
    return _is_generic(w_self, _isspace)

def str_isdigit__String(space, w_self):
    return _is_generic(w_self, _isdigit)

def str_isalpha__String(space, w_self):
    return _is_generic(w_self, _isalpha)

def str_isalnum__String(space, w_self):
    return _is_generic(w_self, _isalnum)

def str_isupper__String(space, w_self):
    return _is_generic(w_self, _isupper)

def str_islower__String(space, w_self):
    return _is_generic(w_self, _islower)

def str_istitle(space, w_self):
    pass

def str_split__String_None(space, w_self, w_none):
    res = []
    inword = 0
    value = w_self._value.value()
    for ch in value:
        if ch.isspace():
            if inword:
                inword = 0
        else:
            if inword:
                res[-1] += ch
            else:
                res.append(ch)
                inword = 1
    for i in range(len(res)):
        res[i] = W_StringObject(space, res[i])
    return W_ListObject(space, res)

def str_split__String_String(space, w_self, w_by):
    res = []
    start = 0
    value = w_self._value.value()
    by = w_by._value.value()
    while 1:
        next = value.find(by, start)
        if next < 0:
            res.append(value[start:])
            break
        res.append(value[start:next])
        start = next + len(by)
    for i in range(len(res)):
        res[i] = W_StringObject(w_self.space, res[i])
    return W_ListObject(w_self.space, res)


def str_join__String_ANY(space, w_self, w_list):
    list = space.unpackiterable(w_list)
    if list:
        firstelem = 1
        listlen = 0
        reslen = 0  
        for w_item in list:
            reslen = reslen + w_item._value.len
            listlen = listlen + 1

        reslen = reslen + (listlen - 1) * w_self._value.len
        res = CharArraySize(reslen)

        pos = 0
        for w_item in list:
            if firstelem:
                res.setsubstring(pos, w_item._value.value())
                pos = pos + w_item._value.len 
                firstelem = 0
            else:
                res.setsubstring(pos, w_self._value.value())
                pos = pos + w_self._value.len
                res.setsubstring(pos, w_item._value.value())
                pos = pos + w_item._value.len
        return W_StringObject(space, res.value())
    else:
        return W_StringObject(space, "")

def unwrap__String(space, w_str):
    return w_str._value.value()

def is_true__String(space, w_str):
    return w_str._value.len != 0

def hash__String(space, w_str):
    return W_IntObject(space, w_str._value.hash())


EQ = 1
LE = 2
GE = 3
GT = 4
LT = 5
NE = 6

def string_richcompare(space, w_str1, w_str2, op):
    str1 = w_str1._value
    str2 = w_str2._value

    if space.is_true(space.is_(w_str1, w_str2)):
        if op == EQ or op == LE or op == GE:
            return space.w_True
        elif op == GT or op == LT or op == NE:
            return space.w_False
    if 0:
        pass
    else:
        if op == EQ:
            if str1.len == str2.len:
                for i in range(str1.len):
                    if ord(str1.charat(i)) != ord(str2.charat(i)):
                        return space.w_False
                return space.w_True
            else:
                return space.w_False
        else:
            if str1.len > str2.len:
                min_len = str2.len
            else:
                min_len = str1.len

            c = 0
            idx = 0
            if (min_len > 0):
                while (c == 0) and (idx < min_len):
                    c = ord(str1.charat(idx)) - ord(str2.charat(idx))
                    idx = idx + 1
            else:
                c = 0

        if (c == 0):
            if str1.len < str2.len:
                c = -1
            elif str1.len > str2.len:
                c = 1
            else:
                c = 0

        if op == LT:
            return space.newbool(c < 0)
        elif op == LE:
            return space.newbool(c <= 0)
        elif op == NE:
            return space.newbool(c != 0)
        elif op == GT:
            return space.newbool(c > 0)
        elif op == GE:
            return space.newbool(c >= 0)
        else:
            raise NotImplemented

def lt__String_String(space, w_str1, w_str2):
    return string_richcompare(space, w_str1, w_str2, LT)

def le__String_String(space, w_str1, w_str2):
    return string_richcompare(space, w_str1, w_str2, LE)

def eq__String_String(space, w_str1, w_str2):
    return string_richcompare(space, w_str1, w_str2, EQ)

def ne__String_String(space, w_str1, w_str2):
    return string_richcompare(space, w_str1, w_str2, NE)

def gt__String_String(space, w_str1, w_str2):
    return string_richcompare(space, w_str1, w_str2, GT)

def ge__String_String(space, w_str1, w_str2):
    return string_richcompare(space, w_str1, w_str2, GE)

def getitem__String_Int(space, w_str, w_int):
    ival = w_int.intval
    slen = w_str._value.len
    if ival < 0:
        ival += slen
    if ival < 0 or ival >= slen:
        exc = space.call_function(space.w_IndexError,
                                  space.wrap("string index out of range"))
        raise OperationError(space.w_IndexError, exc)
    return W_StringObject(space, w_str._value.charat(ival))

def getitem__String_Slice(space, w_str, w_slice):
    return space.gethelper(applicationfile).call(
        "getitem_string_slice", [w_str, w_slice])
    w = space.wrap
    u = space.unwrap
    w_start, w_stop, w_step, w_sl = w_slice.indices(w(w_str._value.len))
    start = u(w_start)
    stop = u(w_stop)
    step = u(w_step)
    sl = u(w_sl)
    r = [None] * sl
    for i in range(sl):
        r[i] = space.getitem(w_str, w(start + i*step))
    w_r = space.newlist(r)
    w_empty = space.newstring([])
    return str_join(space, w_empty, w_r)

def add__String_String(space, w_left, w_right):
    buf = CharArraySize(w_left._value.len + w_right._value.len)
    buf.setsubstring(0, w_left._value.value())
    buf.setsubstring(w_left._value.len, w_right._value.value())
    return W_StringObject(space, buf.value())

#def mod_str_ANY(space, w_left, w_right):
#    notImplemented
 
#def mod_str_tuple(space, w_format, w_args):
#    notImplemented

def len__String(space, w_str):
    return space.wrap(w_str._value.len)

def str__String(space, w_str):
    return w_str

def repr__String(space, w_str):
    # XXX this is bogus -- mwh
    return space.wrap(repr(space.unwrap(w_str)))
    a = space.add
    q = space.wrap("'")
    return a(a(q, w_str), q)

def ord__String(space, w_str):
    return space.wrap(ord(space.unwrap(w_str)))

def mod__String_ANY(space, w_str, w_item):
    return str_mod_tuple(space, w_str, space.newtuple([w_item]))

def mod__String_Tuple(space, w_str, w_tuple):
    return space.wrap(space.unwrap(w_str)%space.unwrap(w_tuple))


# register all methods 
for name, obj in vars().items():
        try: 
            funcname, sig = name.split('__')
        except ValueError: 
            continue
        l=[]
        for i in sig.split('_'):
            i = i=='ANY' and W_ANY or vars()['W_%sObject' % i]
            l.append(i)
        try:
            getattr(StdObjSpace, funcname).register(obj, *l)
        except AttributeError:
            getattr(W_StringType, funcname).register(obj, *l)


From arigo at tunes.org  Thu Jun 12 13:18:37 2003
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 12 Jun 2003 13:18:37 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030610173619.H21086@prim.han.de>
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de> <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de> <20030610134353.GA5452@magma.unil.ch>
	<20030610160300.G21086@prim.han.de> <20030610143026.GC5452@magma.unil.ch>
	<20030610173619.H21086@prim.han.de>
Message-ID: <20030612111837.GA11354@magma.unil.ch>

Hello Holger,

On Tue, Jun 10, 2003 at 05:36:19PM +0200, holger krekel wrote:
> Please find attached a modified "stringobject.py" where i 
> implemented this convention. I found that the naming scheme
> was confused sometimes and the new "enforcement" makes this
> very regular and readable (IMHO, of course).

It seems pretty reasonable, to say the least. What do other people think about 
this point ?

If accepted, the introspection-registering loop at the end of the file should
probably be moved in a common location and called from the end of each
xxxobject.py file.


A bient?t,

Armin.

From mwh at python.net  Thu Jun 12 13:53:36 2003
From: mwh at python.net (Michael Hudson)
Date: Thu, 12 Jun 2003 12:53:36 +0100
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
References: <20030605153259.GA14596@magma.unil.ch>
	<20030607085151.O9377@prim.han.de>
	<20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de>
	<20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de>
	<20030610134353.GA5452@magma.unil.ch>
	<20030610160300.G21086@prim.han.de>
	<20030610143026.GC5452@magma.unil.ch>
	<20030610173619.H21086@prim.han.de>
	<20030612111837.GA11354@magma.unil.ch>
Message-ID: <2mn0gn5x6n.fsf@starship.python.net>

Armin Rigo <arigo at tunes.org> writes:

> Hello Holger,
>
> On Tue, Jun 10, 2003 at 05:36:19PM +0200, holger krekel wrote:
>> Please find attached a modified "stringobject.py" where i 
>> implemented this convention. I found that the naming scheme
>> was confused sometimes and the new "enforcement" makes this
>> very regular and readable (IMHO, of course).
>
> It seems pretty reasonable, to say the least. What do other people think about 
> this point ?

+0.a bit

> If accepted, the introspection-registering loop at the end of the file should
> probably be moved in a common location and called from the end of each
> xxxobject.py file.

But for heavens sake, please leave an explicit call at the end of each module!

StdObjSpace.register_from_namespace(globals())

or something.

Am I right in thinking that for each object space method we have one
multimethod instance shared between all instances of StdObjSpace?
That's not a very good idea, is it?

Cheers,
M.

-- 
  incidentally, asking why things are "left out of the language" is
  a good sign that the asker is fairly clueless.
                                        -- Erik Naggum, comp.lang.lisp


From mwh at python.net  Thu Jun 12 14:10:22 2003
From: mwh at python.net (Michael Hudson)
Date: Thu, 12 Jun 2003 13:10:22 +0100
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
References: <5467712B.6B233B39.9ADE5C6A@netscape.net>
	<2mllwagnss.fsf@starship.python.net>
Message-ID: <2mk7br5wep.fsf@starship.python.net>

Michael Hudson <mwh at python.net> writes:

> roccomoretti at netscape.net (Rocco Moretti) writes:
>
>> Michael Hudson <mwh at python.net> wrote:
>>
>>>roccomoretti at netscape.net (Rocco Moretti) writes:
>>
>>>> Secondly, the testing framework as written will only run tests at the
>>>> interpreter level. Some tests (such as test_exceptcomp and test_exec)
>>>> should be run at the application level.
>>>
>>>Yes. ?I don't see how to easily and gracefully allow this.
>>
>> Naming convention? test_excptcomp_app.py and test_exec_app.py for example?
>
> Could do.  But what would such a file look like?  I don't think pypy
> can import unittest yet, and I doubt that's something we really want
> to do.

I've had another idea: give unittest_w.TestCase_w a metaclass that
looks for methods called 'test_app_whatever' and arrange for such
methods to be called as objectspace methods.

So, e.g. 

    def test_trivial(self):
        x = self.codetest('''
def g(): return 42''', 'g', [])
        self.assertEquals(x, 42)

becomes:

    def test_app_trivial(self):
        def g():
            return 42
        self.assertEquals(g(), 42)

I have an implementation of this (attached), but it was significantly
more of a pain than I would have liked (mostly because 2.2's unittest
doesn't cope with TestCase classes being new-style...).

It also only works in the standard object space, and only really works
there by fluke (it's the self.assertEquals business that casues the
problems -- can probably try harder and come up with a better solution).

Cheers,
M.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: app-space-tests.diff
Type: text/x-patch
Size: 10635 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030612/156c285d/attachment.bin>
-------------- next part --------------

-- 
  And then the character-only displays went away (leading to
  increasingly silly graphical effects and finally to ads on
  web pages).                      -- John W. Baxter, comp.lang.python

From arigo at tunes.org  Thu Jun 12 14:26:22 2003
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 12 Jun 2003 14:26:22 +0200
Subject: [pypy-dev] W_XxxObjects
Message-ID: <20030612122622.GB11354@magma.unil.ch>

Hello !

I have slowly come to think that wrapping every object we want to see at the 
application-level in a W_XxxObject class is conceptually a good idea but not 
necessarily pratically required. I used to think that it would be useful for 
the translator-to-C. Now I'm no longer convinced. Contrast these two 
implementations of integer comparison :

def int_int_lt(space, w_int1, w_int2):
    i = w_int1.intval
    j = w_int2.intval
    return space.newbool( i < j )

vs.

def lt__int_int(space, i, j):
    return bool(i < j)

It seems to me that the first implementation is a typical example of the 
boilerplate kind of code that we hoped the project would remove, isn't it?

How I see things right now would be that any type (or class) of restricted
Python would be suitable for implementing Python objects. For complex objects
nothing changes:  we still need classes acting as structures, like a
W_ListObject with the over-allocation machinery fields. But the most simple
objects could be implemented just directly with simple interpreter-level
objects instead of requiring them to be classes W_XxxObject.

This idea comes from the fact that over the time the W_XxxObject class
definitions themselves have become more and more skeletical, with all
interesting functions defined as multimethods outside the class.  The only
relevant piece of information left in the W_XxxObject classes today is the
'statictype' attribute, which would be easily to implement in some other way.

Yes I know I was the guy who pushed for explicit W_XxxObjects in the first 
place :-)

This simplification could also avoid some problems with objects that are not
properly wrapped, because the simple interpreter-level objects don't need any
explicit wrapping any more. Actually "wrapping" may become a transparent
operation: the translator-to-C will detect the need automatically. Example: if
there is an (interpreter-level) variable 'x' containing an integer, and if at
some point we use it in a call to a function that has been found to expect a
wrapped object ( = any one of the available implementation of objects) then we
must add some tag that says the int is really an int (as opposed to any of the
other implementations of objects). The translator will have to do this kind of
operations anyway for a lot of other cases. It just seems to me that it can
do it pretty well for this case too. Wrapping would now be automatical.

It would make the PyPy code slightly less strictly-typed, i.e. more Pythonic
:-)  and as we will need the translator at some point (probably soon) we may
rest confident that typing bugs will eventually be caught by this analysis
phase.

In other words I am suggesting that we can simply use the basic types we
allowed in restricted python to implement some of the xxxobject.py:

* intobject.py  -  int
(ints could be forced to r_ints before the operations for which it matter, 
e.g. "def lshift__int_int(self, i, j): ... return r_int(i) << j")

* stringobject.py  -  str
(this makes sense, as it enforces strings to be immutable once they're 
built; for mutable strings (e.g. during building) we can use rarray.CharArray 
or maybe just lists of characters?)

* boolobject.py  -  bool
(in Python 2.2 we could actually define a bool class, as long as we write 
"def lt__int_int(space, i, j): return bool(i < j)" with the explicit bool())

* floatobject.py  -  float

* noneobject.py  -  guess

* tupleobject.py  -  list
(this is just a suggestion, a list being in restricted python the equivalent 
of allocated memory.)


A bient?t,

Armin.

From arigo at tunes.org  Thu Jun 12 14:30:03 2003
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 12 Jun 2003 14:30:03 +0200
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
In-Reply-To: <2mn0gn5x6n.fsf@starship.python.net>
References: <20030607130050.GA19854@magma.unil.ch>
	<20030610000255.B21086@prim.han.de> <20030610121323.GA491@magma.unil.ch>
	<20030610152636.D21086@prim.han.de> <20030610134353.GA5452@magma.unil.ch>
	<20030610160300.G21086@prim.han.de> <20030610143026.GC5452@magma.unil.ch>
	<20030610173619.H21086@prim.han.de> <20030612111837.GA11354@magma.unil.ch>
	<2mn0gn5x6n.fsf@starship.python.net>
Message-ID: <20030612123003.GC11354@magma.unil.ch>

Hello Michael,

On Thu, Jun 12, 2003 at 12:53:36PM +0100, Michael Hudson wrote:
> But for heavens sake, please leave an explicit call at the end of each module!

Sure !

> Am I right in thinking that for each object space method we have one
> multimethod instance shared between all instances of StdObjSpace?
> That's not a very good idea, is it?

Multimethod tables are essentially static, so it doesn't matter if several
object spaces run in parallel. User-defined types don't actually cause
anything to be added to the multimethods; there is in essence just one
*implementation* for all user-defined objects, and that is W_UserObject.


A bient?t,

Armin.

From arigo at tunes.org  Thu Jun 12 14:42:01 2003
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 12 Jun 2003 14:42:01 +0200
Subject: [pypy-dev] Re: deep breakage
In-Reply-To: <200306091424.h59EOaW04165@odiug.zope.com>
References: <20030609100002.26FE45BFF7@thoth.codespeak.net>
	<200306091424.h59EOaW04165@odiug.zope.com>
Message-ID: <20030612124201.GD11354@magma.unil.ch>

Hello Guido,

I'm just wondering: as we are heading towards Exception being a new-style 
class (no old-style classes being supported currently) I can see a very 
theoretical problem which must already have been discussed in python-dev (has 
it?):

   raise x

should be "raise x()" if issubclass(x, Exception) and "raise x" if 
isinstance(x, Exception). What if:

>>> class exception(object): pass
... 
>>> class f(type, exception): pass
... 
>>> x = f('x', (exception,), {})
>>> isinstance(x, exception)
1
>>> issubclass(x, exception)
1

Ok that's kind of a joke :-)

Armin

From lac at strakt.com  Thu Jun 12 14:44:06 2003
From: lac at strakt.com (Laura Creighton)
Date: Thu, 12 Jun 2003 14:44:06 +0200
Subject: [pypy-dev] OSCON paper
Message-ID: <200306121244.h5CCi6qW000854@ratthing-b246.strakt.com>

What do you all suggest I report?  Due Wednesday, but will of course be
revised to cover what we do between now and OSCON (July 7-11).

Laura

From guido at python.org  Thu Jun 12 17:04:03 2003
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2003 11:04:03 -0400
Subject: [pypy-dev] Re: deep breakage
In-Reply-To: Your message of "Thu, 12 Jun 2003 14:42:01 +0200."
             <20030612124201.GD11354@magma.unil.ch> 
References: <20030609100002.26FE45BFF7@thoth.codespeak.net>
	<200306091424.h59EOaW04165@odiug.zope.com>  
	<20030612124201.GD11354@magma.unil.ch> 
Message-ID: <200306121504.h5CF43x11585@odiug.zope.com>

> Hello Guido,
> 
> I'm just wondering: as we are heading towards Exception being a new-style 
> class (no old-style classes being supported currently) I can see a very 
> theoretical problem which must already have been discussed in python-dev (has 
> it?):
> 
>    raise x
> 
> should be "raise x()" if issubclass(x, Exception) and "raise x" if 
> isinstance(x, Exception). What if:
> 
> >>> class exception(object): pass
> ... 
> >>> class f(type, exception): pass
> ... 
> >>> x = f('x', (exception,), {})
> >>> isinstance(x, exception)
> 1
> >>> issubclass(x, exception)
> 1
> 
> Ok that's kind of a joke :-)
> 
> Armin

I suggest only calling it if it's not already an instance of
Exception.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From roccomoretti at netscape.net  Fri Jun 13 05:34:05 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Thu, 12 Jun 2003 23:34:05 -0400
Subject: [pypy-dev] Objects and types in the stdobjspace
Message-ID: <20CB5902.4FA28EFB.9ADE5C6A@netscape.net>

Michael Hudson <mwh at python.net> wrote:

>I've had another idea: give unittest_w.TestCase_w a metaclass that
>looks for methods called 'test_app_whatever' and arrange for such
>methods to be called as objectspace methods.

Sounds like a wonderful solution, but I have to confess that my eyes glaze 
over when trying to understand metaclasses - I probably should put more of 
an effort into learning about them one of these days...

Is using metaclasses really nessasary? Before I realized that metaclasses 
were involved, I first thought of using a special subclass of TestCase - 
probably by overloading TestCase.__call__() to appropriately wrap the 
function before it is run.

If we can use the subclass approach, we could potentially simplify things 
by having two seperate TestCase subclasses, one for interpreter level and 
one for application level tests. This would potentially require splitting 
of a single test case into two, one each for application and interpreter 
levels, whereas your implementation appears to allow both types of tests 
in the same class definition.

But feel free to ignore this as I haven't tried putting it into code and 
have no clue what the problems with it may be.

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From mwh at python.net  Fri Jun 13 13:01:45 2003
From: mwh at python.net (Michael Hudson)
Date: Fri, 13 Jun 2003 12:01:45 +0100
Subject: [pypy-dev] Re: Objects and types in the stdobjspace
References: <20CB5902.4FA28EFB.9ADE5C6A@netscape.net>
Message-ID: <2m4r2ugs12.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> Michael Hudson <mwh at python.net> wrote:
>
>>I've had another idea: give unittest_w.TestCase_w a metaclass that
>>looks for methods called 'test_app_whatever' and arrange for such
>>methods to be called as objectspace methods.
>
> Sounds like a wonderful solution, but I have to confess that my eyes glaze 
> over when trying to understand metaclasses - I probably should put more of 
> an effort into learning about them one of these days...

I should probably be worried at the willingness with which I now reach
for them.

> Is using metaclasses really nessasary?

Heh.  No, probably not.

> Before I realized that metaclasses were involved, I first thought of
> using a special subclass of TestCase - probably by overloading
> TestCase.__call__() to appropriately wrap the function before it is
> run.

Yes, that would probably work.  It would also mean that we wouldn't
have to make testsupport.TestCase new-style which would mean we
wouldn't have to steal 2.3's TestLoader, which would be a good thing.

> If we can use the subclass approach, we could potentially simplify
> things by having two seperate TestCase subclasses, one for
> interpreter level and one for application level tests.

Yes, that would be one approach.

> This would potentially require splitting of a single test case into
> two, one each for application and interpreter levels, whereas your
> implementation appears to allow both types of tests in the same
> class definition.

Yes it does.  The subclassing approach could allow that, but doesn't
have to.  I don't feel that splitting the tests up would necessarily
be a bad thing, either.

> But feel free to ignore this as I haven't tried putting it into code
> and have no clue what the problems with it may be.

No, you've probably done a sensible job of restraining my obfuscatory
tendencies!

Cheers,
M.

-- 
  Or here's an even simpler indicator of how much C++ sucks: Print
  out the C++ Public Review Document.  Have someone  hold it about
  three feet  above your head and then drop it.  Thus  you will be
  enlightened.                                        -- Thant Tessman


From arigo at tunes.org  Fri Jun 13 16:00:40 2003
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 13 Jun 2003 16:00:40 +0200
Subject: [pypy-dev] dis.dis(dis.dis)
Message-ID: <20030613140040.GA20516@magma.unil.ch>

Hello everybody,

The profiling data Michael posted to this list is a result of some work we did 
to get dis-goal.py working (i.e. dis.dis(dis.dis)).

It still doesn't work, but a pre-version does, checked in in dis-pregoal.py, 
which reads:

    dis.disassemble(dis.dis.func_code)

which takes a good few minutes to complete :-)


Armin

From mwh at python.net  Fri Jun 13 15:51:29 2003
From: mwh at python.net (Michael Hudson)
Date: Fri, 13 Jun 2003 14:51:29 +0100
Subject: [pypy-dev] Profiling results
Message-ID: <2m1xxygk66.fsf@starship.python.net>

Armin and I are on IRC talking about mhy my new types module takes so
long to load and this is a bit big for that medium :-)

>>> profile.run("main.run_string('import types', '', space)")
the slice is empty
         1167131 function calls (876696 primitive calls) in 36.760 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   36.760   36.760 <string>:1(?)
        4    0.000    0.000    0.000    0.000 appfile.py:38(__init__)
       44    0.000    0.000    0.380    0.009 appfile.py:45(get)
     44/1    0.000    0.000   36.680   36.680 appfile.py:51(call)
        4    0.000    0.000    0.090    0.022 appfile.py:58(runbytecode)
        4    0.000    0.000    0.090    0.022 appfile.py:73(__init__)
      208    0.060    0.000    0.180    0.001 baseobjspace.py:108(unpacktuple)
      706    0.040    0.000    0.720    0.001 baseobjspace.py:122(exception_match)
        2    0.000    0.000    0.020    0.010 baseobjspace.py:154(call_function)
        3    0.000    0.000    0.000    0.000 baseobjspace.py:37(get_builtin_module)
      101    0.040    0.000    0.040    0.000 baseobjspace.py:49(getexecutioncontext)
       44    0.010    0.000    0.100    0.002 baseobjspace.py:61(gethelper)
    22094    2.340    0.000   16.390    0.001 baseobjspace.py:74(is_)
    22362    0.180    0.000    0.180    0.000 baseobjspace.py:80(newbool)
      197    0.060    0.000    0.610    0.003 baseobjspace.py:86(unpackiterable)
    44367    0.350    0.000    0.350    0.000 boolobject.py:27(bool_is_true)
       31    0.000    0.000    0.010    0.000 builtin.py:103(len)
        1    0.000    0.000    0.010    0.010 builtin.py:112(getattr)
       58    0.000    0.000    0.080    0.001 builtin.py:139(issubclass)
      3/1    0.000    0.000   36.600   36.600 builtin.py:36(__import__)
        3    0.000    0.000    0.000    0.000 builtin.py:71(compile)
       32    0.000    0.000    0.000    0.000 cpythonobject.py:131(cpython_f)
       56    0.000    0.000    0.040    0.001 cpythonobject.py:139(cpython_f)
      121    0.000    0.000    0.000    0.000 cpythonobject.py:20(cpython_unwrap)
       52    0.000    0.000    0.000    0.000 cpythonobject.py:8(__init__)
    44158    1.780    0.000    3.480    0.000 default.py:37(default_id)
        7    0.000    0.000    0.010    0.001 default.py:46(default_not)
       38    0.000    0.000    0.010    0.000 default.py:64(default_inplace)
      810    0.010    0.000    0.010    0.000 dictobject.py:10(__init__)
        8    0.000    0.000    0.000    0.000 dictobject.py:109(len_dict)
     1783    0.030    0.000    0.040    0.000 dictobject.py:13(get)
      452    0.010    0.000    0.010    0.000 dictobject.py:18(set)
        7    0.000    0.000    0.000    0.000 dictobject.py:21(make_empty)
    42721    0.270    0.000    0.270    0.000 dictobject.py:26(is_empty)
      475    0.080    0.000    0.270    0.001 dictobject.py:38(__init__)
     2436    0.580    0.000    0.840    0.000 dictobject.py:47(non_empties)
      452    0.040    0.000    0.350    0.001 dictobject.py:51(_cell)
      240    0.010    0.000    0.010    0.000 dictobject.py:71(dict_is_true)
     2181    1.350    0.001   30.100    0.014 dictobject.py:84(getitem_dict_any)
      452    0.030    0.000    0.390    0.001 dictobject.py:93(setitem_dict_any_any)
        7    0.000    0.000    0.170    0.024 dictobject.py:99(delitem_dict_any)
        4    0.000    0.000    0.000    0.000 executioncontext.py:18(get_w_builtins)
      197    0.000    0.000    0.000    0.000 executioncontext.py:202(__init__)
     2948    0.030    0.000    0.030    0.000 executioncontext.py:205(push)
     2948    0.030    0.000    0.030    0.000 executioncontext.py:208(pop)
      434    0.000    0.000    0.000    0.000 executioncontext.py:211(top)
      110    0.000    0.000    0.000    0.000 executioncontext.py:216(depth)
      142    0.010    0.000    0.010    0.000 executioncontext.py:219(empty)
        4    0.000    0.000    0.000    0.000 executioncontext.py:24(make_standard_w_globals)
        8    0.000    0.000    0.000    0.000 executioncontext.py:31(exception_trace)
        1    0.000    0.000    0.000    0.000 executioncontext.py:5(__init__)
      804    0.010    0.000    0.010    0.000 executioncontext.py:56(__init__)
        8    0.000    0.000    0.000    0.000 executioncontext.py:62(record_application_traceback)
      698    0.050    0.000    0.890    0.001 executioncontext.py:65(match)
        8    0.000    0.000    0.000    0.000 executioncontext.py:88(record_interpreter_traceback)
     98/1    0.010    0.000   36.730   36.730 executioncontext.py:9(eval_frame)
     93/1    0.010    0.000   36.600   36.600 extmodule.py:40(eval_code)
        3    0.000    0.000    0.000    0.000 extmodule.py:7(__get__)
       62    0.000    0.000    0.000    0.000 funcobject.py:11(__init__)
    215/1    0.010    0.000   36.670   36.670 funcobject.py:35(func_call)
       44    0.000    0.000    0.010    0.000 funcobject.py:47(func_get)
        1    0.000    0.000    0.010    0.010 funcobject.py:52(func_getattr)
    42/25    0.000    0.000   14.460    0.578 instmethobject.py:31(instmeth_call)
       44    0.010    0.000    0.010    0.000 instmethobject.py:9(__init__)
       31    0.000    0.000    0.000    0.000 intobject.py:102(int_int_gt)
       15    0.000    0.000    0.000    0.000 intobject.py:108(int_int_ge)
       61    0.010    0.000    0.010    0.000 intobject.py:141(int_int_add)
       46    0.010    0.000    0.010    0.000 intobject.py:153(int_int_sub)
       23    0.000    0.000    0.000    0.000 intobject.py:177(int_int_floordiv)
    46676    1.490    0.000    1.810    0.000 intobject.py:26(__init__)
       39    0.000    0.000    0.000    0.000 intobject.py:323(int_is_true)
       16    0.000    0.000    0.000    0.000 intobject.py:385(int_int_and)
     1167    0.010    0.000    0.010    0.000 intobject.py:46(int_unwrap)
      114    0.000    0.000    0.000    0.000 intobject.py:78(int_int_lt)
       23    0.000    0.000    0.000    0.000 intobject.py:84(int_int_le)
    22102    0.370    0.000    0.550    0.000 intobject.py:90(int_int_eq)
      393    0.010    0.000    0.470    0.001 iterobject.py:22(next_seqiter)
      205    0.000    0.000    0.000    0.000 iterobject.py:8(__init__)
        8    0.000    0.000    0.000    0.000 listobject.py:109(list_int_mul)
       20    0.000    0.000    0.000    0.000 listobject.py:13(__init__)
       15    0.000    0.000    0.000    0.000 listobject.py:166(setitem_list_int)
       19    0.000    0.000    0.000    0.000 listobject.py:211(_roundupsize)
       29    0.000    0.000    0.000    0.000 listobject.py:260(_list_resize)
        2    0.000    0.000    0.010    0.005 listobject.py:35(list_unwrap)
       16    0.000    0.000    0.000    0.000 listobject.py:52(getitem_list_int)
        8    0.000    0.000    0.000    0.000 listobject.py:84(list_iter)
        1    0.000    0.000    0.000    0.000 listobject.py:90(list_add)
        1    0.000    0.000   36.760   36.760 main.py:7(run_string)
        7    0.000    0.000    0.060    0.009 moduleobject.py:21(getattr_module_any)
        2    0.000    0.000    0.000    0.000 moduleobject.py:33(setattr_module_any_any)
        3    0.000    0.000    0.010    0.003 moduleobject.py:9(__init__)
       37    0.000    0.000    0.000    0.000 multimethod.py:103(is_empty)
   140658    1.070    0.000    1.070    0.000 multimethod.py:109(__init__)
140654/53    6.020    0.000   36.720    0.693 multimethod.py:113(__call__)
        4    0.000    0.000    0.000    0.000 multimethod.py:12(__init__)
140685/53    9.800    0.000   36.720    0.693 multimethod.py:136(perform_call)
       33    0.000    0.000    0.000    0.000 multimethod.py:158(is_empty)
      116    0.000    0.000    0.000    0.000 multimethod.py:24(register)
   140788    3.330    0.000    4.400    0.000 multimethod.py:36(__get__)
        4    0.000    0.000    0.000    0.000 multimethod.py:41(cache_dependency)
   140685    1.380    0.000    1.560    0.000 multimethod.py:44(buildchoices)
 3065/134    0.180    0.000    0.180    0.001 multimethod.py:56(internal_buildchoices)
        4    0.000    0.000    0.000    0.000 multimethod.py:96(slicetable)
      173    0.000    0.000    0.030    0.000 objectobject.py:120(object_type)
      164    0.060    0.000    0.770    0.005 objectobject.py:28(object_getattr)
       17    0.000    0.000    0.070    0.004 objectobject.py:74(object_setattr)
       12    0.000    0.000    0.000    0.000 objecttype.py:14(objecttype_new)
    52116    0.380    0.000    0.380    0.000 objspace.py:16(__init__)
      389    0.010    0.000    0.030    0.000 objspace.py:173(get_typeinstance)
5405/5372    0.280    0.000    0.830    0.000 objspace.py:183(wrap)
       79    0.000    0.000    0.000    0.000 objspace.py:19(get_builtin_impl_class)
      378    0.040    0.000    0.060    0.000 objspace.py:215(newtuple)
       11    0.000    0.000    0.000    0.000 objspace.py:219(newlist)
      472    0.040    0.000    0.310    0.001 objspace.py:223(newdict)
       16    0.000    0.000    0.000    0.000 objspace.py:227(newslice)
       62    0.000    0.000    0.000    0.000 objspace.py:232(newfunction)
        3    0.000    0.000    0.010    0.003 objspace.py:237(newmodule)
        1    0.000    0.000    0.000    0.000 objspace.py:26(registerimplementation)
        7    0.000    0.000    0.010    0.001 opcode.py:14(__call__)
       15    0.000    0.000    4.590    0.306 opcode.py:140(slice)
        7    0.000    0.000    2.060    0.294 opcode.py:149(SLICE_1)
        8    0.000    0.000    2.530    0.316 opcode.py:153(SLICE_2)
       31    0.000    0.000    0.040    0.001 opcode.py:205(STORE_SUBSCR)
      190    0.010    0.000    0.120    0.001 opcode.py:23(__call__)
        7    0.000    0.000    0.000    0.000 opcode.py:238(BREAK_LOOP)
        1    0.000    0.000    3.780    3.780 opcode.py:244(RAISE_VARARGS)
        2    0.000    0.000    0.000    0.000 opcode.py:271(LOAD_LOCALS)
       97    0.000    0.000    0.000    0.000 opcode.py:274(RETURN_VALUE)
        1    0.000    0.000    0.000    0.000 opcode.py:278(YIELD_VALUE)
       14    0.000    0.000    0.000    0.000 opcode.py:297(POP_BLOCK)
        2    0.000    0.000    0.020    0.010 opcode.py:313(BUILD_CLASS)
       69    0.000    0.000    0.140    0.002 opcode.py:322(STORE_NAME)
        7    0.000    0.000    0.170    0.024 opcode.py:328(DELETE_NAME)
     1123    0.110    0.000    8.770    0.008 opcode.py:35(LOAD_FAST)
       17    0.000    0.000    0.710    0.042 opcode.py:350(STORE_ATTR)
       60    0.000    0.000    6.860    0.114 opcode.py:376(LOAD_NAME)
      254    0.020    0.000   10.160    0.040 opcode.py:406(LOAD_GLOBAL)
       49    0.000    0.000    0.010    0.000 opcode.py:477(BUILD_TUPLE)
      350    0.030    0.000    0.070    0.000 opcode.py:48(LOAD_CONST)
       11    0.010    0.001    0.010    0.001 opcode.py:483(BUILD_LIST)
        8    0.000    0.000    0.000    0.000 opcode.py:489(BUILD_MAP)
      156    0.020    0.000    0.610    0.004 opcode.py:495(LOAD_ATTR)
      114    0.020    0.000    0.060    0.001 opcode.py:503(cmp_lt)
       23    0.000    0.000    0.000    0.000 opcode.py:504(cmp_le)
        8    0.000    0.000    0.000    0.000 opcode.py:505(cmp_eq)
       31    0.010    0.000    0.010    0.000 opcode.py:507(cmp_gt)
       15    0.000    0.000    0.000    0.000 opcode.py:508(cmp_ge)
       62    0.000    0.000    0.010    0.000 opcode.py:514(cmp_is)
        8    0.000    0.000    0.000    0.000 opcode.py:518(cmp_exc_match)
      344    0.020    0.000    0.460    0.001 opcode.py:52(STORE_FAST)
      261    0.000    0.000    0.090    0.000 opcode.py:534(COMPARE_OP)
      3/1    0.000    0.000   36.730   36.730 opcode.py:544(IMPORT_NAME)
        1    0.000    0.000    0.100    0.100 opcode.py:557(IMPORT_FROM)
      104    0.000    0.000    0.000    0.000 opcode.py:564(JUMP_FORWARD)
      374    0.070    0.000    0.180    0.000 opcode.py:567(JUMP_IF_FALSE)
       15    0.000    0.000    0.000    0.000 opcode.py:572(JUMP_IF_TRUE)
       23    0.000    0.000    0.000    0.000 opcode.py:577(JUMP_ABSOLUTE)
      391    0.010    0.000    0.010    0.000 opcode.py:58(POP_TOP)
        8    0.000    0.000    0.000    0.000 opcode.py:580(GET_ITER)
       16    0.000    0.000    0.000    0.000 opcode.py:585(FOR_ITER)
       17    0.000    0.000    0.000    0.000 opcode.py:599(SETUP_LOOP)
       38    0.000    0.000    0.000    0.000 opcode.py:603(SETUP_EXCEPT)
    164/1    0.010    0.000   36.610   36.610 opcode.py:611(call_function_extra)
    156/1    0.000    0.000   36.610   36.610 opcode.py:637(CALL_FUNCTION)
        8    0.000    0.000   15.470    1.934 opcode.py:640(CALL_FUNCTION_VAR)
       30    0.010    0.000    0.030    0.001 opcode.py:649(MAKE_FUNCTION)
       15    0.000    0.000    0.000    0.000 opcode.py:67(ROT_THREE)
     1040    0.000    0.000    0.000    0.000 opcode.py:684(SET_LINENO)
     5339    0.050    0.000    0.050    0.000 opcode.py:715(has_arg)
  804/577    0.020    0.000    4.780    0.008 opcode.py:718(dispatch_noarg)
   4535/6    0.140    0.000   36.730    6.122 opcode.py:726(dispatch_arg)
       24    0.010    0.000    0.010    0.000 opcode.py:85(DUP_TOP)
        6    0.000    0.000    0.000    0.000 posixpath.py:168(exists)
        6    0.000    0.000    0.000    0.000 posixpath.py:44(join)
        1    0.000    0.000   36.760   36.760 profile:0(main.run_string('import types', '', space))
        0    0.000             0.000          profile:0(profiler)
     93/1    0.000    0.000   36.660   36.660 pycode.py:113(eval_code)
      493    0.000    0.000    0.000    0.000 pycode.py:29(__init__)
  215/153    0.080    0.000   13.660    0.089 pycode.py:37(build_arguments)
       37    0.000    0.000    0.000    0.000 pycode.py:76(__init__)
     37/7    0.000    0.000    0.000    0.000 pycode.py:91(_from_code)
       55    0.000    0.000    0.000    0.000 pyframe.py:140(__init__)
       55    0.000    0.000    0.000    0.000 pyframe.py:144(cleanupstack)
       14    0.000    0.000    0.000    0.000 pyframe.py:148(cleanup)
        7    0.000    0.000    0.000    0.000 pyframe.py:160(unroll)
    34/12    0.000    0.000   24.180    2.015 pyframe.py:179(unroll)
       98    0.000    0.000    4.690    0.048 pyframe.py:22(__init__)
   112/30    0.010    0.000   24.180    0.806 pyframe.py:250(unrollstack)
       97    0.000    0.000    0.000    0.000 pyframe.py:283(emptystack)
        1    0.000    0.000    0.000    0.000 pyframe.py:291(unrollstack)
     98/1    0.200    0.002   36.730   36.730 pyframe.py:33(eval)
    14409    0.220    0.000    0.220    0.000 pyframe.py:70(nextop)
     4535    0.180    0.000    0.290    0.000 pyframe.py:75(nextarg)
      350    0.000    0.000    0.000    0.000 pyframe.py:80(getconstant)
     1467    0.010    0.000    0.010    0.000 pyframe.py:83(getlocalvarname)
      567    0.010    0.000    0.010    0.000 pyframe.py:86(getname)
       98    0.010    0.000    4.690    0.048 pyframe.py:99(load_builtins)
     4053    0.040    0.000    0.040    0.000 rarray.py:19(__init__)
     4053    0.150    0.000    0.150    0.000 rarray.py:22(setvalue)
      910    0.040    0.000    0.040    0.000 rarray.py:30(hash)
     1239    0.000    0.000    0.000    0.000 rarray.py:35(value)
    27946    0.210    0.000    0.210    0.000 rarray.py:43(charat)
     4053    0.180    0.000    0.370    0.000 rarray.py:8(CharArrayFromStr)
       16    0.000    0.000    0.000    0.000 restricted_int.py:104(__and__)
       71    0.000    0.000    0.000    0.000 restricted_int.py:158(__new__)
       30    0.000    0.000    0.000    0.000 restricted_int.py:161(__add__)
       11    0.000    0.000    0.000    0.000 restricted_int.py:177(__mul__)
       11    0.000    0.000    0.000    0.000 restricted_int.py:215(__lshift__)
       84    0.000    0.000    0.000    0.000 restricted_int.py:22(__add__)
       11    0.000    0.000    0.000    0.000 restricted_int.py:230(__rrshift__)
       46    0.000    0.000    0.000    0.000 restricted_int.py:28(__sub__)
       31    0.000    0.000    0.000    0.000 restricted_int.py:38(__mul__)
       23    0.000    0.000    0.000    0.000 restricted_int.py:46(__div__)
        8    0.000    0.000    0.000    0.000 restricted_int.py:76(__lshift__)
       24    0.000    0.000    0.000    0.000 restricted_int.py:88(__rshift__)
       16    0.000    0.000    0.000    0.000 sliceobject.py:13(__init__)
       15    0.000    0.000    4.550    0.303 sliceobject.py:18(indices)
       45    0.030    0.001    0.180    0.004 sliceobject.py:29(getattr_slice_any)
        1    0.000    0.000    0.010    0.010 slicetype.py:9(slicetype_new)
      329    0.010    0.000    0.010    0.000 stringobject.py:188(str_unwrap)
     4053    0.100    0.000    0.500    0.000 stringobject.py:19(__init__)
      910    0.030    0.000    0.110    0.000 stringobject.py:198(str_hash)
    21284    2.040    0.000   21.370    0.001 stringobject.py:211(string_richcompare)
    21284    0.490    0.000   21.860    0.001 stringobject.py:277(str_str_eq)
      492    0.010    0.000    0.030    0.000 tupleobject.py:10(__init__)
        8    0.000    0.000    0.000    0.000 tupleobject.py:29(tuple_is_true)
      231    0.000    0.000    0.030    0.000 tupleobject.py:34(tuple_len)
      860    0.020    0.000    0.060    0.000 tupleobject.py:40(getitem_tuple_int)
       15    0.020    0.001    4.580    0.305 tupleobject.py:51(getitem_tuple_slice)
      197    0.000    0.000    0.000    0.000 tupleobject.py:67(tuple_iter)
       50    0.000    0.000    0.000    0.000 tupleobject.py:73(tuple_add)
        8    0.000    0.000    0.080    0.010 tupletype.py:11(tupletype_new)
       32    0.000    0.000    0.000    0.000 typeobject.py:109(getdefaults)
       64    0.000    0.000    0.000    0.000 typeobject.py:116(slice)
       31    0.010    0.000    0.610    0.020 typeobject.py:130(eval_code)
    49/37    0.000    0.000   15.000    0.405 typeobject.py:157(type_call)
       70    0.000    0.000    0.070    0.001 typeobject.py:164(type_issubtype)
       36    0.000    0.000    0.100    0.003 typeobject.py:174(type_getattr)
        6    0.030    0.005    0.040    0.007 typeobject.py:18(__init__)
      454    0.000    0.000    0.000    0.000 typeobject.py:31(getbases)
  538/238    0.040    0.000    0.440    0.002 typeobject.py:39(getmro)
      145    0.010    0.000    0.780    0.005 typeobject.py:47(lookup)
      233    0.030    0.000    0.110    0.000 typeobject.py:57(lookup_exactly_here)
       18    0.000    0.000    0.000    0.000 typeobject.py:69(acceptclass)
       13    0.000    0.000    0.000    0.000 typeobject.py:89(hack_out_multimethods)
      456    0.010    0.000    0.010    0.000 typeobject.py:96(__init__)
       16    0.000    0.000    0.100    0.006 typetype.py:13(typetype_new)
        1    0.010    0.010    0.010    0.010 userobject.py:1(?)
       12    0.000    0.000    0.140    0.012 userobject.py:20(__init__)
       31    0.000    0.000    0.000    0.000 userobject.py:52(get_builtin_impl_class)
       12    0.000    0.000    0.090    0.008 userobject.py:59(getsinglebuiltintype)
        1    0.000    0.000    0.000    0.000 userobject.py:6(W_UserObject)
       15    0.010    0.001    0.010    0.001 userobject.py:74(user_type)
       30    0.000    0.000    0.000    0.000 userobject.py:80(user_getdict)
       62    0.000    0.000    0.000    0.000 userobject.py:92(build_user_operation)
       31    0.000    0.000    1.290    0.042 userobject.py:93(user_operation)
        2    0.000    0.000    0.020    0.010 usertype.py:15(__init__)
      149    0.010    0.000    0.400    0.003 usertype.py:25(getbases)
       94    0.000    0.000    0.380    0.004 usertype.py:32(lookup_exactly_here)
       12    0.010    0.001   14.790    1.232 usertype.py:45(usertype_new)

-- 
  Hey, if I thought I was wrong, I'd change my mind.  :)
                                    -- Grant Edwards, comp.lang.python


From jantzen at netcologne.de  Fri Jun 13 23:09:10 2003
From: jantzen at netcologne.de (=?iso-8859-1?Q?G=FCnter_Jantzen?=)
Date: Fri, 13 Jun 2003 23:09:10 +0200
Subject: [pypy-dev] Pythonic
References: <20030612122622.GB11354@magma.unil.ch>
Message-ID: <01b901c331f0$0a1e3610$0100a8c0@Lieschen>

Armin Rigo wrote
"It would make the PyPy code slightly less strictly-typed, i.e. more
Pythonic
:-)  and as we will need the translator at some point (probably soon) we may
rest confident that typing bugs will eventually be caught by this analysis
phase."

Every language has its own strength.
A very good language will be even good in domains for which it was not
designed in the first place

Python is useful, understandable, dynamic, expressive  and not restricted

Restricted Python will be different from Python
It will be restricted and still be useful, understandable, dynamic and
expressive.
But not as dynamic as Python.

Its strength will be, that it can be translated.
For this purpose it will have some (not much) static, strict and
declarative elements.

Every little tag, every annotation will change the language.
We should be aware of this.  Maybe restricted Python will be rewritten
sometimes.
Because we need a good design.





From hpk at trillke.net  Sat Jun 14 19:32:51 2003
From: hpk at trillke.net (holger krekel)
Date: Sat, 14 Jun 2003 19:32:51 +0200
Subject: [pypy-dev] Objects and types in the stdobjspace
In-Reply-To: <20030612111837.GA11354@magma.unil.ch>;
	from arigo@tunes.org on Thu, Jun 12, 2003 at 01:18:37PM +0200
References: <20030607085151.O9377@prim.han.de>
	<20030607130050.GA19854@magma.unil.ch> <20030610000255.B21086@prim.han.de>
	<20030610121323.GA491@magma.unil.ch> <20030610152636.D21086@prim.han.de>
	<20030610134353.GA5452@magma.unil.ch> <20030610160300.G21086@prim.han.de>
	<20030610143026.GC5452@magma.unil.ch> <20030610173619.H21086@prim.han.de>
	<20030612111837.GA11354@magma.unil.ch>
Message-ID: <20030614193251.Z21086@prim.han.de>

[Armin Rigo Thu, Jun 12, 2003 at 01:18:37PM +0200]
> Hello Holger,
> 
> On Tue, Jun 10, 2003 at 05:36:19PM +0200, holger krekel wrote:
> > Please find attached a modified "stringobject.py" where i 
> > implemented this convention. I found that the naming scheme
> > was confused sometimes and the new "enforcement" makes this
> > very regular and readable (IMHO, of course).
> 
> It seems pretty reasonable, to say the least. What do other people think about 
> this point ?
> 
> If accepted, the introspection-registering loop at the end of the file should
> probably be moved in a common location and called from the end of each
> xxxobject.py file.

i just checked in Revision 809 which does this for all the xxxobject.py
files and a few xxxtype.py files. For some details see 

    http://codespeak.net/pipermail/pypy-svn/2003-June/000347.html

The only exception is intobject.py because test/test_intobject.py
directly uses the names in intobject.py instead of going through
the StdObjSpace or Type object.  I didn't fully understand the
OverflowError/restricted_int business so i left the intobject
for the time beeing. Any comment appreciated. 

After changing the registration mechanism i think the xxxobject.py
files are now in a more consistent and easier-to-read state. 
Hopefully i didn't break anything. 

greetings,

    holger

From hpk at trillke.net  Sat Jun 14 20:16:52 2003
From: hpk at trillke.net (holger krekel)
Date: Sat, 14 Jun 2003 20:16:52 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <20030612122622.GB11354@magma.unil.ch>;
	from arigo@tunes.org on Thu, Jun 12, 2003 at 02:26:22PM +0200
References: <20030612122622.GB11354@magma.unil.ch>
Message-ID: <20030614201652.A21086@prim.han.de>

Hello Armin!

[Armin Rigo Thu, Jun 12, 2003 at 02:26:22PM +0200]
> Hello !
> 
> I have slowly come to think that wrapping every object we want to see at the 
> application-level in a W_XxxObject class is conceptually a good idea but not 
> necessarily pratically required. I used to think that it would be useful for 
> the translator-to-C. Now I'm no longer convinced. Contrast these two 
> implementations of integer comparison :
> 
> def int_int_lt(space, w_int1, w_int2):
>     i = w_int1.intval
>     j = w_int2.intval
>     return space.newbool( i < j )
> 
> vs.
> 
> def lt__int_int(space, i, j):
>     return bool(i < j)
> 
> It seems to me that the first implementation is a typical example of the 
> boilerplate kind of code that we hoped the project would remove, isn't it?

Interesting. But the second example should still be

    def lt__Int_Int(space, i, j):
        return space.newbool(i < j)

because using builtins in restricted python should be avoided, shouldn't it? 
Especially in the presence of multiple object spaces this would loose
context.  And i am not sure if just defining "lt__ANY_ANY" could already suffice,
because we rely on dispatching to restricted python (objects), anyway. 

> How I see things right now would be that any type (or class) of restricted
> Python would be suitable for implementing Python objects. For complex objects
> nothing changes:  we still need classes acting as structures, like a
> W_ListObject with the over-allocation machinery fields. But the most simple
> objects could be implemented just directly with simple interpreter-level
> objects instead of requiring them to be classes W_XxxObject.
> 
> This idea comes from the fact that over the time the W_XxxObject class
> definitions themselves have become more and more skeletical, with all
> interesting functions defined as multimethods outside the class.  The only
> relevant piece of information left in the W_XxxObject classes today is the
> 'statictype' attribute, which would be easily to implement in some other way.

But the W_XXXObject is also used as a tag for a specific implementation of a
type. IOW there is no 1:1 relationship of W_XXXObject and W_XXXType,
right? 

> This simplification could also avoid some problems with objects that are not
> properly wrapped, because the simple interpreter-level objects don't need any
> explicit wrapping any more. Actually "wrapping" may become a transparent
> operation: the translator-to-C will detect the need automatically. Example: if
> there is an (interpreter-level) variable 'x' containing an integer, and if at
> some point we use it in a call to a function that has been found to expect a
> wrapped object ( = any one of the available implementation of objects) then we
> must add some tag that says the int is really an int (as opposed to any of the
> other implementations of objects). The translator will have to do this kind of
> operations anyway for a lot of other cases. It just seems to me that it can
> do it pretty well for this case too. Wrapping would now be automatical.
> 
> It would make the PyPy code slightly less strictly-typed, i.e. more Pythonic
> :-)  and as we will need the translator at some point (probably soon) we may
> rest confident that typing bugs will eventually be caught by this analysis
> phase.
> 
> In other words I am suggesting that we can simply use the basic types we
> allowed in restricted python to implement some of the xxxobject.py:

It seems to me that "restricted python" is largely vapor-ware so i have
problems following.  Or am i missing something? 

As usual, i ask you to recap the current state of concepts, 
this time on the inter-dependencies of 

    "Restricted Python" Objects 

    "Interpreter-Level" Objects

    "StdObjSpace Objects" / Types / Implementations (object model for short) 

   
Either way, I think there should be nothing like "__builtin__" functions/types 
because it utterly confuses at least me how they intermingle with the
different levels we have to deal with. So instead of 

    bool(...)

i suggest always writing something like

    space.newbool(...)

or so. We should find a way to disable "builtins" in order to force explicit
name bindings (like clearing the __builtin__.__dict__ or so). 

greetings,

    holger

From tismer at tismer.com  Sun Jun 15 04:39:13 2003
From: tismer at tismer.com (Christian Tismer)
Date: Sun, 15 Jun 2003 04:39:13 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <20030612122622.GB11354@magma.unil.ch>
References: <20030612122622.GB11354@magma.unil.ch>
Message-ID: <3EEBDC51.5030408@tismer.com>

Armin Rigo wrote:

> Hello !
> 
> I have slowly come to think that wrapping every object we want to see at the 
> application-level in a W_XxxObject class is conceptually a good idea but not 
> necessarily pratically required. I used to think that it would be useful for 
> the translator-to-C. Now I'm no longer convinced. Contrast these two 
> implementations of integer comparison :
> 
> def int_int_lt(space, w_int1, w_int2):
>     i = w_int1.intval
>     j = w_int2.intval
>     return space.newbool( i < j )
> 
> vs.
> 
> def lt__int_int(space, i, j):
>     return bool(i < j)
> 
> It seems to me that the first implementation is a typical example of the 
> boilerplate kind of code that we hoped the project would remove, isn't it?

It is the kind of code that *you* introduced into the project.
Your new definitions seems to do an unwrap before calling the
function. Or aren't they even wrapped?

> How I see things right now would be that any type (or class) of restricted
> Python would be suitable for implementing Python objects. For complex objects
> nothing changes:  we still need classes acting as structures, like a
> W_ListObject with the over-allocation machinery fields. But the most simple
> objects could be implemented just directly with simple interpreter-level
> objects instead of requiring them to be classes W_XxxObject.

For me, it is not the question whether they have to be a class
or not, but I want the levels be kept apart, properly.
Where do you now make the distinction between a machine
level int and an interpreter level integer object.

> This idea comes from the fact that over the time the W_XxxObject class
> definitions themselves have become more and more skeletical, with all
> interesting functions defined as multimethods outside the class.  The only
> relevant piece of information left in the W_XxxObject classes today is the
> 'statictype' attribute, which would be easily to implement in some other way.

I don't take this, if set as simple as you did.
Please, explain how this concept keeps the
concept of object spaces intact.

...

> In other words I am suggesting that we can simply use the basic types we
> allowed in restricted python to implement some of the xxxobject.py:
> 
> * intobject.py  -  int
> (ints could be forced to r_ints before the operations for which it matter, 
> e.g. "def lshift__int_int(self, i, j): ... return r_int(i) << j")

I don't understand why this is suddenly valid.
Please explain, in short words.
Or is this a smaller change than I see it.

Anyway, such sudden design changes happen quite often, and
they are not helpful to make the project understandable,
when this is introduced by emails, with a few simple claims.
I have been on all sprints so far, and even I have difficulties
with this. It must be incrediblly hard for anybody outside
this group to follow at all.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guenter.jantzen at netcologne.de  Sun Jun 15 07:19:19 2003
From: guenter.jantzen at netcologne.de (=?iso-8859-1?Q?G=FCnter_Jantzen?=)
Date: Sun, 15 Jun 2003 07:19:19 +0200
Subject: [pypy-dev] AbstractObjectSpace
References: <20030612122622.GB11354@magma.unil.ch>
	<3EEBDC51.5030408@tismer.com>
Message-ID: <000801c332fd$ad337050$0100a8c0@Lieschen>

Hello,

on
http://codespeak.net/moin/pypy/moin.cgi/AbstractObjectSpace

I found:
"
Typically, however, abstract object spaces are a (little) bit less abstract,
still maintaining a minimal amount of information about the objects. For
example, a wrapped object could be represented by its type. You then define
the object space's add to return int when the two arguments are int and int.
That way, you abstractedly call a function with the input argument's types
and what the interpreter will do is a type inference. (Here also there are
subtle problems, even besides the remark that integer operations can
overflow and actually return longs in a real Python implementation.)
"
This concept will only work when output types can be determined by input
types (independent of concrete values).
This will be a language restriction. For example, the following function
will not be valid in restricted Python:

def spamrelease(i):
   if i > 0:
      return i - 1
   else
      return "counter is closed"

Consequently we should work with function prototypes. Not as restrictive as
in C. More a set of rules or patterns
for calculating output types from input types.

G?nter



From tismer at tismer.com  Mon Jun 16 00:05:38 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 16 Jun 2003 00:05:38 +0200
Subject: [pypy-dev] Re: [pypy-svn] rev 809 - pypy/trunk/src/pypy/objspace/std
In-Reply-To: <20030614171518.496895A415@thoth.codespeak.net>
References: <20030614171518.496895A415@thoth.codespeak.net>
Message-ID: <3EECEDB2.2090206@tismer.com>

hpk at codespeak.net wrote:

> Author: hpk
> Date: Sat Jun 14 19:15:16 2003
> New Revision: 809
> 
> Modified:

19 modified source files.

> Log:
> revised multimethod registration machinery 
> 
> - now functions whose name is of the form "X__Y1_Y2_..."
>   are registered on the multimethod "X" on StdObjSpace or
>   (if that doesn't exist) on an alternatively provided namespace's
>   multimethod.

Etc.
I don't recall that this change has been agreed upon. I
understood this as just an idea, posted for discussion.
Am I missing something from the last messages?
Otherwise I would like to ask this admittedly anarchical group,
to define *how* anarchical it is planning to be in the future.

...

> - remaining is "intobject.py" because test/test_intobject.py 
>   uses the intobject.* names directly instead of going through
>   the StdObjSpace. Actually, at first i also changed intobject 
>   and test_intobject but i didn't understand the "OverflowError"
>   handling so i reverted my changes to not destroy anything
> 
>   could someone explain the r_int / int / long / overflow business 
>   on pypy-dev? 

You have been pondering about intobject in another mail to pypy-dev
as well. I would like to remind you that this was implemented
by me on the first sprint, as one of the very first, really complete
implementations of anything not faked. It is therefore in the
style of that time, which has changed in the between, of course.

Most probably, you are  unaware of this, or you would surely have
asked me, directly.

On your questions concerning restricted integers:
Please see
http://codespeak.net/moin/pypy/moin.cgi/RestrictedPython
and the chapter there named "Example: Integer Types",
which I also wrote on that sprint.

If you need any information in extent of this, please let me know.
Maybe there are some bigger omissions.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From hpk at trillke.net  Mon Jun 16 00:29:02 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 16 Jun 2003 00:29:02 +0200
Subject: [pypy-dev] Re: [pypy-svn] rev 809 - pypy/trunk/src/pypy/objspace/std
In-Reply-To: <3EECEDB2.2090206@tismer.com>;
	from tismer@tismer.com on Mon, Jun 16, 2003 at 12:05:38AM +0200
References: <20030614171518.496895A415@thoth.codespeak.net>
	<3EECEDB2.2090206@tismer.com>
Message-ID: <20030616002902.I21086@prim.han.de>

[Christian Tismer Mon, Jun 16, 2003 at 12:05:38AM +0200]
> hpk at codespeak.net wrote:
> 
> > Author: hpk
> > Date: Sat Jun 14 19:15:16 2003
> > New Revision: 809
> > 
> > Modified:
> 
> 19 modified source files.
> 
> > Log:
> > revised multimethod registration machinery 
> > 
> > - now functions whose name is of the form "X__Y1_Y2_..."
> >   are registered on the multimethod "X" on StdObjSpace or
> >   (if that doesn't exist) on an alternatively provided namespace's
> >   multimethod.
> 
> Etc.
> I don't recall that this change has been agreed upon. I
> understood this as just an idea, posted for discussion.

I took three more or less positive oppinions as enough to
give it a go. Maybe that was too hasty. 

> Am I missing something from the last messages?
> Otherwise I would like to ask this admittedly anarchical group,
> to define *how* anarchical it is planning to be in the future.

sure enough, the code base is still pretty much in flux. 
OTOH the revision-809 change is not supposed to change any
architecture but to reduce boilerplate and improve readability. 
I went through all the files and didn't encounter any problems but
in fact fixed some bugs and typos which went unnoticed before. 

> ...
> 
> > - remaining is "intobject.py" because test/test_intobject.py 
> >   uses the intobject.* names directly instead of going through
> >   the StdObjSpace. Actually, at first i also changed intobject 
> >   and test_intobject but i didn't understand the "OverflowError"
> >   handling so i reverted my changes to not destroy anything
> > 
> >   could someone explain the r_int / int / long / overflow business 
> >   on pypy-dev? 
> 
> You have been pondering about intobject in another mail to pypy-dev
> as well. I would like to remind you that this was implemented
> by me on the first sprint, as one of the very first, really complete
> implementations of anything not faked. It is therefore in the
> style of that time, which has changed in the between, of course.

No doubt, the intobject is the most detailed implementation among
the objspace's xxxobjects.  And I knew that you implemented it - 
also from some comments :-)
But i checked with "svn log intobject.py" and noticed that the
last five commits came from Armin and also involved Multimethods...

> Most probably, you are  unaware of this, or you would surely have
> asked me, directly.

...I have sent you a couple of private mails lately and you didn't
respond so i thought you may be busy with other stuff. 
So all in all i thought i'd be better off asking in the wild 
especially because other people might be interested in any answers. 

> On your questions concerning restricted integers:
> Please see
> http://codespeak.net/moin/pypy/moin.cgi/RestrictedPython
> and the chapter there named "Example: Integer Types",
> which I also wrote on that sprint.

thanks for the pointer. 

cheers,

    holger

From tismer at tismer.com  Mon Jun 16 02:47:41 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 16 Jun 2003 02:47:41 +0200
Subject: [pypy-dev] Very relevant stuff (was: [pypy-svn] rev 812 - in
 pypy/trunk/src/pypy:interpreter objspace/std objspace/std/test)
In-Reply-To: <20030615185124.773945A41A@thoth.codespeak.net>
References: <20030615185124.773945A41A@thoth.codespeak.net>
Message-ID: <3EED13AD.4040509@tismer.com>

Howdy,

I thought it would make sense to post the most relevant
text of this check-in to the list.
It also would make sense to add this to the PyPy Wiki.

arigo at codespeak.net wrote:

> Added: pypy/trunk/src/pypy/objspace/std/mm_notes.txt
> ==============================================================================
Notes on Multimethods
*********************

Interpreter-level classes correspond to implementations of application-level
types. Several implementations can be given for the same type (e.g. several
ways to code strings or dictionaries), and conversely the same 
implementation
can cover several types (e.g. all instances of user-defined types currently
share the same implementation).

The hierarchy among the classes used for the implementations is convenient
for implementation purposes. It is not related to any application-level type
hierarchy.


Dispatch
--------

Multimethods dispatch by looking in a set of registered functions. Each
registered function has a signature, which defines which object 
implementation
classes are accepted at the corresponding argument position.

The name 'W_ANY' is a synonym for 'W_Object' (currently, possibly 'object'
later). As it accepts anything, it is the only way to guarantee that the
registered function will be called with exactly the same object as was
passed originally. ATTENTION: in all other cases the argument received by
the function may have been converted in some way. It must thus not be
considered to be 'id'entical to the original argument. For example it should
not be stored in a data structure, nor be queried for type, nor be used for
another multimethod dispatch -- the only thing you should do is read and
write its internal data.

For example, 'getattr(obj, attr)' is implemented with a W_StringObject 
second
argument when all it needs is just the name of the attr, and with a W_ANY
when the 'attr' object could be used as a key in obj.__dict__.


Delegation
----------

Delegation is a transparent convertion mechanism between object
implementations. The convertion can give a result of a different type
(e.g. int -> float) or of the same type (e.g. W_VeryLongString -> str).
There is a global table of delegators. We should not rely on the delegators
to be tried in any particlar order, or at all (e.g. the int -> float 
delegator
could be ignored when we know that no registered function will accept a 
float
anyway).

Delegation is also used to emulate inheritance between built-in types
(e.g. bool -> int). This is done by delegation because there is no reason
that a particular implementation of a sub-type can be trivially typecast
to some other particular implementation of the parent type; the process 
might
require some work.


Types
-----

Types are implemented by the class W_TypeObject. This is where 
inheritance and the Method Resolution Order are defined, and where
attribute look-ups are done.

Instances of user-defined types are implementated as W_UserObjects.
A user-defined type can inherit from built-in types (maybe more
than one, although this is incompatible with CPython). The
W_UserObject delegator converts the object into any of these
"parent objects" if needed. This is how user-defined types appear
to inherit all built-in operator implementations.

Delegators should be able to invoke user code; this would let us
implement special methods like __int__() by calling them within a
W_UserObject -> int delegator.


Specifics of multimethods
-------------------------

Multimethods dispatch more-specific-first, left-to-right (i.e. if
there is an exact match for the first argument it will always be
tried first).

Delegators are automatically chained (i.e. A -> B and B -> C would be
combined to allow for A -> C delegation).

Delegators do not publish the class of the converted object in advance,
so that the W_UserObject delegator can potentially produce any other
built-in implementation. This means chaining and chain loop detection 
cannot be done statically (at least without help from an analysis
tool like the translator-to-C). To break loops, we can assume (unless
a particular need arises) that delegators are looping when they return
an object of an already-seen class.


Registration
------------

The register() method of multimethods adds a function to its database of
functions, with the given signature. A function that raises
FailedToImplement causes the next match to be tried.

'delegate' is the special unary multimethod that should try to convert
its argument to something else. For greater control, it can also return
a list of 2-tuples (class, object), or an empty list for failure to 
convert the argument to anything. All delegators will potentially be
tried, and recursively on each other's results to do chaining.

A priority ordering between delegators is used. See objspace.PRIORITY_*.


Translation
-----------

The code in multimethod.py is not supposed to be read by the
translator-to-C. Special optimized code will be generated instead
(typically some kind of precomputed dispatch tables).

Delegation is special-cased too. Most delegators will be found
to return an object of a statically known class, which means that
most of the chaining and loop detection can be done in advance.


Multimethod slicing
-------------------

Multimethods are visible to user code as (bound or unbound) methods
defined for the corresponding types. (At some point built-in functions
like len() and the operator.xxx() should really directly map to the
multimethods themselves, too.)

To build a method from a multimethod (e.g. as in 'l.append' or
'int.__add__'), the result is actually a "slice" of the whole
multimethod, i.e. a sub-multimethod in which the registration table has
been trimmed down. (Delegation mechanisms are not restricted for sliced
multimethods.)

Say that C is the class the new method is attached to (in the above
examples, respectively, C=type(l) and C=int). The restriction is
based on the registered class of the first argument ('self' for the
new method) in the signature. If this class corresponds to a fixed
type (as advertized by 'statictype'), and this fixed type is C or a
superclass of C, then we keep it.

Some multimethods can also be sliced along their second argument,
e.g. for __radd__().

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From tismer at tismer.com  Mon Jun 16 11:24:19 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 16 Jun 2003 11:24:19 +0200
Subject: [pypy-dev] Re: [pypy-svn] rev 809 - pypy/trunk/src/pypy/objspace/std
In-Reply-To: <20030616002902.I21086@prim.han.de>
References: <20030614171518.496895A415@thoth.codespeak.net>
	<3EECEDB2.2090206@tismer.com> <20030616002902.I21086@prim.han.de>
Message-ID: <3EED8CC3.7050407@tismer.com>

holger krekel wrote:

> [Christian Tismer Mon, Jun 16, 2003 at 12:05:38AM +0200]

I wrote:
>>Am I missing something from the last messages?
>>Otherwise I would like to ask this admittedly anarchical group,
>>to define *how* anarchical it is planning to be in the future.

Holger:
> sure enough, the code base is still pretty much in flux. 
> OTOH the revision-809 change is not supposed to change any
> architecture but to reduce boilerplate and improve readability. 
> I went through all the files and didn't encounter any problems but
> in fact fixed some bugs and typos which went unnoticed before. 

Ok, now I understand. I was misinterpreting something
and didn't follow the checkins closely enough.

Please ignore my former rant, this was pointless.

See you soon - cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From mwh at python.net  Mon Jun 16 12:20:19 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 16 Jun 2003 11:20:19 +0100
Subject: [pypy-dev] Re: [pypy-svn] rev 810 -
	pypy/trunk/src/pypy/interpreter/test
In-Reply-To: <20030614173856.0FBF35A415@thoth.codespeak.net>
	(hpk@codespeak.net's
	message of "Sat, 14 Jun 2003 19:38:56 +0200 (MEST)")
References: <20030614173856.0FBF35A415@thoth.codespeak.net>
Message-ID: <2my902fhng.fsf@starship.python.net>

hpk at codespeak.net writes:

> Author: hpk
> Date: Sat Jun 14 19:38:55 2003
> New Revision: 810
>
> Modified:
>    pypy/trunk/src/pypy/interpreter/test/test_interpreter.py
> Log:
> fixed test_raise test checking for wrong error message
>

This will break the test when run under the trivial object space.

I'm hoping that soon we'll be able to write all these tests in
application space, and so some of the bizarre hacks in (particularly)
test_interpreter will be able to go away.

Cheers,
M.

-- 
  > I'm a little confused.
  That's because you're Australian!  So all the blood flows to
  your head, away from the organ most normal guys think with.
                        -- Mark Hammond & Tim Peters, comp.lang.python

From mwh at python.net  Mon Jun 16 12:59:34 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 16 Jun 2003 11:59:34 +0100
Subject: [pypy-dev] Re: [pypy-svn] rev 810 -
	pypy/trunk/src/pypy/interpreter/test
References: <20030614173856.0FBF35A415@thoth.codespeak.net>
	<2my902fhng.fsf@starship.python.net>
Message-ID: <2mvfv6ffu1.fsf@starship.python.net>

Michael Hudson <mwh at python.net> writes:

> hpk at codespeak.net writes:
>
>> Author: hpk
>> Date: Sat Jun 14 19:38:55 2003
>> New Revision: 810
>>
>> Modified:
>>    pypy/trunk/src/pypy/interpreter/test/test_interpreter.py
>> Log:
>> fixed test_raise test checking for wrong error message
>>
>
> This will break the test when run under the trivial object space.

Eh, that was a lie.  Ignore me.

> I'm hoping that soon we'll be able to write all these tests in
> application space, and so some of the bizarre hacks in (particularly)
> test_interpreter will be able to go away.

This wasn't.

Cheers,
M.

-- 
  I'm not particularly fond of singing GSTQ because she stands for
  some things I don't, but it's not really worth letting politics
  getting in the way of a good bawling.     -- Dan Sheppard, ucam.chat


From tismer at tismer.com  Mon Jun 16 14:55:39 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 16 Jun 2003 14:55:39 +0200
Subject: [pypy-dev] PyPy -- Lua
Message-ID: <3EEDBE4B.20002@tismer.com>

Hi all,

just by chance, I stumbled over a post from Gustavo
Niemeyer on the python list: "Python articles".

He mentioned his website, which an article
"- Why Lua was embedded in APT-RPM".
https://moin.conectiva.com.br/GustavoNiemeyer

Here that particular excerpt:
"""
Why Lua?

One might think I've used Lua just because I live in the same country as 
its core developers (Brazil), but that's not the case. Indeed, I've done 
a pretty intensive research about embeddable languages before choosing 
Lua. I was looking for a fast, and small language. When you have a 
library which has about 500kb, you can't embed a large interpreter to 
extend the functionality, otherwise you'd be extending the interpreter, 
not the library. One might think that the interpreter library would be 
in the system anyway, so that wouldn't be a real problem. Unfortunately, 
that doesn't apply to APT-RPM, since it is used in small systems, and in 
installer environments. The current Lua interpreter is still under 
100kb, and is very fast if compared to other interpreters. I really 
think the Lua interpreter has no current competitors in that specific area.
"""

How is this related to PyPy?

Well, you guessed it.

PyPy still has a particular branch of thought in it:
Minimal Python. That is, as one target of PyPy,
we should eventually be able to spit
out some very tiny embeddable interpreter, that
can compete with, say, Lua.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From lac at strakt.com  Mon Jun 16 15:56:29 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 16 Jun 2003 15:56:29 +0200
Subject: [pypy-dev] 
 Did Armin or Michael log the conversation they had on Friday on irc?
Message-ID: <200306161356.h5GDuTPZ012051@ratthing-b246.strakt.com>

Jacob read it and thought it was wonderful, and I would like to read it
if one of you logged it -- or if anybody else was there and logged it.

Should we ask Moshez for a bot?

Laura

From arigo at tunes.org  Mon Jun 16 16:16:07 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 16 Jun 2003 16:16:07 +0200
Subject: [pypy-dev] Very relevant stuff (was: [pypy-svn] rev 812 - in
	pypy/trunk/src/pypy:interpreter objspace/std objspace/std/test)
In-Reply-To: <3EED13AD.4040509@tismer.com>
References: <20030615185124.773945A41A@thoth.codespeak.net>
	<3EED13AD.4040509@tismer.com>
Message-ID: <20030616141607.GA6532@magma.unil.ch>

Hello Christian,

On Mon, Jun 16, 2003 at 02:47:41AM +0200, Christian Tismer wrote:
> I thought it would make sense to post the most relevant
> text of this check-in to the list.
> 
> >Added: pypy/trunk/src/pypy/objspace/std/mm_notes.txt
> >...

Done. This was a design document for me, I wrote it to clear out how 
delegation should cleanly work before I went out and re-implemented it to 
actually work like that.


Armin

From arigo at tunes.org  Mon Jun 16 16:41:54 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 16 Jun 2003 16:41:54 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <3EEBDC51.5030408@tismer.com>
References: <20030612122622.GB11354@magma.unil.ch>
	<3EEBDC51.5030408@tismer.com>
Message-ID: <20030616144154.GB6532@magma.unil.ch>

Hello Christian,

On Sun, Jun 15, 2003 at 04:39:13AM +0200, Christian Tismer wrote:
> >It seems to me that the first implementation is a typical example of the 
> >boilerplate kind of code that we hoped the project would remove, isn't it?
> 
> It is the kind of code that *you* introduced into the project.
> Your new definitions seems to do an unwrap before calling the
> function. Or aren't they even wrapped?

Sorry about that, I tend to write long confusing e-mails about subjects that
keep evolving a bit in my head. I will try to avoid that in the future 
(probably just writing an e-mail, trashing it, and writing it again the next 
day would be a good start).

In that case the changes I proposed are not as big as they might seem. To make
it short, say for integers for example, they would just amount to defining
"W_IntObject = r_int", instead of a class with a single field which is an
r_int.

So code like that:

    def lt__Int_Int(space, i, j):
        return space.newbool(i < j)

would work just because W_IntObject is r_int. There is no auto-unwrapping
magic; the r_int is the wrapped object -- as well as the unwrapped one, which
is I admit not much help in clearing the confusion. (It might thus not be such
a good idea -- although it would certainly be a boilerplate-reductor.)

This would not change anything at all for the code in pypy.interpreter. It
would only be meaningful to the internals of objspace.std. It would mean that
if we know, say, that "W_StringObject = str", then we would know that for
reasonably short strings space.wrap("hello") is actually just "hello", thus
removing some boilerplate.

This *only* applies within the StdObjSpace. Code in pypy.interpreter cannot
assume anything like that. Also, it does not prevent alternate implementations
of integers or strings to coexist; they would be done as usual with, say, some
new W_VeryLongStringObject, which is unconcerned about the existence of the
previous string implementation as "W_StringObject = str".


As a possibly clearer alternate to that, defining a delegation from
"W_StringObject" to "str" would be a matter of two lines of code, and would do
the transparent unwrapping you mentioned.

Damn, I have again been changing my mind between the start and the end of the 
e-mail.


A bient?t,

Armin.

From mwh at python.net  Mon Jun 16 16:34:21 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 16 Jun 2003 15:34:21 +0100
Subject: [pypy-dev] 
 Re: Did Armin or Michael log the conversation they had on Friday on
 irc?
References: <200306161356.h5GDuTPZ012051@ratthing-b246.strakt.com>
Message-ID: <2mznkiytua.fsf@starship.python.net>

Laura Creighton <lac at strakt.com> writes:

> Jacob read it and thought it was wonderful, and I would like to read it
> if one of you logged it -- or if anybody else was there and logged it.

It's still sitting in my emacs buffer, I'll try to remember to upload
it somewhere before I quit.

> Should we ask Moshez for a bot?

Might be an idea.

Cheers,
M.

-- 
  Lisp does badly because we refuse to lie.  When people ask us if 
  we can solve insoluble problems we say that we can't, and because 
  they expect us to lie to them, they find some other language
  where the truth is less respected.   -- Tim Bradshaw, comp.lang.lisp


From lac at strakt.com  Mon Jun 16 16:55:33 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 16 Jun 2003 16:55:33 +0200
Subject: [pypy-dev] reorganized Wiki
Message-ID: <200306161455.h5GEtXwf012315@ratthing-b246.strakt.com>

where did the rest of the things about the Hildesheim and Gothenburg
Sprints go?  I want to refer to them for the OSCON talk.  Where's the
picture of TrilkeGut?

Laura

From lac at strakt.com  Mon Jun 16 16:56:43 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 16 Jun 2003 16:56:43 +0200
Subject: [pypy-dev] Very relevant stuff (was: [pypy-svn] rev 812 - in
	pypy/trunk/src/pypy:interpreter objspace/std objspace/std/test) 
In-Reply-To: Message from Armin Rigo <arigo@tunes.org> 
	of "Mon, 16 Jun 2003 16:16:07 +0200."
	<20030616141607.GA6532@magma.unil.ch> 
References: <20030615185124.773945A41A@thoth.codespeak.net>
	<3EED13AD.4040509@tismer.com> <20030616141607.GA6532@magma.unil.ch> 
Message-ID: <200306161456.h5GEuhOs012337@ratthing-b246.strakt.com>

In a message of Mon, 16 Jun 2003 16:16:07 +0200, Armin Rigo writes:
>Hello Christian,
>
>On Mon, Jun 16, 2003 at 02:47:41AM +0200, Christian Tismer wrote:
>> I thought it would make sense to post the most relevant
>> text of this check-in to the list.
>> 
>> >Added: pypy/trunk/src/pypy/objspace/std/mm_notes.txt
>> >...
>
>Done. This was a design document for me, I wrote it to clear out how 
>delegation should cleanly work before I went out and re-implemented it to
>actually work like that.
>
>
>Armin

Is it living someplace on the Wiki now?  or in our doc directory?

Laura

From pedronis at bluewin.ch  Mon Jun 16 17:01:01 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon, 16 Jun 2003 17:01:01 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <20030616144154.GB6532@magma.unil.ch>
References: <3EEBDC51.5030408@tismer.com>
 <20030612122622.GB11354@magma.unil.ch>
 <3EEBDC51.5030408@tismer.com>
Message-ID: <5.2.1.1.0.20030616164914.02475450@pop.bluewin.ch>

At 16:41 16.06.2003 +0200, Armin Rigo wrote:
>Hello Christian,
>
>On Sun, Jun 15, 2003 at 04:39:13AM +0200, Christian Tismer wrote:
> > >It seems to me that the first implementation is a typical example of the
> > >boilerplate kind of code that we hoped the project would remove, isn't it?
> >
> > It is the kind of code that *you* introduced into the project.
> > Your new definitions seems to do an unwrap before calling the
> > function. Or aren't they even wrapped?
>
>Sorry about that, I tend to write long confusing e-mails about subjects that
>keep evolving a bit in my head. I will try to avoid that in the future
>(probably just writing an e-mail, trashing it, and writing it again the next
>day would be a good start).
>
>In that case the changes I proposed are not as big as they might seem. To make
>it short, say for integers for example, they would just amount to defining
>"W_IntObject = r_int", instead of a class with a single field which is an
>r_int.
>
>So code like that:
>
>     def lt__Int_Int(space, i, j):
>         return space.newbool(i < j)
>
>would work just because W_IntObject is r_int. There is no auto-unwrapping
>magic; the r_int is the wrapped object -- as well as the unwrapped one, which
>is I admit not much help in clearing the confusion. (It might thus not be such
>a good idea -- although it would certainly be a boilerplate-reductor.)

If we want to reduce the boilerplate and avoid the confusion we should 
really think in terms of auto-unwrapping.

In terms of a translated impl r_int is an unboxed int, W_IntObject a boxed 
one. It is true that as long as we are hosted by CPython and it is 
implicitly clear what is boxed/unboxed we can use the same objects for 
wrapped(boxed)/unwrapped(unboxed) values for some types and have 
wrapping/unwrapping be the identity for them. It is also true that having 
code in terms of unboxed objects is maybe better to generate instruction 
sequence templates for native code generation.

But for a translated impl. wrapping/unwrapping won't be no-ops.  So the 
code is probably clearer for the uninitiated as it is. If we want to reduce 
boilerplate we should add some auto-unwrapping/wrapping logic where it can 
make sense.

regards. 


From hpk at trillke.net  Mon Jun 16 17:04:49 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 16 Jun 2003 17:04:49 +0200
Subject: [pypy-dev] reorganized Wiki
In-Reply-To: <200306161455.h5GEtXwf012315@ratthing-b246.strakt.com>;
	from lac@strakt.com on Mon, Jun 16, 2003 at 04:55:33PM +0200
References: <200306161455.h5GEtXwf012315@ratthing-b246.strakt.com>
Message-ID: <20030616170449.A21086@prim.han.de>

[Laura Creighton Mon, Jun 16, 2003 at 04:55:33PM +0200]
> where did the rest of the things about the Hildesheim and Gothenburg
> Sprints go?  I want to refer to them for the OSCON talk.  Where's the
> picture of TrilkeGut?

Most of them can be found under the TitleIndex (letter "S")

    http://codespeak.net/moin/pypy/moin.cgi/TitleIndex

but they are pretty uninteresting from today's point of view, 
because they are mostly organizational stuff. 

The "winter Trillke-Gut" picture is here

    http://www.trillke.net/images/trillke_schnee.png

and today's is here

    http://www.trillke.net/images/trillke-front.jpg


We should have one person writing summaries of each
sprint. They should be the main documents when refering
to "old" sprints. 

greetings,

    holger

From arigo at tunes.org  Mon Jun 16 17:09:27 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 16 Jun 2003 17:09:27 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <20030614201652.A21086@prim.han.de>
References: <20030612122622.GB11354@magma.unil.ch>
	<20030614201652.A21086@prim.han.de>
Message-ID: <20030616150927.GC6532@magma.unil.ch>

Hello Holger,

On Sat, Jun 14, 2003 at 08:16:52PM +0200, holger krekel wrote:
> Interesting. But the second example should still be
> 
>     def lt__Int_Int(space, i, j):
>         return space.newbool(i < j)
> 
> because using builtins in restricted python should be avoided, shouldn't it? 

Not necessarily. See again
http://codespeak.net/moin/pypy/moin.cgi/RestrictedPython, "builtin functions".

> Especially in the presence of multiple object spaces this would loose
> context.

No context is needed. The built-in functions we are using here are the real 
low-level essential stuff, at the same level as the " < " operator between 
RPython's integers.

These have nothing to do with the functions redefined in module.builtin*; the 
list of built-in functions we allow in RPython is restricted, and how you use 
them too, so that they are no burden to the translator. Still, they are 
nothing more and nothing less than all other operator: we allow some, and 
disallow some.

> But the W_XXXObject is also used as a tag for a specific implementation of a
> type. IOW there is no 1:1 relationship of W_XXXObject and W_XXXType,
> right?

The names are misleading. There is one W_XXXType, but there might be a lot of 
classes implementing it. The existence of one particular such class with the 
exact name W_XXXObject is misleading. What I was suggesting was that a small 
number of these specific W_XXXObject classes might be equal to a restricted 
Python type instead of being a class at all.

> It seems to me that "restricted python" is largely vapor-ware so i have
> problems following.  Or am i missing something? 

This is a most interesting point. I think that trying to actually get
somewhere in this direction could be a subject of the upcoming sprint. The
first sprint was (very roughly) the interpreter, the second sprint the
stdobjspace, let the third one be the third main missing piece of the puzzle
:-)


Armin

From mwh at python.net  Mon Jun 16 17:13:54 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 16 Jun 2003 16:13:54 +0100
Subject: [pypy-dev] 
 Re: Did Armin or Michael log the conversation they had on Friday on
 irc?
In-Reply-To: <200306161356.h5GDuTPZ012051@ratthing-b246.strakt.com> (Laura
 Creighton's message of "Mon, 16 Jun 2003 15:56:29 +0200")
References: <200306161356.h5GDuTPZ012051@ratthing-b246.strakt.com>
Message-ID: <2mwufmys0d.fsf@starship.python.net>

Laura Creighton <lac at strakt.com> writes:

> Jacob read it and thought it was wonderful, and I would like to read it
> if one of you logged it -- or if anybody else was there and logged it.

I've put a (very lightly edited) version here:

    http://starship.python.net/crew/mwh/hacks/pypy-2003-06-13-log.txt

We waffle speculatively for a while, talk about how we expect the as
yet mythical RPython to C translator to work, implement a neutered
types module, get dis-goal.py nearly working, speed up dis-pregoal.py
by about 40% and wonder inconclusively where the rest of the time was
going.

> Should we ask Moshez for a bot?

Might be an idea.

Cheers,
M.

-- 
  Finding a needle in a haystack is a lot easier if you burn down
  the haystack and scan the ashes with a metal detector.
      -- the Silicon Valley Tarot (another one nicked from David Rush)

From lac at strakt.com  Mon Jun 16 17:17:04 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 16 Jun 2003 17:17:04 +0200
Subject: [pypy-dev] reorganized Wiki 
In-Reply-To: Message from holger krekel <hpk@trillke.net> 
   of "Mon, 16 Jun 2003 17:04:49 +0200." <20030616170449.A21086@prim.han.de> 
References: <200306161455.h5GEtXwf012315@ratthing-b246.strakt.com>
	<20030616170449.A21086@prim.han.de> 
Message-ID: <200306161517.h5GFH4Os012447@ratthing-b246.strakt.com>

In a message of Mon, 16 Jun 2003 17:04:49 +0200, holger krekel writes:
>[Laura Creighton Mon, Jun 16, 2003 at 04:55:33PM +0200]
>> where did the rest of the things about the Hildesheim and Gothenburg
>> Sprints go?  I want to refer to them for the OSCON talk.  Where's the
>> picture of TrilkeGut?
>
>Most of them can be found under the TitleIndex (letter "S")
>
>    http://codespeak.net/moin/pypy/moin.cgi/TitleIndex
>
>but they are pretty uninteresting from today's point of view, 
>because they are mostly organizational stuff. 

Thanks Holger.

>
>The "winter Trillke-Gut" picture is here
>
>    http://www.trillke.net/images/trillke_schnee.png

Ah, pretty.

>
>and today's is here
>
>    http://www.trillke.net/images/trillke-front.jpg

Pretty today, too.

>We should have one person writing summaries of each
>sprint. They should be the main documents when refering
>to "old" sprints. 

You want to write up Hildesheim and I will write up Gothenburg?

Also, do we have a 'how this came to be' document?  I'd like 1 paragraph
for the OSCON talk, but I cannot write it, because I wasn't there ...

>
>greetings,
>
>    holger

Thanks very much, Holger ...

Laura


From hpk at trillke.net  Mon Jun 16 17:22:44 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 16 Jun 2003 17:22:44 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <5.2.1.1.0.20030616164914.02475450@pop.bluewin.ch>;
	from pedronis@bluewin.ch on Mon, Jun 16, 2003 at 05:01:01PM +0200
References: <3EEBDC51.5030408@tismer.com>
	<20030612122622.GB11354@magma.unil.ch> <3EEBDC51.5030408@tismer.com>
	<20030616144154.GB6532@magma.unil.ch>
	<5.2.1.1.0.20030616164914.02475450@pop.bluewin.ch>
Message-ID: <20030616172244.B21086@prim.han.de>

Hello Samuele!

[Samuele Pedroni Mon, Jun 16, 2003 at 05:01:01PM +0200]
> At 16:41 16.06.2003 +0200, Armin Rigo wrote:
> >Hello Christian,
> >
> >On Sun, Jun 15, 2003 at 04:39:13AM +0200, Christian Tismer wrote:
> > > >It seems to me that the first implementation is a typical example of the
> > > >boilerplate kind of code that we hoped the project would remove, isn't it?
> > >
> > > It is the kind of code that *you* introduced into the project.
> > > Your new definitions seems to do an unwrap before calling the
> > > function. Or aren't they even wrapped?
> >
> >Sorry about that, I tend to write long confusing e-mails about subjects that
> >keep evolving a bit in my head. I will try to avoid that in the future
> >(probably just writing an e-mail, trashing it, and writing it again the next
> >day would be a good start).
> >
> >In that case the changes I proposed are not as big as they might seem. To make
> >it short, say for integers for example, they would just amount to defining
> >"W_IntObject = r_int", instead of a class with a single field which is an
> >r_int.
> >
> >So code like that:
> >
> >     def lt__Int_Int(space, i, j):
> >         return space.newbool(i < j)
> >
> >would work just because W_IntObject is r_int. There is no auto-unwrapping
> >magic; the r_int is the wrapped object -- as well as the unwrapped one, which
> >is I admit not much help in clearing the confusion. (It might thus not be such
> >a good idea -- although it would certainly be a boilerplate-reductor.)
> 
> If we want to reduce the boilerplate and avoid the confusion we should 
> really think in terms of auto-unwrapping.

Introducing the "w_*" convention was meant to un-confuse
interpreter-level and user-level objects :-)

In the above lt__Int_Int method there is another level of confusion IMO. 
One would think that when we evaluate the user level expression 

        1 < 2 

then this eventually dispatches a call to 

        lt__Int_Int(space, i, j)

which itself invokes 

        i < j

 ... leading to infinite recursion. Of course, it
doesn't but instead the "i < j" is typically performed as 

        space.unwrap(i) < space.unwrap(j)

thus resulting in a RPython operation of r_ints. Armin's example
above would again loose the object space instance  because the
"i<j" won't have the "space" context.  

IMO the challenge would be to introduce autowrap/unwrap without confusing
the programmer or reader of the source. currently, i consider

def lt__Int_Int(space, w_i, w_j):
    return space.newbool(space.unwrap(w_i) < space.unwrap(w_j))

as pretty well readable and not too much boilerplate. 

> But for a translated impl. wrapping/unwrapping won't be no-ops.  So the 
> code is probably clearer for the uninitiated as it is. If we want to reduce 
> boilerplate we should add some auto-unwrapping/wrapping logic where it can 
> make sense.

Which kind of contradicts your introductory statement but i agree :-)

cheers,

    holger

From arigo at tunes.org  Mon Jun 16 17:27:48 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 16 Jun 2003 17:27:48 +0200
Subject: [pypy-dev] AbstractObjectSpace
In-Reply-To: <000801c332fd$ad337050$0100a8c0@Lieschen>
References: <20030612122622.GB11354@magma.unil.ch>
	<3EEBDC51.5030408@tismer.com> <000801c332fd$ad337050$0100a8c0@Lieschen>
Message-ID: <20030616152748.GD6532@magma.unil.ch>

Hello G?nter,

On Sun, Jun 15, 2003 at 07:19:19AM +0200, G?nter Jantzen wrote:
> "
> Typically, however, abstract object spaces are a (little) bit less abstract,
> still maintaining a minimal amount of information about the objects. For
> example, a wrapped object could be represented by its type. You then define
> the object space's add to return int when the two arguments are int and int.
> "
> This concept will only work when output types can be determined by input
> types (independent of concrete values).

Yes, the example was a bit rough. One way to make it more complete is by 
giving the following precision: by "type" I did not necessarily mean a type 
object in Python's sense. It could be any kind of subset on the set of all 
Python objects. Thus in your example:

> def spamrelease(i):
>    if i > 0:
>       return i - 1
>    else
>       return "counter is closed"

the return value can be determined to be of a type that is the union of
Python's int and str types. (If you work slightly harder you can determine
that it must be of type "either an integer in range(0, sys.maxint) or the
constant "counter is closed".)

> Consequently we should work with function prototypes. Not as restrictive as
> in C. More a set of rules or patterns for calculating output types from input types.

I had in mind a variable-oriented type inference (using disjunctions in their
types), not a function-oriented one that would analyse several versions of
them, even if the latter might be more precise. But then any would be
possible. It's just that at this point I think that our translator should not
need to emit several versions of the same function for different argument
types.


A bient?t,

Armin.

From arigo at tunes.org  Mon Jun 16 17:48:03 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 16 Jun 2003 17:48:03 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <20030616172244.B21086@prim.han.de>
References: <3EEBDC51.5030408@tismer.com>
	<20030612122622.GB11354@magma.unil.ch> <3EEBDC51.5030408@tismer.com>
	<20030616144154.GB6532@magma.unil.ch>
	<5.2.1.1.0.20030616164914.02475450@pop.bluewin.ch>
	<20030616172244.B21086@prim.han.de>
Message-ID: <20030616154803.GE6532@magma.unil.ch>

Hello Holger,

On Mon, Jun 16, 2003 at 05:22:44PM +0200, holger krekel wrote:
> (...)
> doesn't but instead the "i < j" is typically performed as 
> 
>         space.unwrap(i) < space.unwrap(j)

Now I understand your point about the context in built-ins. Things like "i <
j" automatically unwrapping their arguments are bad ideas. In this specific
example, I was thinking that the argument would enter the function as a plain
int in the first place (or r_int, which is just a Python 2.1-style of int, one
that overflows), not that the " < " operator would know anything about the
W_IntObject class.


Armin

From arigo at tunes.org  Mon Jun 16 17:55:30 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 16 Jun 2003 17:55:30 +0200
Subject: [pypy-dev] Very relevant stuff (was: [pypy-svn] rev 812 - in
	pypy/trunk/src/pypy:interpreter objspace/std objspace/std/test)
In-Reply-To: <200306161456.h5GEuhOs012337@ratthing-b246.strakt.com>
References: <20030615185124.773945A41A@thoth.codespeak.net>
	<3EED13AD.4040509@tismer.com> <20030616141607.GA6532@magma.unil.ch>
	<200306161456.h5GEuhOs012337@ratthing-b246.strakt.com>
Message-ID: <20030616155530.GF6532@magma.unil.ch>

Hello Laura,

On Mon, Jun 16, 2003 at 04:56:43PM +0200, Laura Creighton wrote:
> Is it living someplace on the Wiki now?  or in our doc directory?

As the commit note pointed out it is in src/pypy/objspace/std/.

As your e-mail points out it should better go to doc/ instead. I'm moving it 
right now.

On the Wiki it is at http://codespeak.net/moin/pypy/moin.cgi/MultiMethod.


Armin

From hpk at trillke.net  Mon Jun 16 18:02:22 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 16 Jun 2003 18:02:22 +0200
Subject: [pypy-dev] W_XxxObjects
In-Reply-To: <20030616150927.GC6532@magma.unil.ch>;
	from arigo@tunes.org on Mon, Jun 16, 2003 at 05:09:27PM +0200
References: <20030612122622.GB11354@magma.unil.ch>
	<20030614201652.A21086@prim.han.de> <20030616150927.GC6532@magma.unil.ch>
Message-ID: <20030616180222.C21086@prim.han.de>

[Armin Rigo Mon, Jun 16, 2003 at 05:09:27PM +0200]
> Hello Holger,
> 
> On Sat, Jun 14, 2003 at 08:16:52PM +0200, holger krekel wrote:
> > Interesting. But the second example should still be
> > 
> >     def lt__Int_Int(space, i, j):
> >         return space.newbool(i < j)
> > 
> > because using builtins in restricted python should be avoided, shouldn't it? 
> 
> Not necessarily. See again
> http://codespeak.net/moin/pypy/moin.cgi/RestrictedPython, "builtin functions".

Hmmm.

> > Especially in the presence of multiple object spaces this would loose
> > context.
> 
> No context is needed. The built-in functions we are using here are the real 
> low-level essential stuff, at the same level as the " < " operator between 
> RPython's integers.

But when you want to auto-unwrap (which your above example seems to
imply) how do you do that? 

> These have nothing to do with the functions redefined in module.builtin*; the 
> list of built-in functions we allow in RPython is restricted, and how you use 
> them too, so that they are no burden to the translator. Still, they are 
> nothing more and nothing less than all other operator: we allow some, and 
> disallow some.

Sure. I just thought it's clearer to say RPython avoids "builtins"
alltogether but uses explicit references. E.g.

    def lt__Int_Int(space, w_i, w_j):
        i,j = space.unwrap(w_i, w_j)  # returns a tuple 
        return space.newbool(RPY.lt(i, j))

Thus making it very explicit that there is a "change of level". 
Then again, for all basic RPython types we could easily dynamically
generate the above kind of multimethods reducing the need for repetitive
functions completly.  e.g.

for op in ('lt','le','eq','ge','gt'):    
    for typ in (W_StringObject, W_IntObject, ...):
        def tmp_op(space, w_i, w_j):
            i,j = space.unwrap(w_i, w_j)  
            rpy_op = getattr(RPY, op)
            return space.newbool(rpy_op(i, j))
        getattr(StdObjSpace, op).register(tmp_op, typ, typ)

would reduce boilerplate quite a bit. Probably we would like
to make something like this another "tool" function which can 
be invoked from intobject.py, stringobject.py like

    register_rpython_adapter_operators(W_IntObject), 
        "lt", "le", "eq", "ne", "ge", "gt")

or so. 

> > But the W_XXXObject is also used as a tag for a specific implementation of a
> > type. IOW there is no 1:1 relationship of W_XXXObject and W_XXXType,
> > right?
> 
> The names are misleading. There is one W_XXXType, but there might be a lot of 
> classes implementing it. The existence of one particular such class with the 
> exact name W_XXXObject is misleading. What I was suggesting was that a small 
> number of these specific W_XXXObject classes might be equal to a restricted 
> Python type instead of being a class at all.

OK, understood (hopefully). 

> > It seems to me that "restricted python" is largely vapor-ware so i have
> > problems following.  Or am i missing something? 
> 
> This is a most interesting point. I think that trying to actually get
> somewhere in this direction could be a subject of the upcoming sprint. The
> first sprint was (very roughly) the interpreter, the second sprint the
> stdobjspace, let the third one be the third main missing piece of the puzzle
> :-)

sounds like a large design discussion unless you again bring some brilliant 
new idea to the sprint :-)

Btw, apparently all of Pyrex is implemented in Python and generates
C-files which will compile and bind against CPython. It seems to
have its own Compiler (written in Python). 

Wouldn't using Pyrex as a starting point for implementing/generating 
"RPython" make sense?  Later on, we could try to make a PyPyrex which 
generates stand-alone C-files. At least it would again boost the 
"nestedness" of PyPy :-)

Without this starting point i am skeptical for our four-day sprint 
to provide enough time for coming up with C translators/generators etc. 

cheers,

    holger

From mwh at python.net  Mon Jun 16 20:13:24 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 16 Jun 2003 19:13:24 +0100
Subject: [pypy-dev] App-space tests
References: <20CB5902.4FA28EFB.9ADE5C6A@netscape.net>
	<2m4r2ugs12.fsf@starship.python.net>
Message-ID: <2msmq9nb5n.fsf_-_@starship.python.net>

Michael Hudson <mwh at python.net> writes:

> roccomoretti at netscape.net (Rocco Moretti) writes:
>
>> Michael Hudson <mwh at python.net> wrote:
>>
>>>I've had another idea: give unittest_w.TestCase_w a metaclass that
>>>looks for methods called 'test_app_whatever' and arrange for such
>>>methods to be called as objectspace methods.
>>
>> Sounds like a wonderful solution, but I have to confess that my eyes glaze 
>> over when trying to understand metaclasses - I probably should put more of 
>> an effort into learning about them one of these days...
>
> I should probably be worried at the willingness with which I now reach
> for them.
>
>> Is using metaclasses really nessasary?
>
> Heh.  No, probably not.
>
>> Before I realized that metaclasses were involved, I first thought of
>> using a special subclass of TestCase - probably by overloading
>> TestCase.__call__() to appropriately wrap the function before it is
>> run.
>
> Yes, that would probably work.

And it does, after a fashion.  Well, it works fine as far as I can
tell, it's just that running the assertFoo methods in app-space *too*
(what the attached does) isn't ideal.  OTOH, the fact that doing so
*works* is something we can be proud of.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pypy-appspace-tests-2.diff
Type: text/x-patch
Size: 6332 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030616/21319424/attachment.bin>
-------------- next part --------------

The only decision to be made is: do we distinguish between app-space
tests by a naming convention (as attached: any test starting test_app_
runs in app space) or by inheriting from a different TestCase
subclass?  I can't make my mind up which I prefer.

In either case, I want to check something like this in, soon.  The
implementation can be made more elegant/faster in due course.

Cheers,
M.

-- 
  Just point your web browser at http://www.python.org/search/ and
  look for "program", "doesn't", "work", or "my". Whenever you find
  someone else whose program didn't work, don't do what they
  did. Repeat as needed.    -- Tim Peters, on python-help, 16 Jun 1998

From roccomoretti at netscape.net  Tue Jun 17 05:01:14 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Mon, 16 Jun 2003 23:01:14 -0400
Subject: [pypy-dev] App-space tests
Message-ID: <1763B367.2B5D56C5.9ADE5C6A@netscape.net>

Michael Hudson <mwh at python.net> wrote:

>Michael Hudson <mwh at python.net> writes:
>
>> roccomoretti at netscape.net (Rocco Moretti) writes:
>>
>>> Before I realized that metaclasses were involved, I first thought of
>>> using a special subclass of TestCase - probably by overloading
>>> TestCase.__call__() to appropriately wrap the function before it is
>>> run.
>>
>> Yes, that would probably work.
>
>And it does, after a fashion. ?Well, it works fine as far as I can
>tell, it's just that running the assertFoo methods in app-space *too*
>(what the attached does) isn't ideal. ?OTOH, the fact that doing so
>*works* is something we can be proud of.

Glad to hear it works! 

Re: running method in appspace - I hoped that was something that 
was "fixed" with the addition of the PyByteCode Object. I'm assuming (as I 
wasn't at the sprint and haven't heard about it otherwise) that a regular 
bytecode object gets run on the CPython interpreter, and a PyByteCode 
object gets run on the PyPy interpreter. 

Although that's what I assume, I haven't been able to confirm this in the 
code, as I loose track of the control flow after calling ObjSpace.call(). 
(I started tracking from the CALL_FUNCTION opcode in opcode.py)

>The only decision to be made is: do we distinguish between app-space
>tests by a naming convention (as attached: any test starting test_app_
>runs in app space) or by inheriting from a different TestCase
>subclass?  I can't make my mind up which I prefer.

I'd vote for seperate TestCases. App space should be suitably separated 
from interpreter level such that we avoid confusion. This also allows setup
() and teardown() methods to work semi-sanely. It will also give a 
consistent interface to the assert*() methods (whether they are being 
called from interpreter level or app. level.)

>In either case, I want to check something like this in, soon.  The
>implementation can be made more elegant/faster in due course.

I'm +1 on checkin - as you say, it doesn't break anything, and we can 
always improve it later.

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From mwh at python.net  Tue Jun 17 11:52:29 2003
From: mwh at python.net (Michael Hudson)
Date: Tue, 17 Jun 2003 10:52:29 +0100
Subject: [pypy-dev] Re: App-space tests
References: <1763B367.2B5D56C5.9ADE5C6A@netscape.net>
Message-ID: <2msmq9gheq.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> Michael Hudson <mwh at python.net> wrote:
>
>>Michael Hudson <mwh at python.net> writes:
>>
>>> roccomoretti at netscape.net (Rocco Moretti) writes:
>>>
>>>> Before I realized that metaclasses were involved, I first thought of
>>>> using a special subclass of TestCase - probably by overloading
>>>> TestCase.__call__() to appropriately wrap the function before it is
>>>> run.
>>>
>>> Yes, that would probably work.
>>
>>And it does, after a fashion. ?Well, it works fine as far as I can
>>tell, it's just that running the assertFoo methods in app-space *too*
>>(what the attached does) isn't ideal. ?OTOH, the fact that doing so
>>*works* is something we can be proud of.
>
> Glad to hear it works! 
>
> Re: running method in appspace - I hoped that was something that 
> was "fixed" with the addition of the PyByteCode Object. I'm assuming (as I 
> wasn't at the sprint and haven't heard about it otherwise) that a regular 
> bytecode object gets run on the CPython interpreter, and a PyByteCode 
> object gets run on the PyPy interpreter. 

Ah, no.  All functions are somewhat wrapped -- even in the trivial
object space.  app level functions have a (interpreter-level)
func_code attribute of type PyByteCode; interpreter level functions
have a func_code attribute of type PyBuiltinCode.  It's the
func.func_code.eval_code method that determines how a function is
executed.  This means that PyPy's "builtin functions" are much more
introspectable than CPython's builtins -- they have meaningful
func_code.co_argcount & func_code.co_varnames parameters, for
instance.  We think this is a good thing.

I think I've improved the state of builtin function support to allow
running the assertFoo methods as builtins now, so it's a question of
preference now (although my implementation of varargs & so on for
builtin functions may be excessively clever & should probably be run
past someone else...).

> Although that's what I assume, I haven't been able to confirm this in the 
> code, as I loose track of the control flow after calling ObjSpace.call(). 
> (I started tracking from the CALL_FUNCTION opcode in opcode.py)

I recommend reading interpreter/pycode.py, interpreter/extmodule.py
and objspace/trivial.py:newfunction() to get an idea of how this works.

Mmm, this should probably be documented somewhere... but it's not that
hard once you get the idea, IMHO.

>>The only decision to be made is: do we distinguish between app-space
>>tests by a naming convention (as attached: any test starting test_app_
>>runs in app space) or by inheriting from a different TestCase
>>subclass?  I can't make my mind up which I prefer.
>
> I'd vote for seperate TestCases. 

OK, I was leaning that way too.

> App space should be suitably separated from interpreter level such
> that we avoid confusion. This also allows setup () and teardown()
> methods to work semi-sanely. It will also give a consistent
> interface to the assert*() methods (whether they are being called
> from interpreter level or app. level.)

I think the last comment is a bit of a red herring -- you can't
present unittest_w.TestCase_w.assertEquals_w as a builtin method
directly because it doesn't expect a wrapped 'msg' argument (and
similarly for most of the others).

>>In either case, I want to check something like this in, soon.  The
>>implementation can be made more elegant/faster in due course.
>
> I'm +1 on checkin - as you say, it doesn't break anything, and we can 
> always improve it later.

I'll tweak my code to do the "separate subclasses" thing & check
something in later.

Cheers,
M.

-- 
  All programs evolve until they can send email.      -- Richard Letts
  Except Microsoft Exchange.                                    -- Art
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From ale at bang-olufsen.dk  Tue Jun 17 12:17:26 2003
From: ale at bang-olufsen.dk (Anders Lehmann)
Date: Tue, 17 Jun 2003 12:17:26 +0200
Subject: [pypy-dev] Funding
Message-ID: <bcmprl$dhe$1@main.gmane.org>

What is the status for the funding issue ?

Are there some things that we could prepare before the sprint? (Like what
programmes are we aiming at, size of the budget ...)

I think I need to begin priming my boss this week, because the holiday
season sets in after EuroPython, and then it will be august before we are up
to speed again

Kind regards

Anders Lehmann




From hpk at trillke.net  Tue Jun 17 12:45:28 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 17 Jun 2003 12:45:28 +0200
Subject: [pypy-dev] Funding
In-Reply-To: <bcmprl$dhe$1@main.gmane.org>;
	from ale@bang-olufsen.dk on Tue, Jun 17, 2003 at 12:17:26PM +0200
References: <bcmprl$dhe$1@main.gmane.org>
Message-ID: <20030617124527.S21086@prim.han.de>

[Anders Lehmann Tue, Jun 17, 2003 at 12:17:26PM +0200]
> What is the status for the funding issue ?

I am interested in information about this, too :-)

> Are there some things that we could prepare before the sprint? (Like what
> programmes are we aiming at, size of the budget ...)
> 
> I think I need to begin priming my boss this week, because the holiday
> season sets in after EuroPython, and then it will be august before we are up
> to speed again

good point. 

I'd definitely like to discuss future "funding" directions at the
upcoming sprint. One of the problems with funding is that there
is money involved and the question who gets to decide on how it 
is spent.  For anything involving money to work i think
that we should define the group of "core developers" which decides
and agrees on anything involving interfacing with the "money-world". 
Let's try to keep it simple and transparent, aehm .. pythonic :-)

regards,

    holger

From ale at bang-olufsen.dk  Tue Jun 17 13:07:25 2003
From: ale at bang-olufsen.dk (Anders Lehmann)
Date: Tue, 17 Jun 2003 13:07:25 +0200
Subject: [pypy-dev] Re: Funding
References: <bcmprl$dhe$1@main.gmane.org>
Message-ID: <bcmspc$pj1$1@main.gmane.org>

I forgot to introduce myself.

My name is Anders Lehmann, and I am a Pythonist. I ahve been using Python
for fun (and a little profit) for the last three years.

My interest  in PyPy is in order to address the two problems I have when
trying to convince people to use Python.

The first one is speed. Even if I do the traditional Python Speed Dance (use
the development time saved by using Python to do optimizations), I often
feel that I am using a unsupported claim to cover a weakness ( ie it feels
like I am making an excuse ).

The other one is footprint, in terms of program size, memory consumption and
the like. In my company we are mainly doing embedded programming. it is hard
to justify extra ram and rom/flash.

It is my hope that pypy will change that, and I think that my company could
benefit from that.

Allthough I have been lurking around on the newsgroups for three years, I am
pretty sure I wont be able contribute much to the solution on the hard
design parts of PyPy.

Thats why I am pushing for helping with the funding issue.

See you next week

"Anders Lehmann" <ale at bang-olufsen.dk> wrote in message
news:bcmprl$dhe$1 at main.gmane.org...
> What is the status for the funding issue ?
>
> Are there some things that we could prepare before the sprint? (Like what
> programmes are we aiming at, size of the budget ...)
>
> I think I need to begin priming my boss this week, because the holiday
> season sets in after EuroPython, and then it will be august before we are
up
> to speed again
>
> Kind regards
>
> Anders Lehmann
>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>




From lac at strakt.com  Tue Jun 17 13:21:22 2003
From: lac at strakt.com (Laura Creighton)
Date: Tue, 17 Jun 2003 13:21:22 +0200
Subject: [pypy-dev] Re: Funding 
In-Reply-To: Message from "Anders Lehmann" <ale@bang-olufsen.dk> 
   of "Tue, 17 Jun 2003 13:07:25 +0200." <bcmspc$pj1$1@main.gmane.org> 
References: <bcmprl$dhe$1@main.gmane.org> <bcmspc$pj1$1@main.gmane.org> 
Message-ID: <200306171121.h5HBLMOs016025@ratthing-b246.strakt.com>


I am working on getting us European Union funding.  I want to discuss
this with you in person asap.  The best way, I am told, to get funding
is to build a coalition between universities and small and medium
sized businesses.  Do you have any experience with this?  The Bang &
Olufsen name will be incredibly useful, in itself. I am hideously busy
today -- can this wait until we meet in person?  I have the OSCON PyPy
paper to submit today (although Vee, the Papers chair, is well aware
that the whole thing will change based on next week).  You all can
tell me what I ought to have said instead ...

I am really looking forward to meeting you.

Laura 

From hpk at trillke.net  Tue Jun 17 14:39:36 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 17 Jun 2003 14:39:36 +0200
Subject: [pypy-dev] sprints goals / restricted python? 
Message-ID: <20030617143936.W21086@prim.han.de>

Hello pypy and hello Armin :-)

ASFAIK your idea of restricted python is to implement a 
TranslationObjSpace which emits C-code. We would e.g. 
run the interpreter (by running e.g. tests) against 
StdObjSpace and use the TranslationObjSpace to generate  
a C-file which is - when compiled and run - our restricted 
interpreter which can execute bytecode. 

I don't know but i think going the TranslationObjSpace way
might turn into complicated code. For example how do you
"detect" a loop from an objectspace's POV? It just sees 
a series of a series of ObjSpace-Method accesses and it seems hard 
to reverse engineer the loop. (I guess you say this loop-unfolding 
is basically a space/time trade-off :-)

please correct me, if/where i am wrong.

Anyway, I think the above approach is pretty ambitious for a three 
to four day sprint but this isn't the only "goal" criterium, of course. 

If i am not mistaken, we don't have any framework to generate
C-code and make it "self-runnable" at the moment.  That's why i 
started thinking about Pyrex which provides a framework (written in 
python) for generating C-code. The input is mostly python-compatible 
with some additional "type" information.  

It seems to me that if we  (focus on)

- make the interpreter record/annotate names with types
  (all names: function names, local names, global names, ...)

- use this "annotation" information to generate Pyrex modules 

- use the pyrexc compiler to generate C-Modules from the generated
  Pyrex modules,

we might get a working (partial-)C-interpreter module by the 
end of the sprint. 

To actually perform the annotation we could use an AnnotationObjSpace
which proxies to the StdObjSpace but additionally annotates "names" 
with the types of the objects beeing referenced. Bascially 
we might just need to "intercept" all

    setattr__ANY_String_ANY

calls so that for a given 

    AnnotationObjSpace.setattr(x,y,z) 

call we roughly perform some hands-on annotation by storing
it in "x.__pypytypemap__" which is a dictionary containing
name-dictionary pairs. the latter dictionary then records
all the types (or even values?) that have ever been associated 
with the name. 

Of course, the above generation series would still require CPython 
(because the generated C file compiles against C-Python). 
But later on, we could

- hack up pyrex to generate C-files which don't require a 
  full CPython anymore

- hack up pyrex to be used with a TranslationObjSpace as a 
  second means to generate the restricted interpreter. 

One of the positive points about this approach is that 
we use existing tools/frameworks (written in python) 
and we can probably split the group on various sub-tasks easily. 

But i am very open to your corrections ...

cheers,

    holger

From pedronis at bluewin.ch  Tue Jun 17 17:31:48 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Tue, 17 Jun 2003 17:31:48 +0200
Subject: [pypy-dev] sprints goals / restricted python? 
In-Reply-To: <20030617143936.W21086@prim.han.de>
Message-ID: <5.2.1.1.0.20030617171735.02505e70@pop.bluewin.ch>

At 14:39 17.06.2003 +0200, holger krekel wrote:
>Hello pypy and hello Armin :-)
>
>ASFAIK your idea of restricted python is to implement a
>TranslationObjSpace which emits C-code. We would e.g.
>run the interpreter (by running e.g. tests) against
>StdObjSpace and use the TranslationObjSpace to generate
>a C-file which is - when compiled and run - our restricted
>interpreter which can execute bytecode.
>
>I don't know but i think going the TranslationObjSpace way
>might turn into complicated code. For example how do you
>"detect" a loop from an objectspace's POV? It just sees
>a series of a series of ObjSpace-Method accesses and it seems hard
>to reverse engineer the loop. (I guess you say this loop-unfolding
>is basically a space/time trade-off :-)
>
>please correct me, if/where i am wrong.

you can use some kind of ObjSpace to do abstract interpretation,
but you can't simply use all the normal interpreter mechanism.

You either start from the AST and there loops and control flow are clear, 
or from the bytecode
and use it as it is as flow graph, each bytecode is a node and there are 
edges from
each bytecode to its successors in case of execution, or reduce it to basic 
blocks etc...

regards. 


From hpk at trillke.net  Tue Jun 17 18:05:16 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 17 Jun 2003 18:05:16 +0200
Subject: [pypy-dev] sprints goals / restricted python?
In-Reply-To: <5.2.1.1.0.20030617171735.02505e70@pop.bluewin.ch>;
	from pedronis@bluewin.ch on Tue, Jun 17, 2003 at 05:31:48PM +0200
References: <20030617143936.W21086@prim.han.de>
	<5.2.1.1.0.20030617171735.02505e70@pop.bluewin.ch>
Message-ID: <20030617180516.Y21086@prim.han.de>

Hello Samuele,

[Samuele Pedroni Tue, Jun 17, 2003 at 05:31:48PM +0200]
> At 14:39 17.06.2003 +0200, holger krekel wrote:
> >Hello pypy and hello Armin :-)
> >
> >ASFAIK your idea of restricted python is to implement a
> >TranslationObjSpace which emits C-code. We would e.g.
> >run the interpreter (by running e.g. tests) against
> >StdObjSpace and use the TranslationObjSpace to generate
> >a C-file which is - when compiled and run - our restricted
> >interpreter which can execute bytecode.
> >
> >I don't know but i think going the TranslationObjSpace way
> >might turn into complicated code. For example how do you
> >"detect" a loop from an objectspace's POV? It just sees
> >a series of a series of ObjSpace-Method accesses and it seems hard
> >to reverse engineer the loop. (I guess you say this loop-unfolding
> >is basically a space/time trade-off :-)
> >
> >please correct me, if/where i am wrong.
> 
> you can use some kind of ObjSpace to do abstract interpretation,
> but you can't simply use all the normal interpreter mechanism.
> 
> You either start from the AST and there loops and control flow are clear, 
> or from the bytecode
> and use it as it is as flow graph, each bytecode is a node and there are 
> edges from
> each bytecode to its successors in case of execution, or reduce it to basic 
> blocks etc...

but this means you can't simply implement a TranslationObjSpace and 
use it to generate the C-code for a restricted Python interpreter, right? 

cheers,

    holger

From pedronis at bluewin.ch  Tue Jun 17 18:39:29 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Tue, 17 Jun 2003 18:39:29 +0200
Subject: [pypy-dev] sprints goals / restricted python?
In-Reply-To: <20030617180516.Y21086@prim.han.de>
References: <5.2.1.1.0.20030617171735.02505e70@pop.bluewin.ch>
 <20030617143936.W21086@prim.han.de>
 <5.2.1.1.0.20030617171735.02505e70@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20030617182605.02505e70@pop.bluewin.ch>

At 18:05 17.06.2003 +0200, holger krekel wrote:

>but this means you can't simply implement a TranslationObjSpace and
>use it to generate the C-code for a restricted Python interpreter, right?


you should also change the execution engine to work somehow like psyco, or 
some more static approach. 


From lac at strakt.com  Tue Jun 17 20:33:59 2003
From: lac at strakt.com (Laura Creighton)
Date: Tue, 17 Jun 2003 20:33:59 +0200
Subject: [pypy-dev] helps if i have a pypy-dev alias ....
Message-ID: <200306171833.h5HIXxeG017502@ratthing-b246.strakt.com>

Date:    Tue, 17 Jun 2003 20:12:23 +0200
To:      pypy-dev at ratthing-b246.strakt.com
cc:      lac at ratthing-b246.strakt.com
From:    Laura Creighton <lac at ratthing-b246.strakt.com>
Subject: OSCON paper

Return-Path: <lac at ratthing-b246.strakt.com>

I just put my first draft of the OSCON paper up on the Wiki.
It is very rough.  I am not happy with it.  It contains lots of
things from all over the Wiki, as well as some things that I wrote
myself.  I still don't know what it is that anybody wants me to say
at OSCON, so I sort of threw this together, and it shows.

We can edit this like crazy up until the day I give the talk
(July 9), <well, we could afterwards, but what is the point>.

Some of the paper is just ugly because I don't know how to make
code look like code.  Variable names look like they ought to
be links, but I decided not to change that since I want the
original version in the text I send to OSCON.

If anybody wants to just hack it up and make it better, feel
free.  If you just want to send me suggestions on how _I_
can make it better, that is fine as well.

If you want it on some different part of the wiki, less
prominent -- that is probably a good idea as well, but
i didn't know where would be a good place to put it.

take care,
Laura

From lac at strakt.com  Tue Jun 17 20:58:07 2003
From: lac at strakt.com (Laura Creighton)
Date: Tue, 17 Jun 2003 20:58:07 +0200
Subject: [pypy-dev] Oscon Paper
Message-ID: <200306171858.h5HIw7pW017576@ratthing-b246.strakt.com>

Holger had a better idea.  Share it in the doc directory.
it is now in http://codespeak.net:8080/svn/pypy/trunk/doc/oscon2003-paper.txt
and the wiki points to it.

Oddness -- when i did an scn ci in the doc directory, after adding
it, it said that builtins.txt changed.  That is just the file where
we listed whether the builtins were hard or messy or types and the
like.  I can't believe that I changed it since the sprint ....

If somebody changed it, I guess there is a chance that I klobbered it
with an old version, because it looks pretty much as I expect it to..

Laura


From hpk at trillke.net  Wed Jun 18 12:26:41 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 18 Jun 2003 12:26:41 +0200
Subject: [pypy-dev] about to check in refactored test-framework
Message-ID: <20030618122641.J21086@prim.han.de>

hello pypy,

since yesterday evening i cleaned up the all of our test stuff.

I haven't checked it in, yet. I'd like to know if anybody 
has any strong feelings about the way it currently is...

And here is how the refactored test stuff works:

- all the testsupport, testcts, testwice and all the code duplications
  that came with it are gone. 

- not need to set any PYTHONPATH anymore to start a test or
  interpreter/interactive.py or interpreter/main.py 

- you can start any test by simply calling it directly via

    python test_XXX.py 

- you can run all tests by 

    python test_all.py  # in the root directory

- you can run all tests below your current working dir
  say e.g. in interpreter/

    python ../test_all.py  # will start from the current dir

- you can run all tests in one object space via

    python test_all.py -S  # standard objectspace
    python test_all.py -T  # trivial objectspace
    python test_all.py -ST # first standard, then trivial objectspace

- you can pass some regular expressions 

    python test_all.py pypy.interpreter # will only run tests whose path
                                          matches pypy.interpreter

    python test_all.py -S "%pypy.(obj|app)space"
    # will avoid objspace and appspace tests and otherwise run with
    # stdobjspace, the '%' inverts the match, btw 

- autopath configuration. 

  every startable script (the tests among them, but also
  interpreter/interactive.py interpreter/main.py) first does a

    import autopath 

  which configures the python path to include the directory
  which contains 'pypy'. moreover autopath provides some
  attributes 

    pypydir   the 'pypy' directory path itself 
    this_dir  the directory where your "autopath" lives
              useful for finding test data

  if you run pypy/tool/autopath.py then this will copy itself
  over all other 'autopath.py' instances and report "sync" status. 
  No ugly shell scripting anymore. OTOH autopath.py will hardly 
  need to be modified anymore because it just does path manipulation.

- every test_*.py usually does

    import autopath
    from pypy.tool import test

  and derives from test.TestCase or test.AppTestCase and 
  on __name__=='__main__' invokes
    
    test.main()

  to get to the "cmdline switched" objectspace you call

    test.objspace()

  to get to a specific objectspace you call
    
    test.objspace('std') or test.objspace('trivial')

Some stuff regarding running tests successively in StdObjSpace and
TrivialObjSpace (switch "-ST") is not completly right at the moment. But this
is not so much a thing of the test framework, i think. 

As this checkin involves a lot of files i'd like
to do the checkin ASAP to avoid conflicts. 

Any oppinions? 

regards,

    holger

From mwh at python.net  Wed Jun 18 14:38:21 2003
From: mwh at python.net (Michael Hudson)
Date: Wed, 18 Jun 2003 13:38:21 +0100
Subject: [pypy-dev] Re: about to check in refactored test-framework
References: <20030618122641.J21086@prim.han.de>
Message-ID: <2msmq7zhky.fsf@starship.python.net>

holger krekel <hpk at trillke.net> writes:

> hello pypy,
>
> since yesterday evening i cleaned up the all of our test stuff.

Cool!

> I haven't checked it in, yet. I'd like to know if anybody 
> has any strong feelings about the way it currently is...

I have strong feelings about the way it currently is: it sucks.

And I say that as the person most responsible for the current
situation.

> And here is how the refactored test stuff works:
>
> - all the testsupport, testcts, testwice and all the code duplications
>   that came with it are gone. 

Is the functionality of testcts.py still present?

> - not need to set any PYTHONPATH anymore to start a test or
>   interpreter/interactive.py or interpreter/main.py 

It wasn't necessary for main.py already.  But yay!

> - you can start any test by simply calling it directly via
>
>     python test_XXX.py 

This was true already, no?

> - you can run all tests by 
>
>     python test_all.py  # in the root directory

Do you mean python test/all.py?  (From what you said in your checkin
message).

> - you can run all tests below your current working dir
>   say e.g. in interpreter/
>
>     python ../test_all.py  # will start from the current dir

I like this less, I'd prefer it if running the test_all.py script ran
all the tests in the tree independently of where you start it from.

> - you can run all tests in one object space via
>
>     python test_all.py -S  # standard objectspace
>     python test_all.py -T  # trivial objectspace
>     python test_all.py -ST # first standard, then trivial objectspace

It would be nice if the last option only reran the tests that would
actually run differently under the different object spaces.

Have you cleaned up how you switch main.py/interactive.py select their
object space too?

> - you can pass some regular expressions 
>
>     python test_all.py pypy.interpreter # will only run tests whose path
>                                           matches pypy.interpreter
>
>     python test_all.py -S "%pypy.(obj|app)space"
>     # will avoid objspace and appspace tests and otherwise run with
>     # stdobjspace, the '%' inverts the match, btw 

Well, OK.  This is not a feature I'd have thought of, or asked for,
but we'll see if it turns out to be sueful.

> - autopath configuration. 
>
>   every startable script (the tests among them, but also
>   interpreter/interactive.py interpreter/main.py) first does a
>
>     import autopath 
>
>   which configures the python path to include the directory
>   which contains 'pypy'. moreover autopath provides some
>   attributes 
>
>     pypydir   the 'pypy' directory path itself 
>     this_dir  the directory where your "autopath" lives
>               useful for finding test data
>
>   if you run pypy/tool/autopath.py then this will copy itself
>   over all other 'autopath.py' instances and report "sync" status. 
>   No ugly shell scripting anymore. OTOH autopath.py will hardly 
>   need to be modified anymore because it just does path manipulation.

Cool.  Something autopath should perhaps do as well: remove
sys.path[0].  At the moment if you try to run (say)
pypy/interpreter/main.py under Python 2.3 the presence of the
pypy/interpreter directory on sys.path causes problems, as in 2.3
dis.py wants to be able to find a standard library module called
'opcode'...

But then autopath.py needs to act differently iff it is the first
autopath to be imported[*].  Maybe autopath can just ensure that no
directory downtree of pypydir is on sys.path.

[*] Actually, maybe not.  Let me think about this one for a bit.

> - every test_*.py usually does
>
>     import autopath
>     from pypy.tool import test
>
>   and derives from test.TestCase or test.AppTestCase and 
>   on __name__=='__main__' invokes
>     
>     test.main()
>
>   to get to the "cmdline switched" objectspace you call
>
>     test.objspace()
>
>   to get to a specific objectspace you call
>     
>     test.objspace('std') or test.objspace('trivial')
>
> Some stuff regarding running tests successively in StdObjSpace and
> TrivialObjSpace (switch "-ST") is not completly right at the moment. But this
> is not so much a thing of the test framework, i think. 

Ah :-)

> As this checkin involves a lot of files i'd like
> to do the checkin ASAP to avoid conflicts. 
>
> Any oppinions? 

Yes: check it in already.

Cheers,
M.

-- 
  Any form of evilness that can be detected without *too* much effort
  is worth it...  I have no idea what kind of evil we're looking for
  here or how to detect is, so I can't answer yes or no.
                                       -- Guido Van Rossum, python-dev


From arigo at tunes.org  Wed Jun 18 15:42:42 2003
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 18 Jun 2003 15:42:42 +0200
Subject: [pypy-dev] Oscon Paper
In-Reply-To: <200306171858.h5HIw7pW017576@ratthing-b246.strakt.com>
References: <200306171858.h5HIw7pW017576@ratthing-b246.strakt.com>
Message-ID: <20030618134242.GA16021@magma.unil.ch>

Hello Laura,

On Tue, Jun 17, 2003 at 08:58:07PM +0200, Laura Creighton wrote:
> it is now in http://codespeak.net:8080/svn/pypy/trunk/doc/oscon2003-paper.txt

The paper looks good -- I guess I will steal a couple of ideas for my own 
EuroPython presentation if you would allow me to do so :-)


Armin

From arigo at tunes.org  Wed Jun 18 15:49:04 2003
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 18 Jun 2003 15:49:04 +0200
Subject: [pypy-dev] Oscon Paper
In-Reply-To: <200306171858.h5HIw7pW017576@ratthing-b246.strakt.com>
References: <200306171858.h5HIw7pW017576@ratthing-b246.strakt.com>
Message-ID: <20030618134904.GB16021@magma.unil.ch>

Hello Laura,

On Tue, Jun 17, 2003 at 08:58:07PM +0200, Laura Creighton wrote:
> Oddness -- when i did an scn ci in the doc directory, after adding
> it, it said that builtins.txt changed.

According to svn log and svn diff you just changed the annotation for range() 
from 'n' (needs to be done) to 'd' (done), which is a good thing.


Armin

From lac at strakt.com  Wed Jun 18 16:01:00 2003
From: lac at strakt.com (Laura Creighton)
Date: Wed, 18 Jun 2003 16:01:00 +0200
Subject: [pypy-dev] Oscon Paper 
In-Reply-To: Message from Armin Rigo <arigo@tunes.org> 
	of "Wed, 18 Jun 2003 15:42:42 +0200."
	<20030618134242.GA16021@magma.unil.ch> 
References: <200306171858.h5HIw7pW017576@ratthing-b246.strakt.com>
	<20030618134242.GA16021@magma.unil.ch> 
Message-ID: <200306181401.h5IE10Os021033@ratthing-b246.strakt.com>

In a message of Wed, 18 Jun 2003 15:42:42 +0200, Armin Rigo writes:
>Hello Laura,
>
>On Tue, Jun 17, 2003 at 08:58:07PM +0200, Laura Creighton wrote:
>> it is now in http://codespeak.net:8080/svn/pypy/trunk/doc/oscon2003-pap
>er.txt
>
>The paper looks good -- I guess I will steal a couple of ideas for my own
> 
>EuroPython presentation if you would allow me to do so :-)
>
>
>Armin

Of _course_ I don't mind!  Let me know what else should go in there ...
or modify it yourself, if you are so inclined.

Laura

From hpk at trillke.net  Wed Jun 18 16:02:32 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 18 Jun 2003 16:02:32 +0200
Subject: [pypy-dev] Re: about to check in refactored test-framework
In-Reply-To: <2msmq7zhky.fsf@starship.python.net>;
	from mwh@python.net on Wed, Jun 18, 2003 at 01:38:21PM +0100
References: <20030618122641.J21086@prim.han.de>
	<2msmq7zhky.fsf@starship.python.net>
Message-ID: <20030618160232.N21086@prim.han.de>

Hi Michael,

i just checked it in (revision 830 i guess) with some
modifications resulting from your comments. 

I am probably going to refactor some of the cmdline-arg
processing out to it's own tool. Currently all the
tests and scripts (interactive.py/main.py) accept 
the '-S' and '-T' parameter to run on StdObjSpace resp.
TrivialObjSpace. I thought some uniformity can't hurt. 

[Michael Hudson Wed, Jun 18, 2003 at 01:38:21PM +0100]
> holger krekel <hpk at trillke.net> writes:
> 
> > And here is how the refactored test stuff works:
> >
> > - all the testsupport, testcts, testwice and all the code duplications
> >   that came with it are gone. 
> 
> Is the functionality of testcts.py still present?

Just reintegrated it. It's only there to catch all output during
test runs and print some own summary, right?  Maybe there is an
easier way because i introduced our own TestCase/TestSuite, anyway.

I wonder if we need unittest at all. I somehow don't like the 
unittest design too much. It's so java-ish :-)
 
> > - not need to set any PYTHONPATH anymore to start a test or
> >   interpreter/interactive.py or interpreter/main.py 
> 
> It wasn't necessary for main.py already.  But yay!

Right. i just mentioned it so people now what they can
do with the current stuff. 

> > - you can start any test by simply calling it directly via
> >
> >     python test_XXX.py 
> 
> This was true already, no?

Mostly. Some tests were broken IIRC. 

> > - you can run all tests by 
> >
> >     python test_all.py  # in the root directory
> 
> Do you mean python test/all.py?  (From what you said in your checkin
> message).

I decided against a "test" directory for the moment. 'test.py' is
now in 'tool' and that's why you import it by

    from pypy.tool import test

I did this because i wanted an obvious "test_all" file in the
root directory and didn't see a real need for a new directory "test"
or "testing" so far.  We are using subversion so this can easily 
be moved (apart from all the imports breaking :-). 

> > - you can run all tests below your current working dir
> >   say e.g. in interpreter/
> >
> >     python ../test_all.py  # will start from the current dir
> 
> I like this less, I'd prefer it if running the test_all.py script ran
> all the tests in the tree independently of where you start it from.

OK, changed. Now there is a command line switch "-r" which runs only
the tests relative to the current dir. Default is "run all". 

> > - you can run all tests in one object space via
> >
> >     python test_all.py -S  # standard objectspace
> >     python test_all.py -T  # trivial objectspace
> >     python test_all.py -ST # first standard, then trivial objectspace
> 
> It would be nice if the last option only reran the tests that would
> actually run differently under the different object spaces.

There is some brain work and some more refactoring needed to make
this happen, i am afraid. 

> Have you cleaned up how you switch main.py/interactive.py select their
> object space too?

yes, i think so. 

> > - you can pass some regular expressions 
> >
> >     python test_all.py pypy.interpreter # will only run tests whose path
> >                                           matches pypy.interpreter
> >
> >     python test_all.py -S "%pypy.(obj|app)space"
> >     # will avoid objspace and appspace tests and otherwise run with
> >     # stdobjspace, the '%' inverts the match, btw 
> 
> Well, OK.  This is not a feature I'd have thought of, or asked for,
> but we'll see if it turns out to be sueful.

maybe not. the idea is from Zope3. maybe it's only useful if we have
more than a 1000 tests :-)

> [*] Actually, maybe not.  Let me think about this one for a bit.
> 
> > - every test_*.py usually does
> >
> >     import autopath
> >     from pypy.tool import test
> >
> >   and derives from test.TestCase or test.AppTestCase and 
> >   on __name__=='__main__' invokes
> >     
> >     test.main()
> >
> >   to get to the "cmdline switched" objectspace you call
> >
> >     test.objspace()
> >
> >   to get to a specific objectspace you call
> >     
> >     test.objspace('std') or test.objspace('trivial')
> >
> > Some stuff regarding running tests successively in StdObjSpace and
> > TrivialObjSpace (switch "-ST") is not completly right at the moment. But this
> > is not so much a thing of the test framework, i think. 
> 
> Ah :-)

I think i just fixed it. Obviouly the WrappedFunc thing in unittest_w is
stateful. I now check if the testMethod isn't already wrapped and skip
another wrapping try. 

cheers,

    holger

From arigo at tunes.org  Wed Jun 18 16:11:27 2003
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 18 Jun 2003 16:11:27 +0200
Subject: [pypy-dev] sprints goals / restricted python?
In-Reply-To: <20030617143936.W21086@prim.han.de>
References: <20030617143936.W21086@prim.han.de>
Message-ID: <20030618141126.GC16021@magma.unil.ch>

Hello Holger,

On Tue, Jun 17, 2003 at 02:39:36PM +0200, holger krekel wrote:
> I don't know but i think going the TranslationObjSpace way
> might turn into complicated code.

Actually, you describe a nice AnnotationObjSpace that is almost exactly what I
had in mind. In other words I don't think there is much more to the
AnnotationObjSpace until we can make it produce usable C code. (Btw I'm not
sure emitting Pyrex-annotated code would be any different nor particularly
easier.)

This is the crucial example:

> For example how do you
> "detect" a loop from an objectspace's POV? It just sees 
> a series of a series of ObjSpace-Method accesses and it seems hard 
> to reverse engineer the loop.

Here is the idea. You are right in that the object space cannot see that loop
at all; but we don't necessarily want to produce a nice C-style 'for' loop for
it -- who said we wanted to produce *nice* C code ;-)

Instead, the translator keeps variable type annotations like you described for 
AnnotationObjSpace, but it keeps one such structure for *each* bytecode 
position. Hoping that this doesn't competely blow away the memory when the 
translator is run, we can detect when the annotation work is done by the fact 
that we have got back to an already-seen position and our current annotations 
for it at not more general than what we have already recorded.

Translating the result to C is then quite straightforward. The object space's
methods like space.add(), when called, would attach one or a few lines of C
code to the current bytecode position. When the annotation pass completes, we
just emit a C function whose body groups all these lines in the bytecode
order.

For jumps (like loops), we add in the body a label for each bytecode position
(e.g. 'b123:' for the bytecode position 123) and then the C equivalent of
'JUMP_IF_TRUE 123' is 'if (...) goto b123;'. It doesn't make nice C loops --
looks more like assembly code. Similarily I don't imagine that Python "if"  
constructs will give nice "if(...) {...} else {...}" in C, but again some
horrible block-less stuff with gotos.

Actually getting nice C code would be another matter, but I don't really mind
right now.


A bient?t,

Armin.

From hpk at trillke.net  Wed Jun 18 17:02:23 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 18 Jun 2003 17:02:23 +0200
Subject: [pypy-dev] sprints goals / restricted python?
In-Reply-To: <20030618141126.GC16021@magma.unil.ch>;
	from arigo@tunes.org on Wed, Jun 18, 2003 at 04:11:27PM +0200
References: <20030617143936.W21086@prim.han.de>
	<20030618141126.GC16021@magma.unil.ch>
Message-ID: <20030618170223.O21086@prim.han.de>

[Armin Rigo Wed, Jun 18, 2003 at 04:11:27PM +0200]
> Hello Holger,
> 
> On Tue, Jun 17, 2003 at 02:39:36PM +0200, holger krekel wrote:
> > I don't know but i think going the TranslationObjSpace way
> > might turn into complicated code.
> 
> Actually, you describe a nice AnnotationObjSpace that is almost exactly what I
> had in mind. In other words I don't think there is much more to the
> AnnotationObjSpace until we can make it produce usable C code. (Btw I'm not
> sure emitting Pyrex-annotated code would be any different nor particularly
> easier.)

It seems to me that 

a) Pyrex would allow for a smoother transition to RPython because
   we could run parts of our interpreter as C-modules. If we have
   the whole running we can migrate to our own RPython-C-base library. 

b) Pyrex' extended python syntax seems to be easier to generate,
   especially starting from annoated python-source 

Btw, i have no assets with Pyrex and only glanced at documentation+src
without using it much. 
 
> This is the crucial example:
> 
> > For example how do you
> > "detect" a loop from an objectspace's POV? It just sees 
> > a series of a series of ObjSpace-Method accesses and it seems hard 
> > to reverse engineer the loop.
> 
> Here is the idea. You are right in that the object space cannot see that loop
> at all; but we don't necessarily want to produce a nice C-style 'for' loop for
> it -- who said we wanted to produce *nice* C code ;-)

... space-time tradeoff, i knew it :-) ...

> Instead, the translator keeps variable type annotations like you described for 
> AnnotationObjSpace, but it keeps one such structure for *each* bytecode 
> position. Hoping that this doesn't competely blow away the memory when the 
> translator is run, we can detect when the annotation work is done by the fact 
> that we have got back to an already-seen position and our current annotations 
> for it at not more general than what we have already recorded.
> 
> Translating the result to C is then quite straightforward. The object space's
> methods like space.add(), when called, would attach one or a few lines of C
> code to the current bytecode position. When the annotation pass completes, we
> just emit a C function whose body groups all these lines in the bytecode
> order.

ok, now i understand it more. I am not sure if starting from the interpreter 
source would be as/less/more efficient, though :-) Any guesses? 
 
> For jumps (like loops), we add in the body a label for each bytecode position
> (e.g. 'b123:' for the bytecode position 123) and then the C equivalent of
> 'JUMP_IF_TRUE 123' is 'if (...) goto b123;'. It doesn't make nice C loops --
> looks more like assembly code. Similarily I don't imagine that Python "if"  
> constructs will give nice "if(...) {...} else {...}" in C, but again some
> horrible block-less stuff with gotos.

sounds like psyco :-)
 
> Actually getting nice C code would be another matter, but I don't really mind
> right now.

who would. It's nice that we agree about the AnnotationObjSpace so this
is a starting point.  Are you in Louvain-La-Neuve on Friday evening 
or are you staying in Bruxelles? 

cheers,

    holger

From guenter.jantzen at netcologne.de  Thu Jun 19 13:33:37 2003
From: guenter.jantzen at netcologne.de (=?iso-8859-1?Q?G=FCnter_Jantzen?=)
Date: Thu, 19 Jun 2003 13:33:37 +0200
Subject: [pypy-dev] Ideas and Opinions
Message-ID: <001901c33656$a13a8480$0100a8c0@Lieschen>

In the moment my pypy understanding is at the Novice Level
and all my comments are a bit on the surface "Hey Prof! You forgot a closing paranthese".

So I try to browse ideas and opinions:

1)PyPy is not only about Python. Its about compiling and interpreting - translating and executing code.
This two approaches are necessary for every programming language. Translating, because a programming 
language is designed for humans. Executing, because the behaviour of a programm is not decidable.
So even compiled languages are eventually executed on a real machine 
(maybe with some intermediate translation/execution  on virtual machines)

At compile time we try to collect all informations which are necessary in the target environment/execution context.
For example type information in static typed languages, because the target language (assembler) makes a differece 
in adding ints or floats (as I remember from Armins lecture last year)

In Python progamms is not enough type information at compile time, so CPython?s target is not Assembler on a CPU, but 
Python bytecode on a Python virtual machine. CPython interprets this bytecode at runtime with some opcode machinery
PyPy does an abstract interpretation in several ObjectSpaces

Usually execution and translation work on different data structures at different times. 
Translation works at compile time at the static data structure of an abstract syntax tree (given by the parser).
Execution works at runtime on the dynamic sequence of instructions (given by the interpration of bytecode) 
which is the input for abstract interpretation.

If we want to do some magic translation/execution entanglement we 
have to think about doing the right thing at the right time (several times of execution and translation)
and about mappings between our dynamic and static data structures.

For example in interpretation code we enter a loop which we want to optimize.
We can?t do this earlier without receiving a big memory overhead, because earlier we havent 
got the necessary type informations.

So we follow the mapping from our instruction to the the abstract syntax tree, set type labels at some nodes and 
do some local translation and optimization. As a result we receive (local?) changed bytecode for the body of our loop. Give it
to the interpreter and execute it.

This hot spot approach will give us the freedom not to calulate all execution pathes of a programm, 
but only the actual necessary ones. Another approach to filter only important pathes could be in combination with a
test framework. Execute all your tests and cache the dynamic generated code in a database. When your
programm is still slow, you forgot maybe some tests.


2) At a first glance, Pyrex seems to me a beauty example for a concrete syntax which is not in the way of 
the programmer. (of course its more than that). Pyrex can be annotated with C-style static informations. 
But it is not necessary to do this. You can program without annotations. Then its just Python. If you
use annotations, Pyrex generates the C-source for extension types

------example---------
def primes(int kmax):
  cdef int n, k, i
  cdef int p[1000]
  result = []
  if kmax > 1000:
    kmax = 1000
  k = 0
  n = 2
  while k < kmax:
    i = 0
    while i < k and n % p[i] <> 0:
      i = i + 1
    if i == k:
      p[k] = n
      k = k + 1
      result.append(n)
    n = n + 1
  return result
---------------------

If we add some concrete syntax in pypy, we will not only optimize the language core. 
We will give the Application programmer the opportunity to choose her codestyle in a scale from dynamic to static.

For example, all modules of the Python Standard Library should be written as static as possible.

G?nter

P.S: comments and clearing of misunderstandings are welcome







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030619/a2b08db3/attachment.html>

From hpk at trillke.net  Thu Jun 19 16:21:58 2003
From: hpk at trillke.net (holger krekel)
Date: Thu, 19 Jun 2003 16:21:58 +0200
Subject: [pypy-dev] temporary downtimes today (thursday)
Message-ID: <20030619162158.U21086@prim.han.de>

hello pypy, 

the codespeak apache2 setup will be enhanced today. The goal
is to have everything (instead of just subversion) running on apache2. 
If we get this running then the subversion url will eventually be

    http://codespeak.net/svn 
                      ^^^^
                note the "missing" :8080

We will also change ownerships and permissions of apache2 around 
so you might temporarily be unable to access codespeak.net:8080
(current subversion url). We'll report when we are finished. 

    holger & jum

From hpk at trillke.net  Thu Jun 19 20:33:23 2003
From: hpk at trillke.net (holger krekel)
Date: Thu, 19 Jun 2003 20:33:23 +0200
Subject: [pypy-dev] finished apache2 upgrade
Message-ID: <20030619203323.Z21086@prim.han.de>

hello pypy,

the upgrade to apache2 is completed. So from now on you can
access the subversion repository via

    http://codespeak.net/svn/pypy/trunk

and don't need the ":8080" any more. Neverthelss,  port 8080
is served by the same apache2 so that continues to
work. We will fade out port 8080 end of next week, so don't 
hesitate to "recheckout" pypy.  There is no way to switch URL's in
a checked out directory except if you run a very new svn-client
(option "relocate" or so) So you have to check it all out again... 
sorry for the inconvenience. 

Also we automatized some stuff like changing passwords 
with "passwd" on shell-level should change your svn-password.
Plus some other stuff which you probably don't want to know right now.

cheers,

    holger & jum

From tismer at tismer.com  Thu Jun 19 20:41:26 2003
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 19 Jun 2003 20:41:26 +0200
Subject: [pypy-dev] Ideas and Opinions
In-Reply-To: <001901c33656$a13a8480$0100a8c0@Lieschen>
References: <001901c33656$a13a8480$0100a8c0@Lieschen>
Message-ID: <3EF203D6.9060701@tismer.com>

G?nter Jantzen wrote:

...

> 2) At a first glance, Pyrex seems to me a beauty example for a concrete 
> syntax which is not in the way of
> the programmer. (of course its more than that). Pyrex can be annotated 
> with C-style static informations.
> But it is not necessary to do this. You can program without annotations. 
> Then its just Python. If you
> use annotations, Pyrex generates the C-source for extension types

Ok, at this point, the role of restricted Python comes
into the play. It may be everything can be done without
restrictions, but we wanted a simple start-off.
Restricted Python is significally simplified at certain
places, which allows for a lot of early deductions.
There are assumptions about basic types being used
and their behavior, so we don't need type hints.

Then! This restricted Python is now supposed to run
only *one* single program: The new interpreter.
That means, we can analyse the whole program as a
constant thing, which is to be executed by restricted
Python. That should allow us do deduce every type
used whatsoever, and we should be able to produce
C code of similar quality like CPython's.

If we can't do that, then we did something wrong and
must re-iterate.

so long - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From lac at strakt.com  Thu Jun 19 21:41:18 2003
From: lac at strakt.com (Laura Creighton)
Date: Thu, 19 Jun 2003 21:41:18 +0200
Subject: [pypy-dev] More PyFunding Links
Message-ID: <200306191941.h5JJfIoG026793@ratthing-b246.strakt.com>

I found another page of great links.  Here is the text I
added to the wiki, under PyPyFunding:

Here is a Guide for how to write a proposal. http://europa.eu.int/comm/research
/fp6/index_en.html You want to click on the 'participating in European Research
' links. Available in German French Italian and Spanish as well as English.

Laura

From lac at strakt.com  Thu Jun 19 23:01:01 2003
From: lac at strakt.com (Laura Creighton)
Date: Thu, 19 Jun 2003 23:01:01 +0200
Subject: [pypy-dev] EU funding
Message-ID: <200306192101.h5JL11wa027044@ratthing-b246.strakt.com>

I found another page of great links.  Here is the text I
added to the wiki, under PyPyFunding:

Here is a Guide for how to write a proposal. http://europa.eu.int/comm/research/fp6/index_en.html You want to click on the 'participating in European Research' links. Available in German French Italian and Spanish as well as English.

Laura

From guenter.jantzen at netcologne.de  Thu Jun 19 23:06:57 2003
From: guenter.jantzen at netcologne.de (=?iso-8859-1?Q?G=FCnter_Jantzen?=)
Date: Thu, 19 Jun 2003 23:06:57 +0200
Subject: [pypy-dev] Ideas and Opinions
References: <001901c33656$a13a8480$0100a8c0@Lieschen>
	<3EF203D6.9060701@tismer.com>
Message-ID: <003d01c336a6$b95bd460$0100a8c0@Lieschen>

Hallo Christian,

>
> > 2) At a first glance, Pyrex seems to me a beauty example for a concrete
> > syntax which is not in the way of
> > the programmer. (of course its more than that). Pyrex can be annotated
> > with C-style static informations.
> > But it is not necessary to do this. You can program without annotations.
> > Then its just Python. If you
> > use annotations, Pyrex generates the C-source for extension types
>
> Ok, at this point, the role of restricted Python comes
> into the play. It may be everything can be done without
> restrictions, but we wanted a simple start-off.

Of course, for this reason my annotation was labelled with a 2.
I very agree with the approach to restrict Python. Not only for simplicity
Its necessary for translating the interpreter.

And in the example above we can see this. Pyrex generates Python plus
C-Sources for extension types from annotated Python
But this happens only when the input is really restricted (=annotated  in
this example)
If there are just Python Sources, Pyrex does not generate anything:
output = input = Python.

> Restricted Python is significally simplified at certain
> places, which allows for a lot of early deductions.
> There are assumptions about basic types being used
> and their behavior, so we don't need type hints.

Yes we dont need a concrete syntax now.
Its all expressed in the behaviour of our restricted coding.

Or better it should be. Its not so easy to find the right set of
restrictions.
Even when we use only a handful of basic types and still allow functions
like "spamrelease" to return what they want - then we can have still a lot
of dynamic.
Take as another example John Osterhouds language TCL.
A very dynamic language with only one datatype -string (maybe they have more
types now).

>
> Then! This restricted Python is now supposed to run
> only *one* single program: The new interpreter.
> That means, we can analyse the whole program as a
> constant thing, which is to be executed by restricted
> Python. That should allow us do deduce every type
> used whatsoever, and we should be able to produce
> C code of similar quality like CPython's.
> If we can't do that, then we did something wrong and
> must re-iterate.
>

Yes, and when we are finished and exhausted after iteration #476
because we have done so much only for receiving something that already
exists then lets speak again about the whole thing

> so long - chris

I hope I havent tired you. Oh, I have to pack my suitcase now
All looks very untyped here.

bye
G?nter


From lac at strakt.com  Fri Jun 20 13:13:38 2003
From: lac at strakt.com (Laura Creighton)
Date: Fri, 20 Jun 2003 13:13:38 +0200
Subject: [pypy-dev] New FP6 Call
Message-ID: <200306201113.h5KBDcA6029574@ratthing-b246.strakt.com>

17 June 2003 -- This week (Tuesday).

http://fp6.cordis.lu/fp6/call_details.cfm?CALL_ID=74#

Click on 'Areas and Instruments Addressed within this Call'

You will get the new proposals, all of which could be of
interest.  But take the time to read

IST-2002-2.3.2.3   Open development platforms for software and services

If I didn't know any better I would think that somebody in the EU knows
all about PyPy and wants to fund us.  I think they are describing
us to a T.

happily astonished,
Laura

From roccomoretti at netscape.net  Fri Jun 20 06:11:34 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Fri, 20 Jun 2003 00:11:34 -0400
Subject: [pypy-dev] Recent Checkins
Message-ID: <76A3F70D.47894873.9ADE5C6A@netscape.net>


First off, mea culpa.

I'm not sure how unpardonable a sin this is, but I now realize that my 
recent checkins make the test_all.py script fail. This (probably) isn't as 
bad as it sounds. Firstly, it works fine for the trivial object space - it 
only has problems with StdObjSpace. Secondly, it's only one file that has 
problems (test_exec.py), and it seems that the problems were there before 
the checkin, it was just that the unittest wasn't picking them up. (It was 
running at interpreter level, not at application level as it should be - 
and no surprise here; CPython handles the exec statement perfectly :)

I'd try to fix the errors, but they seem to be complex, and not directly 
due to the EXEC_STMT opcode - as I said, TrivialObjSpace passes.

You can run test_all for yourself on revision 839+, but here is a synopsis 
of the errors:

* The StdObjSpace dictionary does not have a "has_key" function attribute.
* You get a "Type Error: an integer is required" when executing a code 
    object. This error message occurs twice in the standard obj space
    [objspace.py, concretespace.py], and an unknown number of times in
    CPython.
- We need more specific error messages.
* An exception within an exec statement does not bubble up to the caller.
- AppTestCase needs more specific messages with self.AssertRaises calls. -
    Currently it only gives a blank AssertionError (with no message).
* Don't pass an integer as the local dictionary to the exec statement. 
    (but an integer for the global dict works fine) 

That's about as far as I can go for now. I hope nobody minds that I 
checked in code that gives error messages.  The best I can say in my 
defense is that it was broken before the checkin.

Safe travels to all the sprinters,

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From mwh at python.net  Fri Jun 20 13:28:45 2003
From: mwh at python.net (Michael Hudson)
Date: Fri, 20 Jun 2003 12:28:45 +0100
Subject: [pypy-dev] Re: New FP6 Call
References: <200306201113.h5KBDcA6029574@ratthing-b246.strakt.com>
Message-ID: <2misr1q976.fsf@starship.python.net>

Laura Creighton <lac at strakt.com> writes:

> 17 June 2003 -- This week (Tuesday).
>
> http://fp6.cordis.lu/fp6/call_details.cfm?CALL_ID=74#
>
> Click on 'Areas and Instruments Addressed within this Call'
>
> You will get the new proposals, all of which could be of
> interest.  But take the time to read
>
> IST-2002-2.3.2.3   Open development platforms for software and services
>
> If I didn't know any better I would think that somebody in the EU knows
> all about PyPy and wants to fund us.  I think they are describing
> us to a T.

Hmm, yes.  Is there any way to find out who's behind it?

Ye gods, I hate marketing speak with a passion.

Cheers,
M.

-- 
  at any rate, I'm satisfied that not only do they know which end of
  the pointy thing to hold, but where to poke it for maximum effect.
                                  -- Eric The Read, asr, on google.com


From mwh at python.net  Fri Jun 20 13:37:07 2003
From: mwh at python.net (Michael Hudson)
Date: Fri, 20 Jun 2003 12:37:07 +0100
Subject: [pypy-dev] Re: Recent Checkins
References: <76A3F70D.47894873.9ADE5C6A@netscape.net>
Message-ID: <2mfzm5q8t8.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> First off, mea culpa.
>
> I'm not sure how unpardonable a sin this is, but I now realize that my 
> recent checkins make the test_all.py script fail. This (probably) isn't as 
> bad as it sounds. Firstly, it works fine for the trivial object space - it 
> only has problems with StdObjSpace. Secondly, it's only one file that has 
> problems (test_exec.py), and it seems that the problems were there before 
> the checkin, it was just that the unittest wasn't picking them up. (It was 
> running at interpreter level, not at application level as it should be - 
> and no surprise here; CPython handles the exec statement perfectly :)
>
> I'd try to fix the errors, but they seem to be complex, and not directly 
> due to the EXEC_STMT opcode - as I said, TrivialObjSpace passes.

Can you drop into IRC?  #pypy on irc.openproject.net.  I'm not doing
much right now, we can try and fix this.

Or maybe you have already?  I don't see any failures.

> You can run test_all for yourself on revision 839+, but here is a synopsis 
> of the errors:
>
> * The StdObjSpace dictionary does not have a "has_key" function attribute.
> * You get a "Type Error: an integer is required" when executing a code 
>     object. This error message occurs twice in the standard obj space
>     [objspace.py, concretespace.py], and an unknown number of times in
>     CPython.
> - We need more specific error messages.
> * An exception within an exec statement does not bubble up to the caller.
> - AppTestCase needs more specific messages with self.AssertRaises calls. -
>     Currently it only gives a blank AssertionError (with no message).

I have changes for this that need polishing.  As they're on my peecee
and not my laptop, it would probably be a good idea if I checked them
in before leaving :-)

Cheers,
M.

-- 
  For every complex problem, there is a solution that is simple,
  neat, and wrong.                                    -- H. L. Mencken


From mwh at python.net  Fri Jun 20 13:44:47 2003
From: mwh at python.net (Michael Hudson)
Date: Fri, 20 Jun 2003 12:44:47 +0100
Subject: [pypy-dev] Re: Recent Checkins
References: <76A3F70D.47894873.9ADE5C6A@netscape.net>
	<2mfzm5q8t8.fsf@starship.python.net>
Message-ID: <2md6h9q8gg.fsf@starship.python.net>

Michael Hudson <mwh at python.net> writes:

> roccomoretti at netscape.net (Rocco Moretti) writes:
>
>> First off, mea culpa.
>>
>> I'm not sure how unpardonable a sin this is, but I now realize that my 
>> recent checkins make the test_all.py script fail. This (probably) isn't as 
>> bad as it sounds. Firstly, it works fine for the trivial object space - it 
>> only has problems with StdObjSpace. Secondly, it's only one file that has 
>> problems (test_exec.py), and it seems that the problems were there before 
>> the checkin, it was just that the unittest wasn't picking them up. (It was 
>> running at interpreter level, not at application level as it should be - 
>> and no surprise here; CPython handles the exec statement perfectly :)
>>
>> I'd try to fix the errors, but they seem to be complex, and not directly 
>> due to the EXEC_STMT opcode - as I said, TrivialObjSpace passes.
>
> Can you drop into IRC?  #pypy on irc.openproject.net.  I'm not doing
> much right now, we can try and fix this.
>
> Or maybe you have already?  I don't see any failures.

Oh, now I do.  I wonder what the test was doing there?  I don't
usually run the appspace tests, as some of them take so long.

> I have changes for this that need polishing.  As they're on my peecee
> and not my laptop, it would probably be a good idea if I checked them
> in before leaving :-)

Done -- try running the tests now, I think you'll like the changes :-)

Cheers,
M.

-- 
  > so python will fork if activestate starts polluting it?
  I find it more relevant to speculate on whether Python would fork
  if the merpeople start invading our cities riding on the backs of 
  giant king crabs.                 -- Brian Quinlan, comp.lang.python


From mwh at python.net  Fri Jun 20 14:50:25 2003
From: mwh at python.net (Michael Hudson)
Date: Fri, 20 Jun 2003 13:50:25 +0100
Subject: [pypy-dev] Re: Recent Checkins
References: <76A3F70D.47894873.9ADE5C6A@netscape.net>
Message-ID: <2madccrjzi.fsf@starship.python.net>

roccomoretti at netscape.net (Rocco Moretti) writes:

> I'd try to fix the errors, but they seem to be complex, and not directly 
> due to the EXEC_STMT opcode - as I said, TrivialObjSpace passes.

I have fixed the errors, and you were right :-)

Cheers,
M.

-- 
  Well, you pretty much need Microsoft stuff to get misbehaviours
  bad enough to actually tear the time-space continuum.  Luckily
  for you, MS Internet Explorer is available for Solaris.
                              -- Calle Dybedahl, alt.sysadmin.recovery


From jum at anubis.han.de  Fri Jun 20 15:31:52 2003
From: jum at anubis.han.de (Jens-Uwe Mager)
Date: Fri, 20 Jun 2003 15:31:52 +0200
Subject: [pypy-dev] Subversion 0.24.2 binaries available
Message-ID: <20030620133152.GX2472@anubis>

I have put the source tar ball and binaries for Windows and Mac OS X for
subversion 0.24.2 at:

http://codespeak.net/~jum/

Besides fixing some memory leaks and other bug fixes this includes a new
command to switch the repository URL, which is quite a useful feature.
For example:

svn switch --relocate http://codespeak.net:8080/ http://codespeak.net/

will switch an existing repository at the current working directory from
our old URL to the new one. Simliarly you can use it to switch between
file: and http: URLs and vice versa.

This does not work yet on thoth.codespeak.net itself, as I did not
upgrade the subversion binaries there. I will do that after the sprint
to avoid accidently breaking a working system.
-- 
Jens-Uwe Mager	<pgp-mailto:F476EBC2>

From ale at bang-olufsen.dk  Fri Jun 20 23:06:07 2003
From: ale at bang-olufsen.dk (Anders Lehmann)
Date: Fri, 20 Jun 2003 23:06:07 +0200
Subject: [pypy-dev] Re: New FP6 Call
References: <200306201113.h5KBDcA6029574@ratthing-b246.strakt.com>
Message-ID: <bcvsvo$651$1@main.gmane.org>

Thats the one I mentioned in my email. I didnt find the link you mention
though. (Is it just me or is it very hard to navigate the EU sites).

But you are right PyPy fits the bill perfectly. And if we are able to
convince Bang & Olufsen to join we could justify the point :

    "Priority will be given to projects in which strong industrial users
join forces with software and service suppliers in building common platforms
with support of academic research partners."

See you monday ( late sunday night, I will come by car between 22-23)

Anders Lehmann

"Laura Creighton" <lac at strakt.com> wrote in message
news:200306201113.h5KBDcA6029574 at ratthing-b246.strakt.com...
> 17 June 2003 -- This week (Tuesday).
>
> http://fp6.cordis.lu/fp6/call_details.cfm?CALL_ID=74#
>
> Click on 'Areas and Instruments Addressed within this Call'
>
> You will get the new proposals, all of which could be of
> interest.  But take the time to read
>
> IST-2002-2.3.2.3   Open development platforms for software and services
>
> If I didn't know any better I would think that somebody in the EU knows
> all about PyPy and wants to fund us.  I think they are describing
> us to a T.
>
> happily astonished,
> Laura
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>




From hpk at trillke.net  Sun Jun 29 00:31:27 2003
From: hpk at trillke.net (holger krekel)
Date: Sun, 29 Jun 2003 00:31:27 +0200
Subject: [pypy-dev] website update / pypy developments
Message-ID: <20030629003127.N3869@prim.han.de>

Hi PyPy,

just updated the website 

    http://codespeak.net/pypy

with some information about what happened at the PyPy-Sprint in Louvain-La-Neuve
before EuroPython and at the conference itself.  We got quite some attention at the 
EuroPython 2003 conference where Armin Rigo gave a pygame-based talk and Guido van 
Rossum was repeatedly seen hacking on PyPy. In his keynote he presented
PyPy as a "very ambitious project", adding "It's just fun to work on it". 
OTOH, everyone was making fun of the fact that PyPy runs 20000 times slower 
than CPython :-)

To bring everybody who is interested up-to-date with recent developments and 
discussions i am giving a bit of an overview on current developments. 

- the interpreter is now in pretty good shape. It works nicely with
  our three object spaces:

    Trivial    dispatches object operations quite directly CPython objects.

    Standard   implements the standard python type system (and methods)
               including Generators, __new__ methods and 
               metaclasses. 

    Annotation implements "abstract interpretation" which heads
               for inferencing types for function parameters 
               and return values of a complete python program. 
               this space will form the basis of the soon-to-come
               code-generator. 

  which doesn't mean that there is not lots of refactoring and 
  improvements on the current code base to do. 

- we decided to put our evolving documentation into the subversion 
  respository instead of the wiki. This allows for more consistent
  documents and notifications about changes, versioning etc. 
  Anna has done quite some work in that area.  We are using the 
  RestructeredText text format which is rendered into html 
  on the codespeak server whenever documentation changes. 

- Laura is going forward to present PyPy at the OSCON 2003 and she
  together with help from other developers improved the paper. 
  IIUC the paper should be transformed into ReST and further
  improved (i am somewhat starting to do that). 

- we discussed going for EU-funding. The general oppinion was
  that the current Call for Proposals fits the possible targets
  of PyPy pretty well. But going for EU-funding is by far a simple
  task. We have already talked to some companies and are also looking
  for research institutions that would want to participate in a 
  big EU-Project. After some discussions especially with Armin 
  i now think that we should promote as a goal to 

    dynamically tie into/adapt arbritrary C-Libraries and
    operation system calls. 

  The latter would basically mean that you could use Python directly to 
  drive your favourite OS. Think rapidly specializing for whatever 
  embedded device without even a libc and doing that with 99% of the 
  code beeing written in Python. 

  However, we only have three month to fullfill the EU-funding 
  conditions. If we want to involve larger companies then we have
  to be very quick because large companies often take month to 
  sign contracts. 

  We are grateful for any hint or person you can recommend in order 
  to support us with matching the EU requirements. In return, we can easily 
  help you to envision usages of PyPy for your company or research
  institution :-)

- the EU-funding will probably make it necessary to form some body 
  representing the pypy developers among companies and researchers. 
  The current idea is to setup a "pypy foundation" or even better 
  "py fundation" which brings together three driving forces: 
  Python, Fun & Funding.  The latest date to found such an 
  organization will be the next sprint (oh dear) ...

- the next PyPy sprint will probably take place in Berlin 
  end of september 2003. It's focus will probably be to emit 
  C-Code in one way or another. After long discussions Samuele,
  Armin and me think that it's a good idea to target Pyrex as
  the first "emitted code" language. It has two advantages:

    - Pyrex handles garbage collection aka hairy details
      of refcounting and is a nice language to generate :-)
      
    - generating runnable CPython modules allows early (and faster!)
      testing. Basically we want to be able to import pypy as an 
      extension module into CPython *taking over* interpretation
      of byte codes. 

- the test framework was improved further but needs more work in various
  areas. Michael and me even considered rewriting the unittest module alltogether 
  because we need a fine-grained flexibilty regarding object spaces.
  Something like a test-coverage tool that tells us which code pathes in
  our programs are not tested. And I'd like a webpage which reports which 
  revision broke which tests and some other nice reports (a nicely
  rendered commit-log for starters). 

- we currently have around 10000 lines of code (LOC) and 4 KLOC of tests.
  it is too early to compare these LOC to CPython's but it's
  nevertheless an interesting number. 

so far for now. 

have fun,

    holger

From hpk at trillke.net  Sun Jun 29 00:50:21 2003
From: hpk at trillke.net (holger krekel)
Date: Sun, 29 Jun 2003 00:50:21 +0200
Subject: [pypy-dev] website update / pypy developments
In-Reply-To: <20030629003127.N3869@prim.han.de>;
	from hpk@trillke.net on Sun, Jun 29, 2003 at 12:31:27AM +0200
References: <20030629003127.N3869@prim.han.de>
Message-ID: <20030629005021.O3869@prim.han.de>

[holger krekel Sun, Jun 29, 2003 at 12:31:27AM +0200]
> with some information about what happened at the PyPy-Sprint in Louvain-La-Neuve
> before EuroPython and at the conference itself.

I forgot to thank Benjamin and Godefroid who organized the sprint location 
in Louvain-La-Neuve which is an interesting city, btw. It is basically build on 
parking lots.  With Francoise (correctly spelled?) they also setup a great 
barbicue for us and the Zope3-sprint people.  Thanks!

If anyone has pictures of the sprint or of pypy-related stuff from
EuroPython, please sent them to me to i can intermingle them on the
website. 

cheers, 

    holger

From stephan.diehl at gmx.net  Mon Jun 30 10:24:37 2003
From: stephan.diehl at gmx.net (Stephan Diehl)
Date: Mon, 30 Jun 2003 10:24:37 +0200
Subject: [pypy-dev] website update / pypy developments
In-Reply-To: <20030629003127.N3869@prim.han.de>
References: <20030629003127.N3869@prim.han.de>
Message-ID: <200306301024.37286.stephan.diehl@gmx.net>

On Sonntag, 29. Juni 2003 00:31, holger krekel wrote:
> - we discussed going for EU-funding. The general oppinion was
>   that the current Call for Proposals fits the possible targets
>   of PyPy pretty well. But going for EU-funding is by far a simple
>   task. We have already talked to some companies and are also looking
>   for research institutions that would want to participate in a
>   big EU-Project. After some discussions especially with Armin
>   i now think that we should promote as a goal to
>
>     dynamically tie into/adapt arbritrary C-Libraries and
>     operation system calls.
>
>   The latter would basically mean that you could use Python directly to
>   drive your favourite OS. Think rapidly specializing for whatever
>   embedded device without even a libc and doing that with 99% of the
>   code beeing written in Python.
>

One idea I had (not really worked out) is, to include some security measures 
already on a language level. While this is not really important on small, 
standalone applications (we have the OS security for that), there is 
definatelly an issue with this in a distributed computing environment 
(easiest case: web application server).
Anyway, if you think as well, that security could be part of the project, I'd 
try to get my company (www.articon-integralis.com) to join (Knowing them, I 
wouldn't hold my breath, though).
In addition, I know some people at German Telekom. Maybe they are interested 
as well.

Cheers

Stephan


From mwh at python.net  Mon Jun 30 11:21:30 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 30 Jun 2003 10:21:30 +0100
Subject: [pypy-dev] Re: [pypy-svn] rev 1064 - in pypy/trunk/src/pypy:
	interpreterobjspace/stdobjspace/std/test
In-Reply-To: <20030629104527.54D155AA42@thoth.codespeak.net>
	(hpk@codespeak.net's
	message of "Sun, 29 Jun 2003 12:45:27 +0200 (MEST)")
References: <20030629104527.54D155AA42@thoth.codespeak.net>
Message-ID: <2mr85bc44l.fsf@starship.python.net>

hpk at codespeak.net writes:

> M    interpreter/executioncontext.py
>      fixed exception printing problem (probably a bootstrap issue 
>      regarding string ops)

You have the wrong fix here, I'll check in the right one momentarily.
It's definitely from the "how the hell did that *ever* work?!"
category.

Cheers,
M.

-- 
  You owe The Oracle a TV with an 'intelligence' control - I've
  tried 'brightness' but that didn't work.
                                      -- Internet Oracularity #1192-01