From rxe at ukshells.co.uk  Sat Jul  1 01:24:16 2006
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Sat, 1 Jul 2006 00:24:16 +0100 (BST)
Subject: [pypy-dev] patch for __getattribute__()
Message-ID: <Pine.LNX.4.62.0607010016280.2028@echo.ukshells.co.uk>


Hi!

The following patch initiated by a chat with Runar Petursson about 
descriptors - increases pypy-c/pypy-llvm by 25% on richards.py.

I couldnt be bothered creating a branch - but if someone wants to sanity 
check it and fix the method repr and check it in, you are more than 
welcome! :-)

Cheers,
Richard

-------------- next part --------------
Index: objspace/descroperation.py
===================================================================
--- objspace/descroperation.py	(revision 29539)
+++ objspace/descroperation.py	(working copy)
@@ -1,7 +1,7 @@
 import operator
 from pypy.interpreter.error import OperationError
 from pypy.interpreter.baseobjspace import ObjSpace
-from pypy.interpreter.function import Function, Method
+from pypy.interpreter.function import Function, Method, descr_function_get
 from pypy.interpreter.argument import Arguments
 from pypy.interpreter.typedef import default_identity_hash
 from pypy.tool.sourcetools import compile2, func_with_new_name
@@ -19,6 +19,10 @@
         name = space.str_w(w_name)
         w_descr = space.lookup(w_obj, name)
         if w_descr is not None:
+            # XXX test performance
+            descr = space.interpclass_w(w_descr)
+            if type(descr) is Function:
+                return descr_function_get(space, w_descr, w_obj)
             if space.is_data_descr(w_descr):
                 return space.get(w_descr, w_obj)
         w_value = w_obj.getdictvalue(space, w_name)
Index: objspace/cpy/objspace.py
===================================================================
--- objspace/cpy/objspace.py	(revision 29539)
+++ objspace/cpy/objspace.py	(working copy)
@@ -62,8 +62,8 @@
             return PyString_FromStringAndSize(x, len(x))
         if isinstance(x, float): 
             return PyFloat_FromDouble(x)
-        if isinstance(x, r_uint):
-            return PyLong_FromUnsignedLong(x)
+        #if isinstance(x, r_uint):
+        #    return PyLong_FromUnsignedLong(x)
         # if we arrive here during RTyping, then the problem is *not* the %r
         # in the format string, but it's that someone is calling space.wrap()
         # on a strange object.
Index: interpreter/function.py
===================================================================
--- interpreter/function.py	(revision 29539)
+++ interpreter/function.py	(working copy)
@@ -385,6 +385,16 @@
             w_class = space.type(self.w_instance)
         else:
             w_class = self.w_class
+        if w_class is None:
+            if self.w_instance is None:
+                s = "<unbound method %s>" % (name)
+                return space.wrap(s)
+            else:
+                objrepr = space.str_w(space.repr(self.w_instance))
+                info = 'bound method %s of %s' % (name, objrepr)
+                # info = "method %s of %s object" % (name, typename)
+                return self.w_instance.getrepr(self.space, info)
+
         typename = w_class.getname(self.space, '?')
         if self.w_instance is None:
             s = "<unbound method %s.%s>" % (typename, name)

From pedronis at strakt.com  Sat Jul  1 10:03:25 2006
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sat, 01 Jul 2006 10:03:25 +0200
Subject: [pypy-dev] patch for __getattribute__()
In-Reply-To: <Pine.LNX.4.62.0607010016280.2028@echo.ukshells.co.uk>
References: <Pine.LNX.4.62.0607010016280.2028@echo.ukshells.co.uk>
Message-ID: <44A62C4D.8090300@strakt.com>

Richard Emslie wrote:
> 
> Hi!
> 
> The following patch initiated by a chat with Runar Petursson about 
> descriptors - increases pypy-c/pypy-llvm by 25% on richards.py.

well, it breaks this:

Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> class C(object):
...    def m(self):
...       return "class"
...
 >>> c=C()
 >>> c.m = lambda: "instance"
 >>>
 >>> c.m()
'instance'
 >>>

vs. patched PyPy

PyPy 0.9.0 in StdObjSpace on top of Python 2.4.1 (startuptime: 4.97 secs)
 >>>> class C(object):
....    def m(self):
....        return "class"
....
 >>>> c=C()
 >>>> c.m = lambda: "instance"
 >>>> c.m()
'class'

functions are not data descriptors, this feature makes increasing the 
performance of method lookup more involved that one would like but
Python semantics expect this kind of overriding to work.

I have added a test about this.


> 
> I couldnt be bothered creating a branch - but if someone wants to sanity 
> check it and fix the method repr and check it in, you are more than 
> welcome! :-)
> 
> Cheers,
> Richard
> 
> 
> ------------------------------------------------------------------------
> 
> Index: objspace/descroperation.py
> ===================================================================
> --- objspace/descroperation.py	(revision 29539)
> +++ objspace/descroperation.py	(working copy)
> @@ -1,7 +1,7 @@
>  import operator
>  from pypy.interpreter.error import OperationError
>  from pypy.interpreter.baseobjspace import ObjSpace
> -from pypy.interpreter.function import Function, Method
> +from pypy.interpreter.function import Function, Method, descr_function_get
>  from pypy.interpreter.argument import Arguments
>  from pypy.interpreter.typedef import default_identity_hash
>  from pypy.tool.sourcetools import compile2, func_with_new_name
> @@ -19,6 +19,10 @@
>          name = space.str_w(w_name)
>          w_descr = space.lookup(w_obj, name)
>          if w_descr is not None:
> +            # XXX test performance
> +            descr = space.interpclass_w(w_descr)
> +            if type(descr) is Function:
> +                return descr_function_get(space, w_descr, w_obj)
>              if space.is_data_descr(w_descr):
>                  return space.get(w_descr, w_obj)
>          w_value = w_obj.getdictvalue(space, w_name)
> Index: objspace/cpy/objspace.py
> ===================================================================
> --- objspace/cpy/objspace.py	(revision 29539)
> +++ objspace/cpy/objspace.py	(working copy)
> @@ -62,8 +62,8 @@
>              return PyString_FromStringAndSize(x, len(x))
>          if isinstance(x, float): 
>              return PyFloat_FromDouble(x)
> -        if isinstance(x, r_uint):
> -            return PyLong_FromUnsignedLong(x)
> +        #if isinstance(x, r_uint):
> +        #    return PyLong_FromUnsignedLong(x)
>          # if we arrive here during RTyping, then the problem is *not* the %r
>          # in the format string, but it's that someone is calling space.wrap()
>          # on a strange object.
> Index: interpreter/function.py
> ===================================================================
> --- interpreter/function.py	(revision 29539)
> +++ interpreter/function.py	(working copy)
> @@ -385,6 +385,16 @@
>              w_class = space.type(self.w_instance)
>          else:
>              w_class = self.w_class
> +        if w_class is None:
> +            if self.w_instance is None:
> +                s = "<unbound method %s>" % (name)
> +                return space.wrap(s)
> +            else:
> +                objrepr = space.str_w(space.repr(self.w_instance))
> +                info = 'bound method %s of %s' % (name, objrepr)
> +                # info = "method %s of %s object" % (name, typename)
> +                return self.w_instance.getrepr(self.space, info)
> +
>          typename = w_class.getname(self.space, '?')
>          if self.w_instance is None:
>              s = "<unbound method %s.%s>" % (typename, name)
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev



From arigo at tunes.org  Sat Jul  1 14:38:03 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 1 Jul 2006 14:38:03 +0200
Subject: [pypy-dev] [pypy-svn] r29534
	-	pypy/dist/pypy/rpython/ootypesystem/test
In-Reply-To: <44A5735A.6090908@gmail.com>
References: <20060630140821.989F310081@code0.codespeak.net>
	<44A56302.2090808@strakt.com> <44A5735A.6090908@gmail.com>
Message-ID: <20060701123803.GA18685@code0.codespeak.net>

Hi Antonio,

On Fri, Jun 30, 2006 at 08:54:18PM +0200, Antonio Cuni wrote:
> Probably it's safer to store the name in some dictionary.

It's even required, yes: no mutating of the low-level type objects
please :-)  The case of Instance is a bit special in this respect.


A bientot,

Armin


From arigo at tunes.org  Sat Jul  1 21:51:53 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 1 Jul 2006 21:51:53 +0200
Subject: [pypy-dev] [dls2006@hpi.uni-potsdam.de: DLS 2006 -- Author
	Notifications Delayed]
Message-ID: <20060701195153.GA22357@code0.codespeak.net>

Hi all,

Looks like we won't know for another week:


----- Forwarded message from dls2006 at hpi.uni-potsdam.de -----

Date: Sat, 01 Jul 2006 21:16:12 +0200
From: dls2006 at hpi.uni-potsdam.de
To: arigo at tunes.org
Subject: DLS 2006 -- Author Notifications Delayed
Reply-To: dls2006 at hpi.uni-potsdam.de

Dear Authors --

Due to unforeseen circumstances author notifications are delayed, expected to be ready mid of next week.

My apologies for the inconvenience,
Robert

----- End forwarded message -----


Armin


From seojiwon at gmail.com  Tue Jul  4 17:55:40 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Tue, 4 Jul 2006 08:55:40 -0700
Subject: [pypy-dev] question on using ctypes
Message-ID: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>

Hi,

I added a module that uses ctypes under pypy/module/ but I'm getting
ImportError while running translate.py for targetpypystandalone.py

The error message is like,

..... cut ...
[translation:ERROR]     w_value = loader(space)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/interpreter/mixedmodule.py", line 135,
in ifileloader
[translation:ERROR]     d[name] = __import__(name, None, None, [name])
[translation:ERROR]  ImportError': No module named tcc
[translation:ERROR]     .. v0 = getattr(self_0, ('w_dict'))
[translation:ERROR] Processing block:
[translation:ERROR]  block at -1 is a <class
'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in
(pypy.interpreter.mixedmodule:33)MixedModule.MixedModule.getdictvalue
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v1 = getattr(space_0, ('finditem'))
[translation:ERROR]        v0 = getattr(self_0, ('w_dict'))
[translation:ERROR]        v2 = simple_call(v1, v0, w_name_0)
[translation:ERROR]        v3 = getattr(self_0, ('lazy'))
[translation:ERROR]        v4 = is_true(v3)
[translation:ERROR]  --end--

and the part I added is pypy/module/tcc/__init__.py and pypy/module/tcc/tcc.py

### __init__.py

# Package initialisation
from pypy.interpreter.mixedmodule import MixedModule

class Module(MixedModule):
    appleveldefs = {}
    interpleveldefs = {
        'print0':   'tcc.tcc_print0'
    }

### tcc.py
from pypy.interpreter.baseobjspace import ObjSpace
from ctypes import *

libtcc = CDLL('/path/to/lib.so')
print0 = libtcc.TCC_Print
print0.argtypes = [c_char_p]
print0.restype = c_int

def tcc_print0(space, msg):
    print0(msg)
tcc_print0.unwrap_spec = [ObjSpace, str]

What have I done wrong?

Thaniks,

-Jiwon


From amauryfa at gmail.com  Tue Jul  4 18:26:09 2006
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 4 Jul 2006 18:26:09 +0200
Subject: [pypy-dev] question on using ctypes
In-Reply-To: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
References: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
Message-ID: <e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>

2006/7/4, Jiwon Seo <seojiwon at gmail.com>:
> Hi,
>
> I added a module that uses ctypes under pypy/module/ but I'm getting
> ImportError while running translate.py for targetpypystandalone.py
>
> The error message is like,
>
> ..... cut ...
> [translation:ERROR]     w_value = loader(space)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/interpreter/mixedmodule.py", line 135,
> in ifileloader
> [translation:ERROR]     d[name] = __import__(name, None, None, [name])
> [translation:ERROR]  ImportError': No module named tcc
> [translation:ERROR]     .. v0 = getattr(self_0, ('w_dict'))
> [translation:ERROR] Processing block:
> [translation:ERROR]  block at -1 is a <class
> 'pypy.objspace.flow.flowcontext.SpamBlock'>
> [translation:ERROR]  in
> (pypy.interpreter.mixedmodule:33)MixedModule.MixedModule.getdictvalue
> [translation:ERROR]  containing the following operations:
> [translation:ERROR]        v1 = getattr(space_0, ('finditem'))
> [translation:ERROR]        v0 = getattr(self_0, ('w_dict'))
> [translation:ERROR]        v2 = simple_call(v1, v0, w_name_0)
> [translation:ERROR]        v3 = getattr(self_0, ('lazy'))
> [translation:ERROR]        v4 = is_true(v3)
> [translation:ERROR]  --end--
>
> and the part I added is pypy/module/tcc/__init__.py and pypy/module/tcc/tcc.py

Hello,

Don't know if it helps, but I have already seen this problem:
sometimes an ImportError can mask another ImportError.
The code in mixedmodule.py looks like this:
    try:
        d[name] = __import__(pkgroot+'.'+name, None, None, [name])
    except ImportError:
        d[name] = __import__(name, None, None, [name])

The first call is supposed to be the good one, but if it fails with an
ImportError, you get the error from the second call.
Is the ctypes module correctly installed?

-- 
Amaury Forgeot d'Arc


From seojiwon at gmail.com  Wed Jul  5 01:43:58 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Tue, 4 Jul 2006 16:43:58 -0700
Subject: [pypy-dev] question on using ctypes
In-Reply-To: <e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>
References: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
	<e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>
Message-ID: <b008462b0607041643t6cc3d1cel125092de786a0f6f@mail.gmail.com>

> Hello,
>
> Don't know if it helps, but I have already seen this problem:
> sometimes an ImportError can mask another ImportError.
> The code in mixedmodule.py looks like this:
>    try:
>        d[name] = __import__(pkgroot+'.'+name, None, None, [name])
>    except ImportError:
>        d[name] = __import__(name, None, None, [name])
>
> The first call is supposed to be the good one, but if it fails with an
> ImportError, you get the error from the second call.
> Is the ctypes module correctly installed?
>

Yeah, I think so. (I installed in my home directory, and I think I
properly set up PYTHONSTARTUP and did sys.path.append... in the
PYTHONSTARTUP file.
And, 'p name' in the pdb tells me it's tcc modue. So, I think ctypes
might not be related to this. (but I could be wrong)

Any other suggestions?

-Jiwon


From seojiwon at gmail.com  Wed Jul  5 09:25:12 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Wed, 5 Jul 2006 00:25:12 -0700
Subject: [pypy-dev] question on using ctypes
In-Reply-To: <b008462b0607041643t6cc3d1cel125092de786a0f6f@mail.gmail.com>
References: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
	<e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>
	<b008462b0607041643t6cc3d1cel125092de786a0f6f@mail.gmail.com>
Message-ID: <b008462b0607050025w723fb306nf8c8c83dd618c77c@mail.gmail.com>

On further inspection, it is actually ctypes problem. It seems like
setting sys.path in PYTHONSTARTUP file isn't enough. I guess I should
just install it in system-wide python site-package directory.

Thanks.

On 7/4/06, Jiwon Seo <seojiwon at gmail.com> wrote:
> > Hello,
> >
> > Don't know if it helps, but I have already seen this problem:
> > sometimes an ImportError can mask another ImportError.
> > The code in mixedmodule.py looks like this:
> >    try:
> >        d[name] = __import__(pkgroot+'.'+name, None, None, [name])
> >    except ImportError:
> >        d[name] = __import__(name, None, None, [name])
> >
> > The first call is supposed to be the good one, but if it fails with an
> > ImportError, you get the error from the second call.
> > Is the ctypes module correctly installed?
> >
>
> Yeah, I think so. (I installed in my home directory, and I think I
> properly set up PYTHONSTARTUP and did sys.path.append... in the
> PYTHONSTARTUP file.
> And, 'p name' in the pdb tells me it's tcc modue. So, I think ctypes
> might not be related to this. (but I could be wrong)
>
> Any other suggestions?
>
> -Jiwon
>


From seojiwon at gmail.com  Wed Jul  5 10:16:27 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Wed, 5 Jul 2006 01:16:27 -0700
Subject: [pypy-dev] question on using ctypes
In-Reply-To: <b008462b0607050025w723fb306nf8c8c83dd618c77c@mail.gmail.com>
References: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
	<e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>
	<b008462b0607041643t6cc3d1cel125092de786a0f6f@mail.gmail.com>
	<b008462b0607050025w723fb306nf8c8c83dd618c77c@mail.gmail.com>
Message-ID: <b008462b0607050116l2fe0ac51m10c611b2d61bbccb@mail.gmail.com>

After properly installing ctypes, translate.py now gives this error.

##################################################################

[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "./translate.py", line 332, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 506, in
proceed
[translation:ERROR]     return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/translator/tool/taskengine.py", line
108, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 195, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 208, in
task_annotate
[translation:ERROR]     s = annotator.build_types(self.entry_point,
self.inputtypes)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 100, in
build_types
[translation:ERROR]     return self.build_graph_types(flowgraph,
inputcells, complete_now=complete_now)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 143, in
build_graph_types
[translation:ERROR]     self.complete()
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 188, in
complete
[translation:ERROR]     self.processblock(graph, block)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 434, in
processblock
[translation:ERROR]     self.flowin(graph, block)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 490, in
flowin
[translation:ERROR]     self.consider_op(block.operations[i])
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 661, in
consider_op
[translation:ERROR]     argcells = [self.binding(a) for a in op.args]
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 238, in
binding
[translation:ERROR]     return self.bookkeeper.immutableconstant(arg)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 297, in
immutableconstant
[translation:ERROR]     return self.immutablevalue(const.value)
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 393, in
immutablevalue
[translation:ERROR]     result = SomePBC([self.getdesc(x)])
[translation:ERROR]    File
"/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 524, in
getdesc
[translation:ERROR]     return self.descs[pyobj]
[translation:ERROR]  TypeError: unhashable type
[translation:ERROR] Processing block:
[translation:ERROR]  block at -1 is a <class
'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (pypy.module.tcc.tcc:9)tcc_print0
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v107 = simple_call((_FuncPtr TCC_Print), msg_0)
[translation:ERROR]  --end--

############# end of error message ##################################

pypy/module/tcc/__init__.py is like following
# Package initialisation
from pypy.interpreter.mixedmodule import MixedModule

class Module(MixedModule):
    appleveldefs = {}
    interpleveldefs = {
        'print0':   'tcc.tcc_print0'
    }

and pypy/module/tcc/tcc.py is like following

from pypy.interpreter.baseobjspace import ObjSpace
from ctypes import *

libtcc = CDLL('/u/jiwon/proj/less-bus/libLESS.so')
myprint0 = libtcc.TCC_Print
myprint0.argtypes = [c_char_p]
myprint0.restype = c_int

def tcc_print0(space, msg):
    return myprint0(msg)
tcc_print0.unwrap_spec = [ObjSpace, str]


What am I missing here?

-Jiwon


From amauryfa at gmail.com  Wed Jul  5 11:47:00 2006
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 5 Jul 2006 11:47:00 +0200
Subject: [pypy-dev] question on using ctypes
In-Reply-To: <e27efe130607050245y5d268500jd8a47db4593bebdd@mail.gmail.com>
References: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
	<e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>
	<b008462b0607041643t6cc3d1cel125092de786a0f6f@mail.gmail.com>
	<b008462b0607050025w723fb306nf8c8c83dd618c77c@mail.gmail.com>
	<b008462b0607050116l2fe0ac51m10c611b2d61bbccb@mail.gmail.com>
	<e27efe130607050245y5d268500jd8a47db4593bebdd@mail.gmail.com>
Message-ID: <e27efe130607050247m9dc6015g545b405cbfde6746@mail.gmail.com>

Hello,

2006/7/5, Jiwon Seo <seojiwon at gmail.com>:
> After properly installing ctypes, translate.py now gives this error.
>
> ##################################################################
>
> [translation:ERROR] Error:
> [translation:ERROR]  Traceback (most recent call last):
> [translation:ERROR]    File "./translate.py", line 332, in main
> [translation:ERROR]     drv.proceed(goals)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 506, in
> proceed
> [translation:ERROR]     return self._execute(goals, task_skip =
> self._maybe_skip())
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/translator/tool/taskengine.py", line
> 108, in _execute
> [translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 195, in _do
> [translation:ERROR]     res = func()
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 208, in
> task_annotate
> [translation:ERROR]     s = annotator.build_types(self.entry_point,
> self.inputtypes)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 100, in
> build_types
> [translation:ERROR]     return self.build_graph_types(flowgraph,
> inputcells, complete_now=complete_now)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 143, in
> build_graph_types
> [translation:ERROR]     self.complete()
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 188, in
> complete
> [translation:ERROR]     self.processblock(graph, block)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 434, in
> processblock
> [translation:ERROR]     self.flowin(graph, block)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 490, in
> flowin
> [translation:ERROR]     self.consider_op(block.operations[i])
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 661, in
> consider_op
> [translation:ERROR]     argcells = [self.binding(a) for a in op.args]
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 238, in
> binding
> [translation:ERROR]     return self.bookkeeper.immutableconstant(arg)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 297, in
> immutableconstant
> [translation:ERROR]     return self.immutablevalue(const.value)
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 393, in
> immutablevalue
> [translation:ERROR]     result = SomePBC([self.getdesc(x)])
> [translation:ERROR]    File
> "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 524, in
> getdesc
> [translation:ERROR]     return self.descs[pyobj]
> [translation:ERROR]  TypeError: unhashable type
> [translation:ERROR] Processing block:
> [translation:ERROR]  block at -1 is a <class
> 'pypy.objspace.flow.flowcontext.SpamBlock'>
> [translation:ERROR]  in (pypy.module.tcc.tcc:9)tcc_print0
> [translation:ERROR]  containing the following operations:
> [translation:ERROR]        v107 = simple_call((_FuncPtr TCC_Print), msg_0)
> [translation:ERROR]  --end--
>
> ############# end of error message ##################################
>
> pypy/module/tcc/__init__.py is like following
> # Package initialisation
> from pypy.interpreter.mixedmodule import MixedModule
>
> class Module(MixedModule):
>     appleveldefs = {}
>     interpleveldefs = {
>         'print0':   'tcc.tcc_print0'
>     }
>
> and pypy/module/tcc/tcc.py is like following
>
> from pypy.interpreter.baseobjspace import ObjSpace
> from ctypes import *
>
> libtcc = CDLL('/u/jiwon/proj/less-bus/libLESS.so')
> myprint0 = libtcc.TCC_Print
> myprint0.argtypes = [c_char_p]
> myprint0.restype = c_int
>
> def tcc_print0(space, msg):
>     return myprint0(msg)
> tcc_print0.unwrap_spec = [ObjSpace, str]
>
>
> What am I missing here?

I think the currently supported way to compile modules using rctypes
is the pypy/bin/compilemodule.py script, and indeed your code
translates well with it.

I suggest (but I did not test) adding this line somewhere in
targetstandalone.py:
    import pypy.rpython.rctypes.implementation
This should register the annotation for calling ctypes functions.


-- 
Amaury Forgeot d'Arc


From stephen at thorne.id.au  Thu Jul  6 08:43:12 2006
From: stephen at thorne.id.au (Stephen Thorne)
Date: Thu, 6 Jul 2006 16:43:12 +1000
Subject: [pypy-dev] Writing os.pipe() using ctypes
In-Reply-To: 0
Message-ID: <20060706064312.29014.218909000.divmod.quotient.24486@ohm>

Hi,

I've been having fun this week trying to get twisted's unit tests running on top of pypy. This has the dual goals of getting twisted to run on a funky platform like pypy, and being able to give pypy a bit of a run around the yard. I have to report this is going "better than expected". I've already used rhymes's fcntl implementation and had to extend the posix module, I intend to do a diff once I get everything sorted.

I have a problem with rpython translation of my implementation of posix.pipe(). I have other ctypes implementations of functions (such as posix.access()) which work okay, but I'm having trouble with this particular case.

man 2 pipe says that I use it like this:

       int pipe(int filedes[2]);

My implementation looks like:

fd_arr = c_int * 2
def pipe(space):
    fds = fd_arr()
    fds[0] = 0
    fds[1] = 0
    if libc.pipe(fds):
        raise makeOSError(space)
    return space.wrap((fds[0], fds[1]))
pipe.unwrap_spec = [ObjSpace]

And the error I get from time {{{python translate.py --text --batch  standalone --thread}}} is:
[translation:ERROR]  Exception': unexpected prebuilt constant: <slot wrapper '__init__' of '_ctypes.Array' objects>
[translation:ERROR]     .. v0 = simple_call((ArrayType c_long_Array_2))
[translation:ERROR] Processing block:
[translation:ERROR]  block at 3 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (pypy.module.posix.ctypes_posix:43)pipe
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v0 = simple_call((ArrayType c_long_Array_2))
[translation:ERROR]        v1 = setitem(v0, (0), (0))
[translation:ERROR]        v2 = setitem(v0, (1), (0))
[translation:ERROR]        v3 = getattr((<CDLL '/usr/lib/libc...20a89f0>), ('pipe'))
[translation:ERROR]        v4 = simple_call(v3, v0)
[translation:ERROR]        v5 = is_true(v4)
[translation:ERROR]  --end--

I've tried a bunch of ways of initialising the 'fds' object as to not make it a 'prebuilt constant', fds = fd_arr(0,0), creating fd_arr dynamically, fd_arr(zero, zero), etc. And I simply cannot seem to get rid of this exception.

So, any ideas as to how I go about making this work?

Stephen.


From amauryfa at gmail.com  Thu Jul  6 11:43:32 2006
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 6 Jul 2006 11:43:32 +0200
Subject: [pypy-dev] Writing os.pipe() using ctypes
In-Reply-To: <20060706064312.29014.218909000.divmod.quotient.24486@ohm>
References: <20060706064312.29014.218909000.divmod.quotient.24486@ohm>
Message-ID: <e27efe130607060243m5eed761fma545b5c976ac66e9@mail.gmail.com>

Hello,

2006/7/6, Stephen Thorne <stephen at thorne.id.au>:
> Hi,
>
> I've been having fun this week trying to get twisted's unit tests running on top of pypy. This has the dual goals of getting twisted to run on a funky platform like pypy, and being able to give pypy a bit of a run around the yard. I have to report this is going "better than expected". I've already used rhymes's fcntl implementation and had to extend the posix module, I intend to do a diff once I get everything sorted.
>
> I have a problem with rpython translation of my implementation of posix.pipe(). I have other ctypes implementations of functions (such as posix.access()) which work okay, but I'm having trouble with this particular case.
>
> man 2 pipe says that I use it like this:
>
>        int pipe(int filedes[2]);
>
> My implementation looks like:
>
> fd_arr = c_int * 2
> def pipe(space):
>     fds = fd_arr()
>     fds[0] = 0
>     fds[1] = 0
>     if libc.pipe(fds):
>         raise makeOSError(space)
>     return space.wrap((fds[0], fds[1]))
> pipe.unwrap_spec = [ObjSpace]
>
> And the error I get from time {{{python translate.py --text --batch  standalone --thread}}} is:
> [translation:ERROR]  Exception': unexpected prebuilt constant: <slot wrapper '__init__' of '_ctypes.Array' objects>
> [translation:ERROR]     .. v0 = simple_call((ArrayType c_long_Array_2))
> [translation:ERROR] Processing block:
> [translation:ERROR]  block at 3 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'>
> [translation:ERROR]  in (pypy.module.posix.ctypes_posix:43)pipe
> [translation:ERROR]  containing the following operations:
> [translation:ERROR]        v0 = simple_call((ArrayType c_long_Array_2))
> [translation:ERROR]        v1 = setitem(v0, (0), (0))
> [translation:ERROR]        v2 = setitem(v0, (1), (0))
> [translation:ERROR]        v3 = getattr((<CDLL '/usr/lib/libc...20a89f0>), ('pipe'))
> [translation:ERROR]        v4 = simple_call(v3, v0)
> [translation:ERROR]        v5 = is_true(v4)
> [translation:ERROR]  --end--
>
> I've tried a bunch of ways of initialising the 'fds' object as to not make it a 'prebuilt constant', fds = fd_arr(0,0), creating fd_arr dynamically, fd_arr(zero, zero), etc. And I simply cannot seem to get rid of this exception.
>
> So, any ideas as to how I go about making this work?
>
> Stephen.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

It seems that translate.py does not "register" annotation for rctypes objects.
In a previous mail I suggested to add
    import pypy.rpython.rctypes.implementation
in pypy/translator/goal/translate.py and translation seems to go further.
(I'm running
    translate.py standalone --usemodules=_demo
after I added your code in demo.py)

Unfortunately the complete translation takes too much memory for the
machine I'm currently working on, so I could not test completely.
And since its a Windows box, I had to remove the libc.pipe call completely...

Hope this helps.
It seems that many people are interested in the new rctypes way to
create modules...

-- 
Amaury Forgeot d'Arc


From arigo at tunes.org  Thu Jul  6 13:13:30 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 6 Jul 2006 13:13:30 +0200
Subject: [pypy-dev] question on using ctypes
In-Reply-To: <e27efe130607050247m9dc6015g545b405cbfde6746@mail.gmail.com>
References: <b008462b0607040855l798a8c2dodd66255066216f72@mail.gmail.com>
	<e27efe130607040926n370069b2uc02bf388c191dd61@mail.gmail.com>
	<b008462b0607041643t6cc3d1cel125092de786a0f6f@mail.gmail.com>
	<b008462b0607050025w723fb306nf8c8c83dd618c77c@mail.gmail.com>
	<b008462b0607050116l2fe0ac51m10c611b2d61bbccb@mail.gmail.com>
	<e27efe130607050245y5d268500jd8a47db4593bebdd@mail.gmail.com>
	<e27efe130607050247m9dc6015g545b405cbfde6746@mail.gmail.com>
Message-ID: <20060706111330.GA29352@code0.codespeak.net>

Hi,

On Wed, Jul 05, 2006 at 11:47:00AM +0200, Amaury Forgeot d'Arc wrote:
> I suggest (but I did not test) adding this line somewhere in
> targetstandalone.py:
>     import pypy.rpython.rctypes.implementation
> This should register the annotation for calling ctypes functions.

Indeed, this module needs to be imported from somewhere.  I'd recommend
to import it from tcc.py directly, even.  See the end of the following
section:

  http://codespeak.net/pypy/dist/pypy/doc/rctypes.html#restrictions


A bientot,

Armin


From cfbolz at gmx.de  Fri Jul  7 14:03:17 2006
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 7 Jul 2006 14:03:17 +0200
Subject: [pypy-dev] PyPy is faster than CPython (given an evil enough
	benchmark)
Message-ID: <348899050607070503o62fe5edaw134ea2953bb17b92@mail.gmail.com>

Hi all!

Together with the optimized string objects that Armin, Pieter,
Alexander and me have been working on, it is possible to construct a
(very corner-casish) benchmark, which exploits the fast cases of PyPy
in such a way, that it is quite a bit faster CPython. See the
following code:

import time, sys

def test_speed(num):
    t1 = time.time()
    result = ""
    intermediate = []
    for i in range(num):
        result += str(i)
        intermediate.append(result)
    t2 = time.time()
    return t2 - t1, intermediate

t1 = time.time()
for i in range(1, 12):
    speed, intermediate = test_speed(i * 1000)
t2 = time.time()
print t2 - t1


It runs 3.4 times faster on pypy-c with WITHSTRSLICE = WITHSTRJOIN =
True. In addition CPython uses 25 times more memory (which of course
accounts for some of the slowness).

Of course the lesson of this is that benchmarks are worthless :-).

Cheers,

Carl Friedrich


From seojiwon at gmail.com  Sat Jul  8 21:51:56 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Sat, 8 Jul 2006 12:51:56 -0700
Subject: [pypy-dev] Question on pypy exception...
Message-ID: <b008462b0607081251r3a07b15dldfd925ade56fe533@mail.gmail.com>

I'm getting an exception with pypy-c, and the exception occurs right
after pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr function in
pypy_g_PyInterpFrame_dispatch_translated.

1. Is "pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr" same as
LL_thread_fused_releaseacquirelock ? I can only see "#define
LL_thread_fused_releaseacquirelock
pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr", but not the other
way around.

2. Is that function there for forcing context switching?

3. What could happen in
pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr() that can set
(&pypy_g_pypy_translator_c_exceptiontransform_ExcData)->ed_inst_exc_type?
the error message that I'm seeing is like following

propagating implement_6.c:100878 pypy_g_PyInterpFrame_dispatch_translated error
propagating implement_5.c:30508 pypy_g_PyFrame_eval error
propagating implement_5.c:37782 pypy_g_EvalFrame_resume error
propagating implement_39.c:41465 pypy_g_ll_deallocator__Address_68 error
propagating implement.c:27685 pypy_g_funccall_star1 error
propagating implement_39.c:32105 pypy_g_ll_deallocator__Address_14 error
propagating implement_39.c:32158 pypy_g_ll_deallocator__Address_15 error
propagating implement_39.c:32294 pypy_g_ll_deallocator__Address_17 error
propagating implement.c:1336 pypy_g_entry_point error

Thanks!

-Jiwon


From mwh at python.net  Sun Jul  9 13:54:17 2006
From: mwh at python.net (Michael Hudson)
Date: Sun, 09 Jul 2006 12:54:17 +0100
Subject: [pypy-dev] PyPy Post-EuroPython 2006 Sprint Report
Message-ID: <2mejwv3seu.fsf@starship.python.net>

Greetings from an assortment of tired brains at CERN, home to a larger
assortment of hopefully less tired physicist brains.  We are reaching
the end of a succesful and well-attended (24 participants!) sprint.

3.5 days times 24 people is a bit more than two man months of work --
we are going to try to cover all we've achieved, but will undoubtedly
forget some stuff!

Summer of Code student Lawrence and his mentor Arre spent the majority
of the time integrating Lawrence's SoC work -- ctypes implementations
of various CPython extension modules -- into PyPy.  He has implemented
the time, mmap and fcntl modules so far, and the work at this sprint
concentrated on putting the time module into the form required for
PyPy -- making it a mixed module, conforming to the restrictions of
rctypes and fighting the extension compiler.

Brian Sutherland and Sad Rejeb also joined in this play, attempting to
convert some prewritten performance sensitive caching code used in
Zope(!) somewhere to a mixed module suitable for feeding to the
extension compiler.  Unfortunately the resulting code was no faster
than the pure Python version... probably because of the ext-compiler
using refcounting for GC.

Antonio appeared to spend his entire time writing commit messages but
somehow must have written some code in between (with the help of *his*
SoC mentor, Armin) because now all of PyPy's Standard Interpreter can
be rtyped with the ootypesystem.  "All" that remains are the dict
implementation and external function support in the CLI backend before
IronPython has a competitor in the shape of PyPy.Net :-)

Guido (not *that* Guido) being the JavaScript deity he is wrote a JS
tool to instrospect the DOM methods and produce useful information for
Maciej's SoC (we only had three SoC students here...) project.
Unfortunately browsers' JavaScript implementations are famously
eccentric and running his tool popped up the Print dialog box twice
and then crashed Epiphany.  It proved marginally less frustrating to
write a scraper for Mozilla's documentation and fix the broken HTML
he found there.

Aur?lien, Ludovic and part of Armin tried to figure out why the
example of coroutine cloning written for the "What can PyPy do for
you?" talk didn't work.  Assorted mind boggling bugs were found and
fixed, with the aggravating result that it now works fine when run on
the llinterpreter but crashes -- after a while -- when translated to
C.  More wizardry will be required here...

Just before the sprint Maciej attempted to improve the annotator's
error messages and he and Samuele spent some time fixing this new code
(it's very frustrating when your error printing routines crash on
you...).

A varying team of people including but not limited to Pieter, Carl
Friedrich, Maciej and Alexander worked on alternative, hopefully
faster implementations of Python data types, particularly strings and
dictionaries.  We now have efficient representations of slices of
strings and for the result of adding two strings, all completely
transparent to application-level.  For a sufficiently cooked
benchmark, we are now up to seven times faster than CPython (and use
30 times less memory).  On the dictionary side, we stole an
optimization for CPython: assume on creation that a dictionary will
only contain strings keys and switch to an alternative implementation
on the insertion of a non-string key.  This makes pystone and richards
both around 20% faster, some benchmarks up to 40% faster (but also
some a few percent slower).

Alexandre and David started to think about configuration and option
handling, but got distracted when they tried to run a translation on a
64-bit machine (or more specifically, a machine with 64-bit userland),
because it broke in obscure ways.  A few obscure fixes later and it
worked again apart from stackless, and another few fixes fixed that
too.

Michael and Fabrizio worked on the stackless code, "un-inlining" some
of the frame-state saving code with the goal of reducing the size of
the generated code.  This involved taking Fabrizio on a whirlwind tour
of one of the scarier areas of the PyPy codebase, but he survived and
the feature was implemented, although the size reduction was not as
great as had been hoped -- more investigation required (we also need
to build a stacklessgc build to check that).

Michael and Fabrizio then worked on AST- and bytecode-level
optimizations, which mostly involved building an infrastructure to
make modifications of the AST easier.  They implemented a simple
constant folder at the AST level and at the bytecode level an
optimization to remove the temporary tuple in code like "x,y=y,x".

Simon and Eric resurrected and experimented more with LLVM's JIT
interface.  Simon then worked with Ludovic and Arre on implementing
RPython support for numarrays and Eric worked on making the output of
genc compilable with C++.

Guido and Holger implemented the beginning and perhaps the middle but
probably not the end of doctest support in py.test.

After some discussion with Alexandre, David and Holger, Guido and Carl
Friedrich began to work on the rather ugly-looking task of taming
pypy's growing menagerie of options.  They started to implement a
general mechanism for defining a hierarchy of options, handling
dependencies and conflicts among them.  From this you can generate
optparse Options for command line interfaces and will one day be able
to generate a web form to choose among the zoo of pypy's options.
They used the config mechanisms to de-insane the selection of the
various object space variants.  This, for example, makes automated
testing of the above-mentioned variant string and dictionary
implementations possible.

Pieter worked on random modules^W^Wthe random module, porting it to
RPython and implementing some missing functions.

So, after a pre-sprint (for some of us), a conference and a
post-sprint, we are well prepared for the usual post-sprint week long
sleep.  Apart from Carl, he has an exam on Friday.  Oops!

Met vriendelijke groeten,
mwh&cfbolz

-- 
   This proposal, if accepted, will probably mean a heck of a lot of
   work for somebody.  But since I don't want it accepted, I don't
   care.                                   -- Laura Creighton, PEP 666



From arigo at tunes.org  Mon Jul 10 16:12:36 2006
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 10 Jul 2006 16:12:36 +0200
Subject: [pypy-dev] [emil@dmr.ath.cx: Re: [pypy-svn] r29899 -
	pypy/extradoc/talk/ep2006]
Message-ID: <20060710141236.GA27105@code0.codespeak.net>

Hi all,

----- Forwarded message from Emil Mikulic <emil at dmr.ath.cx> -----

Date: Mon, 10 Jul 2006 17:23:05 +1000
From: Emil Mikulic <emil at dmr.ath.cx>
To: arigo at codespeak.net
Subject: Re: [pypy-svn] r29899 - pypy/extradoc/talk/ep2006

On Sun, Jul 09, 2006 at 03:35:03PM +0200, arigo at codespeak.net wrote:
> http://snake.cs.uni-duesseldorf.de/tgtest/

I see:

	Bad Gateway

	The proxy server received an invalid response from an upstream server.
	Apache Server at snake.cs.uni-duesseldorf.de Port 80

=/

----- End forwarded message -----


From faassen at infrae.com  Tue Jul 11 14:14:46 2006
From: faassen at infrae.com (Martijn Faassen)
Date: Tue, 11 Jul 2006 14:14:46 +0200
Subject: [pypy-dev] PyPy is faster than CPython (given an evil enough
 benchmark)
In-Reply-To: <348899050607070503o62fe5edaw134ea2953bb17b92@mail.gmail.com>
References: <348899050607070503o62fe5edaw134ea2953bb17b92@mail.gmail.com>
Message-ID: <44B39636.1030804@infrae.com>

Carl Friedrich Bolz wrote:
[snip]
> Of course the lesson of this is that benchmarks are worthless :-).

Cool! I'll go tell everybody that PyPy is already faster than CPython 
now. :)

The best benchmarks will be large real-world codebases running on PyPy.

Regards,

Martijn



From l.oluyede at gmail.com  Tue Jul 11 18:00:12 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Tue, 11 Jul 2006 18:00:12 +0200
Subject: [pypy-dev] Bug hunting in rctime becomes wtf
Message-ID: <9eebf5740607110900l1ba16f2fu4eb220528c978494@mail.gmail.com>

I had a nasty bug like this one in the compilation process of the rctime module:

[translation:ERROR] Error:
[translation:ERROR]  TyperError: non-constant tuple index
[translation:ERROR] .. block at 34 with 1 exits
[translation:ERROR] .. v0 = getitem(tup_0, ___i_0)

You can reproduce it decommenting asctime function from interp_time.py
and in the __init__.py file of the rctime module.

After some digging I've found the source of the problem in the code
Armin and I have checked-in at the sprint to add the support for
"*args" arguments (see visit_args_w in pypy.objspace.cpy.function). I
tried patching it but I'm not very into cpy objspace and the compiler
mechanism under the hood.

The "wtf" thing came out when I tried to use the file with this
"stupid" patch applied:

rhymes at groove:~/scc/pypy/pypy/objspace/cpy $ svn diff
Index: function.py
===================================================================
--- function.py (revision 29953)
+++ function.py (working copy)
@@ -61,7 +61,7 @@
         tramp.inputargs.append('*' + basename)
         tramp.wrappings.append('%s = []' % (argname,))
         tramp.wrappings.append('for ___i in range(len(%s)):' % (basename,))
-        tramp.wrappings.append('    %s.append(___W_Object(%s[___i]))' % (
+        tramp.wrappings.append('    %s.append(___W_Object(%s[0]))' % (
             argname, basename))
         tramp.passedargs.append(argname)

The thing surprised me is the compilation after the patch goes fine
and (!!) the module works correctly. asctime() does it's dirty job.

So, why does it work?

-- 
Lawrence
http://www.oluyede.org/blog


From stephen at thorne.id.au  Wed Jul 12 06:56:35 2006
From: stephen at thorne.id.au (Stephen Thorne)
Date: Wed, 12 Jul 2006 14:56:35 +1000
Subject: [pypy-dev] PyPy is faster than CPython (given an evil enough
 benchmark)
In-Reply-To: <44B39636.1030804@infrae.com>
Message-ID: <20060712045635.29014.1792869848.divmod.quotient.32800@ohm>

On Tue, 11 Jul 2006 14:14:46 +0200, Martijn Faassen <faassen at infrae.com> wrote:
>Carl Friedrich Bolz wrote:
>[snip]
>> Of course the lesson of this is that benchmarks are worthless :-).
>
>Cool! I'll go tell everybody that PyPy is already faster than CPython
>now. :)
>
>The best benchmarks will be large real-world codebases running on PyPy.

This is precisely the reason I'm putting effort into getting the twisted unit tests to run on top of PyPy. Once we get the tests running on translated pypy, we can add a column to the http://twistedmatrix.com/buildbot/ builder, and get PyPy up as a platform we can run a harsh series of non-trivial unit tests on.

My work on this has stalled currently as there are a couple bugs in translation, but with the work Lawrence is doing on things like the posix module and fcntl, we're well on our way to being able to get this going.

FWIW, i've already managed to get twisted to run its own unit tests on its unit testing engine (trial) using py.py. Once we get fcntl translatable, I hope to report what happens when I run the lot (2764 tests) under pypy-c and 'see what happens'.

250 tests took approximately 11 hours to run.

Stephen.


From hpk at trillke.net  Thu Jul 13 08:02:29 2006
From: hpk at trillke.net (holger krekel)
Date: Thu, 13 Jul 2006 08:02:29 +0200
Subject: [pypy-dev] Security ideas
In-Reply-To: <20060524114728.GA6374@code0.codespeak.net>
References: <20060524114728.GA6374@code0.codespeak.net>
Message-ID: <20060713060229.GR32712@solar.trillke>

Hi Armin, 

On Wed, May 24, 2006 at 13:47 +0200, Armin Rigo wrote:
> On Monday I was at an inspiring seminar about (a specific form of)
> language-level security.  I've collected the PyPy-ification of these
> ideas there:
> 
> http://codespeak.net/svn/pypy/dist/pypy/doc/discussion/security-ideas.txt

IIUC, in this discussion you argue against an "Proxy-Object space" solution
because a code block may depend on a condition involving secret values. 

In your alternative "annotator" suggestion, you give the following example: 

    def enter_bid(n):
        if n > highest_bid.value:
            highest_bid.value = n 
    enter_bid = secure(enter_bid)

Here the annotator analysis is supposed to prevent a leak of information
from the secret value.  But if the if-branch additionally contains:

            num_bids += 1

don't you run into a branching/code-dependent-on-secret-condition 
problem again?  Would the annotator prevent the manipulation of 
the global 'num_bids'?  Would it need to be a public value? 

Moreover, i have practical concerns: your proposed
scheme requires RPython annotator analysis which implies to
have the PyPy tool chain available and accessible at programming 
time.  Not impossible but also not a use case that we went for 
so far.  Also it is not clear to which target "secure" would compile 
functions to, C or bytecode or ...?

best, 

    holger


From Ben.Young at risk.sungard.com  Fri Jul 14 13:12:59 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Fri, 14 Jul 2006 12:12:59 +0100
Subject: [pypy-dev] [pypy-svn] r30034 - pypy/dist/pypy/objspace/std
In-Reply-To: <20060714105155.DC18A10090@code0.codespeak.net>
Message-ID: <OFED1A6F46.5AACC121-ON802571AB.003D1B50-802571AB.003D9CD1@risk.sungard.com>

Hi Xoraxax (sorry, I don't know your real name!),

This looks really good! It shows there are still some major (easy) 
performance wins in PyPy.

One question, why would you need any calls to str2object in any of the 
non-mutating methods? Surely the class knows that no non-strings have been 
added, and if __getitem__ is called with a non-string, you could just 
throw KeyError. I guess this is becuase you have to replicate the fact 
that hash is always called, but couldn't you do:

            if space.is_w(w_lookup_type, space.w_str):
                return w_dict.content_str[space.str_w(w_lookup)]
            else:
                w_lookup.__hash__()
                raise KeyError

?

Keep up the good work (to all PyPy's). Things are really starting to look 
good!

Cheers,
Ben

pypy-svn-bounces at codespeak.net wrote on 14/07/2006 11:51:55:

> Author: xoraxax
> Date: Fri Jul 14 12:51:55 2006
> New Revision: 30034
> 
> Modified:
>    pypy/dist/pypy/objspace/std/dictstrobject.py
> Log:
> Speeded up dict creation, added shortcut for look ups of vars with 
> sane hashes, fixed get mm, fixed mutation handling in the iterators.
> (manual merge of changes done in the other branch)
> 
> Modified: pypy/dist/pypy/objspace/std/dictstrobject.py
> 
==============================================================================
> --- pypy/dist/pypy/objspace/std/dictstrobject.py   (original)
> +++ pypy/dist/pypy/objspace/std/dictstrobject.py   Fri Jul 14 12:51:55 
2006
> @@ -1,4 +1,3 @@
> -
>  from pypy.objspace.std.objspace import *
>  from pypy.interpreter import gateway
> 
> @@ -7,52 +6,52 @@
>  class W_DictStrObject(W_Object):
>      from pypy.objspace.std.dicttype import dict_typedef as typedef
> 
> -    def __init__(w_self, space, w_otherdict=None):
> +    def __init__(w_self, space):
>          w_self.space = space
> -        if w_otherdict is None:
> -            w_self.content = None
> -            w_self.content_str = {}
> -        else:
> -            if w_otherdict.content is None:
> -                w_self.content = None
> -                w_self.content_str = w_otherdict.content_str.copy()
> -            else:
> -                w_self.content = w_otherdict.content.copy()
> -                w_self.content_str = None
> +        w_self.content = None
> +        w_self.content_str = {}
> 
>      def initialize_content(w_self, list_pairs_w): # YYY
>          for w_k, w_v in list_pairs_w:
>              w_self.setitem(w_k, w_v)
> 
>      def getitem(w_dict, w_lookup):
> +        """ Helper function, raises rpython exceptions. """
>          space = w_dict.space
>          if w_dict.content is None:
> -            if space.is_w(space.type(w_lookup), space.w_str):
> +            w_lookup_type = space.type(w_lookup)
> +            if space.is_w(w_lookup_type, space.w_str):
>                  return w_dict.content_str[space.str_w(w_lookup)]
>              else:
> +                if w_dict.isSaneHash(w_lookup_type):
> +                    raise KeyError
>                  w_dict.str2object()
>          return w_dict.content[w_lookup]
> 
> -    def get(w_dict, w_lookup):
> +    def get(w_dict, w_lookup, w_default):
> +        """ Helper function, raises rpython exceptions. """
>          space = w_dict.space
>          if w_dict.content is None:
> -            if space.is_w(space.type(w_lookup), space.w_str):
> -                return w_dict.content_str.get(space.str_w(w_lookup), 
None)
> +            w_lookup_type = space.type(w_lookup)
> +            if space.is_w(w_lookup_type, space.w_str):
> +                return w_dict.content_str.get(space.
> str_w(w_lookup), w_default)
>              else:
> +                if w_dict.isSaneHash(w_lookup_type):
> +                    return w_default
>                  w_dict.str2object()
> -        return w_dict.content.get(w_lookup, None)
> +        return w_dict.content.get(w_lookup, w_default)
> 
> 
>      def setitem(w_self, w_k, w_v):
>          space = w_self.space
> -        if w_self.content is None:
> +        if w_self.content is not None:
> +            w_self.content[w_k] = w_v
> +        else:
>              if space.is_w(space.type(w_k), space.w_str):
>                  w_self.content_str[space.str_w(w_k)] = w_v
>              else:
>                  w_self.str2object()
>                  w_self.content[w_k] = w_v
> -        else:
> -            w_self.content[w_k] = w_v
> 
>      set_str_keyed_item = setitem
> 
> @@ -65,7 +64,17 @@
>          for k, w_v in w_self.content_str.items():
>              w_self.content[w_self.space.wrap(k)] = w_v
>          w_self.content_str = None
> - 
> +
> +    def isSaneHash(w_self, w_lookup_type):
> +        """ Handles the case of a non string key lookup.
> +        Types that have a sane hash/eq function should allow us to 
> return True
> +        directly to signal that the key is not in the dict in any case.
> +        XXX The types should provide such a flag. """
> + 
> +        space = w_self.space
> +        # XXX there are much more types
> +        return space.is_w(w_lookup_type, space.w_NoneType) or 
> space.is_w(w_lookup_type, space.w_int)
> +
>      def __repr__(w_self):
>          """ representation for debugging purposes """
>          return "%s(%s,%s)" % (w_self.__class__.__name__, w_self.
> content, w_self.content_str)
> @@ -144,7 +153,11 @@
> 
>  def contains__DictStr_ANY(space, w_dict, w_lookup):
>      if w_dict.content is None:
> -        if not space.is_w(space.type(w_lookup), space.w_str):
> +        w_lookup_type = space.type(w_lookup)
> +        if not space.is_w(w_lookup_type, space.w_str):
> +            if w_dict.isSaneHash(w_lookup_type):
> +                return space.w_False
> +            #foo("degenerated in contains: " + space.
> type(w_lookup).getname(space, '?'))
>              w_dict.str2object()
>          else:
>              return space.newbool(space.str_w(w_lookup) in 
w_dict.content_str)
> @@ -247,7 +260,14 @@
>      return w_res
> 
>  def dict_copy__DictStr(space, w_self):
> -    return W_DictStrObject(space, w_self)
> +    w_new_dict = W_DictStrObject(space)
> +    if w_self.content is None:
> +        w_new_dict.content = None
> +        w_new_dict.content_str = w_self.content_str.copy()
> +    else:
> +        w_new_dict.content = w_self.content.copy()
> +        w_new_dict.content_str = None
> +    return w_new_dict
> 
>  def dict_items__DictStr(space, w_self):
>      if w_self.content is not None:
> @@ -291,9 +311,7 @@
>          w_self.content.clear()
> 
>  def dict_get__DictStr_ANY_ANY(space, w_dict, w_lookup, w_default):
> -    if w_dict.content is None:
> -        return w_dict.content_str.get(space.str_w(w_lookup), w_default)
> -    return w_dict.content.get(w_lookup, w_default)
> +    return w_dict.get(w_lookup, w_default)
> 
>  app = gateway.applevel('''
>      def dictrepr(currently_in_repr, d):
> @@ -348,6 +366,12 @@
>      def return_entry(w_self, w_key, w_value):
>          raise NotImplementedError
> 
> +    def handleMutation(w_self):
> +        space = w_self.space
> +        w_self.len = -1   # Make this error state sticky
> +        raise OperationError(space.w_RuntimeError,
> +                 space.wrap("dictionary changed size during 
iteration"))
> +
>  registerimplementation(W_DictStrIterObject)
> 
>  class W_DictStrIter_Keys(W_DictStrIterObject):
> @@ -435,16 +459,25 @@
>      return w_dictiter
> 
>  def next__DictStrIterObject(space, w_dictiter):
> -    # XXX hum, this is broken currently
> -    if w_dictiter.content_str is not None and w_dictiter.
> w_dictobject.content_str is None:
> -        raise OperationError(space.w_RuntimeError,
> -                space.wrap("you stupid dictionary user - you have 
> just done an operation internally mutating the dictionary"))
> -
> -    if w_dictiter.content is not None:
> +    # iterate over the string dict even if the dictobject's data was 
forced
> +    # to degenerate. just bail out if the obj's dictionary size 
differs.
> +    if (w_dictiter.content_str is not None and w_dictiter.
> w_dictobject.content_str is None
> +        and len(w_dictiter.w_dictobject.content) != w_dictiter.len):
> +        w_dictiter.handleMutation()
> +
> +    if w_dictiter.content_str is not None:
> +        if w_dictiter.len != len(w_dictiter.content_str):
> +            w_dictiter.handleMutation()
> +        # look for the next entry
> +        w_result = w_dictiter.next_entry()
> +        if w_result is not None:
> +            w_dictiter.pos += 1
> +            return w_result
> +        # no more entries
> +        w_dictiter.content_str = None
> +    elif w_dictiter.content is not None:
>          if w_dictiter.len != len(w_dictiter.content):
> -            w_dictiter.len = -1   # Make this error state sticky
> -            raise OperationError(space.w_RuntimeError,
> -                     space.wrap("dictionary changed size during 
iteration"))
> +            w_dictiter.handleMutation()
>          # look for the next entry
>          w_result = w_dictiter.next_entry()
>          if w_result is not None:
> @@ -452,19 +485,6 @@
>              return w_result
>          # no more entries
>          w_dictiter.content = None
> -    else:
> -        if w_dictiter.content_str is not None:
> -            if w_dictiter.len != len(w_dictiter.content_str):
> -                w_dictiter.len = -1   # Make this error state sticky
> -                raise OperationError(space.w_RuntimeError,
> -                         space.wrap("dictionary changed size during
> iteration"))
> -            # look for the next entry
> -            w_result = w_dictiter.next_entry()
> -            if w_result is not None:
> -                w_dictiter.pos += 1
> -                return w_result
> -            # no more entries
> -            w_dictiter.content_str = None
>      raise OperationError(space.w_StopIteration, space.w_None)
> 
>  def len__DictStrIterObject(space, w_dictiter):
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
> 



From hpk at trillke.net  Fri Jul 14 13:40:55 2006
From: hpk at trillke.net (holger krekel)
Date: Fri, 14 Jul 2006 13:40:55 +0200
Subject: [pypy-dev] fixing ireland sprint (21th-27th august)
Message-ID: <20060714114055.GH32712@solar.trillke>

Hi folks, 

i took over trying to organise the Ireland sprint (scheduled
21st - 27th August currently) but i had problems getting feedback 
from the organisers yet.  Therefore, i am not sure about some 
details but i started a draft announcement anyway: 

    extradoc/sprintinfo/ireland-2006/announcement.txt

and also listed people from whom i have heard that they want to come

    extradoc/sprintinfo/ireland-2006/announcement.txt

Please check and add yourself already. 

However, how do we go on: do we just assume that the sprint
works out (it's definitely very likely) and book travels 
arriving 20th evening and leaving 27th or 28th? 

Or do we wait until - say - end next week in order
to be sure?  Anyway, if you book flights already, 
then make sure you have some insurance that you 
can step back and get most of your money back
(that's probably the strategy i am going to follow
myself). 

any comments? 

    holger


From seojiwon at gmail.com  Sat Jul 15 07:17:28 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Fri, 14 Jul 2006 22:17:28 -0700
Subject: [pypy-dev] cpy object space
Message-ID: <b008462b0607142217y3d8b2972sf64534c8d4288058@mail.gmail.com>

pypy/bin/py.py does not work with cpy object space

[llcoolj] bin > ./py.py -o cpy
[cbuild:execute] cc -O2 -pthread -I/usr/include/python2.4 -c
ctypesplatcheck_0.c -o ctypesplatcheck_0.o
[cbuild:execute] cc -pthread /tmp/usession-23/ctypesplatcheck_0.o -lm
-lpthread -o /tmp/usession-23/ctypesplatcheck_0
Traceback (most recent call last):
  File "./py.py", line 204, in ?
    sys.exit(main_(sys.argv))
  File "./py.py", line 82, in main_
    space.setitem(space.sys.w_dict,space.wrap('executable'),space.wrap(argv[0]))
AttributeError: 'CPyObjSpace' object has no attribute 'sys'

Having almost only introductory knowledge of pypy, I thought naively
following might solve the problem

class CPyObjSpace(baseobjspace.ObjSpace):
    def initialize(self):
        self.config.objspace.geninterp = False
        self.wrap_cache = {}
        self.sys = self.getbuiltinmodule("sys") ##### <-- adding just this line


but now it errors that it can't find w_dict attribute in space.sys
(py.py:main_() #82)  - still space.sys is W_Object. Maybe I need to
transform it as proper pypy interpreter module?


From seojiwon at gmail.com  Sat Jul 15 22:26:23 2006
From: seojiwon at gmail.com (Jiwon Seo)
Date: Sat, 15 Jul 2006 13:26:23 -0700
Subject: [pypy-dev] cpy object space
In-Reply-To: <b008462b0607142217y3d8b2972sf64534c8d4288058@mail.gmail.com>
References: <b008462b0607142217y3d8b2972sf64534c8d4288058@mail.gmail.com>
Message-ID: <b008462b0607151326m1db3196fl81c9408e0a18033f@mail.gmail.com>

Never mind. I just heard that cpy obj space is for mixed module translation.
I thought I'd try thunk obj space with cpy obj space, but I rather not. ;)

-jiwon

On 7/14/06, Jiwon Seo <seojiwon at gmail.com> wrote:
> pypy/bin/py.py does not work with cpy object space
>
> [llcoolj] bin > ./py.py -o cpy
> [cbuild:execute] cc -O2 -pthread -I/usr/include/python2.4 -c
> ctypesplatcheck_0.c -o ctypesplatcheck_0.o
> [cbuild:execute] cc -pthread /tmp/usession-23/ctypesplatcheck_0.o -lm
> -lpthread -o /tmp/usession-23/ctypesplatcheck_0
> Traceback (most recent call last):
>  File "./py.py", line 204, in ?
>    sys.exit(main_(sys.argv))
>  File "./py.py", line 82, in main_
>    space.setitem(space.sys.w_dict,space.wrap('executable'),space.wrap(argv[0]))
> AttributeError: 'CPyObjSpace' object has no attribute 'sys'
>
> Having almost only introductory knowledge of pypy, I thought naively
> following might solve the problem
>
> class CPyObjSpace(baseobjspace.ObjSpace):
>    def initialize(self):
>        self.config.objspace.geninterp = False
>        self.wrap_cache = {}
>        self.sys = self.getbuiltinmodule("sys") ##### <-- adding just this line
>
>
> but now it errors that it can't find w_dict attribute in space.sys
> (py.py:main_() #82)  - still space.sys is W_Object. Maybe I need to
> transform it as proper pypy interpreter module?
>


From 2006a at usenet.alexanderweb.de  Fri Jul 14 18:09:13 2006
From: 2006a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Fri, 14 Jul 2006 18:09:13 +0200
Subject: [pypy-dev] [pypy-svn] r30034 - pypy/dist/pypy/objspace/std
References: <20060714105155.DC18A10090@code0.codespeak.net>
	<OFED1A6F46.5AACC121-ON802571AB.003D1B50-802571AB.003D9CD1@risk.sungard.com>
Message-ID: <1p6bpe8fz3vkg$.dlg@usenet.alexanderweb.de>

Hi Ben Young,

On Fri, 14 Jul 2006 12:12:59 +0100, Ben.Young at risk.sungard.com wrote:

> This looks really good! It shows there are still some major (easy) 
> performance wins in PyPy.

Yes, I measured 35% speedup on richards/pystone on "the other branch" (that
solely knows this kind of dictionary unlike the current trunk).

> One question, why would you need any calls to str2object in any of the 
> non-mutating methods?

Because every class can define it's own hash/eq functions that might be
considered insane in case of having them return hashes that match the
hashes of a string. That would mean that {"foo": 1}[myClass()] could return
1. Therefore we have to switch the mode while having a shortcut for types
which are known to have sane hashes only.

>                 w_lookup.__hash__()

I didn't even consider that people would write __hash__ methods with
side-effects :)

Kind regards,
Alexander



From 2006a at usenet.alexanderweb.de  Sun Jul 16 23:40:01 2006
From: 2006a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Sun, 16 Jul 2006 23:40:01 +0200
Subject: [pypy-dev] [pypy-svn] r30034 - pypy/dist/pypy/objspace/std
References: <20060714105155.DC18A10090@code0.codespeak.net>
	<OFED1A6F46.5AACC121-ON802571AB.003D1B50-802571AB.003D9CD1@risk.sungard.com>
Message-ID: <nbhefnx5b0oe.dlg@usenet.alexanderweb.de>

Hi Ben Young,

On Fri, 14 Jul 2006 12:12:59 +0100, Ben.Young at risk.sungard.com wrote:

> This looks really good! It shows there are still some major (easy) 
> performance wins in PyPy.

Yes, I measured 35% speedup on richards/pystone on "the other branch" (that 
solely knows this kind of dictionary unlike the current trunk).

> One question, why would you need any calls to str2object in any of the 
> non-mutating methods?

Because every class can define it's own hash/eq functions that might be 
considered insane in case of having them return hashes that match the 
hashes of a string. That would mean that {"foo": 1}[myClass()] could return 
1. Therefore we have to switch the mode while having a shortcut for types 
which are known to have sane hashes only.

>                 w_lookup.__hash__()

I didn't even consider that people would write __hash__ methods with 
side-effects :)

Kind regards,
Alexander, who has resent this posting 2 days after the initial try - Gmane
problems?



From Ben.Young at risk.sungard.com  Mon Jul 17 09:56:38 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Mon, 17 Jul 2006 08:56:38 +0100
Subject: [pypy-dev] [pypy-svn] r30034 - pypy/dist/pypy/objspace/std
In-Reply-To: <nbhefnx5b0oe.dlg@usenet.alexanderweb.de>
Message-ID: <OF3239899F.71F77371-ON802571AE.002B5B07-802571AE.002BA2EC@risk.sungard.com>

pypy-dev-bounces at codespeak.net wrote on 16/07/2006 22:40:01:

> Hi Ben Young,

Hi Alexander,

> 
> On Fri, 14 Jul 2006 12:12:59 +0100, Ben.Young at risk.sungard.com wrote:
> 
> > This looks really good! It shows there are still some major (easy) 
> > performance wins in PyPy.
> 
> Yes, I measured 35% speedup on richards/pystone on "the other branch" 
(that 
> solely knows this kind of dictionary unlike the current trunk).
> 
> > One question, why would you need any calls to str2object in any of the 

> > non-mutating methods?
> 
> Because every class can define it's own hash/eq functions that might be 
> considered insane in case of having them return hashes that match the 
> hashes of a string. That would mean that {"foo": 1}[myClass()] could 
return 
> 1. Therefore we have to switch the mode while having a shortcut for 
types 
> which are known to have sane hashes only.

Ouch! Yes, I see the problem now. Roll on parameretised collections :)

> 
> >                 w_lookup.__hash__()
> 
> I didn't even consider that people would write __hash__ methods with 
> side-effects :)
> 
> Kind regards,
> Alexander, who has resent this posting 2 days after the initial try - 
Gmane
> problems?
>

Not me!
 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 



From arigo at tunes.org  Mon Jul 17 20:02:57 2006
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 17 Jul 2006 20:02:57 +0200
Subject: [pypy-dev] Security ideas
In-Reply-To: <20060713060229.GR32712@solar.trillke>
References: <20060524114728.GA6374@code0.codespeak.net>
	<20060713060229.GR32712@solar.trillke>
Message-ID: <20060717180257.GA18722@code0.codespeak.net>

Hi Holger,

On Thu, Jul 13, 2006 at 08:02:29AM +0200, holger krekel wrote:
>     def enter_bid(n):
>         if n > highest_bid.value:
>             highest_bid.value = n 
>     enter_bid = secure(enter_bid)
> 
> Here the annotator analysis is supposed to prevent a leak of information
> from the secret value.  But if the if-branch additionally contains:
> 
>             num_bids += 1
> 
> don't you run into a branching/code-dependent-on-secret-condition 
> problem again?  Would the annotator prevent the manipulation of 
> the global 'num_bids'?  Would it need to be a public value? 

You can't modify global values in RPython anyway.  But more generally,
yes, the annotator would follow all mutations and propagate security
levels.

> Moreover, i have practical concerns: your proposed
> scheme requires RPython annotator analysis which implies to
> have the PyPy tool chain available and accessible at programming 
> time.  Not impossible but also not a use case that we went for 
> so far.

Well, we haven't considered much at all.  This is just one approach,
which we didn't think of earlier, but there are of course others.

> Also it is not clear to which target "secure" would compile 
> functions to, C or bytecode or ...?

I don't know.  That's not too important for the security aspect;
anything would work, maybe even to the point of not actually compiling
at all but just using the annotator for the security proof (although I'd
regard this as less secure somehow, given the subtle differences between
RPython and full Python).


A bientot,

Armin


From hpk at trillke.net  Tue Jul 18 08:34:21 2006
From: hpk at trillke.net (holger krekel)
Date: Tue, 18 Jul 2006 08:34:21 +0200
Subject: [pypy-dev] Security ideas
In-Reply-To: <20060717180257.GA18722@code0.codespeak.net>
References: <20060524114728.GA6374@code0.codespeak.net>
	<20060713060229.GR32712@solar.trillke>
	<20060717180257.GA18722@code0.codespeak.net>
Message-ID: <20060718063421.GS28134@solar.trillke.net>

Hi Armin, 

On Mon, Jul 17, 2006 at 20:02 +0200, Armin Rigo wrote:
> On Thu, Jul 13, 2006 at 08:02:29AM +0200, holger krekel wrote:
> >     def enter_bid(n):
> >         if n > highest_bid.value:
> >             highest_bid.value = n 
> >     enter_bid = secure(enter_bid)
> > 
> > Here the annotator analysis is supposed to prevent a leak of information
> > from the secret value.  But if the if-branch additionally contains:
> > 
> >             num_bids += 1
> > 
> > don't you run into a branching/code-dependent-on-secret-condition 
> > problem again?  Would the annotator prevent the manipulation of 
> > the global 'num_bids'?  Would it need to be a public value? 
> 
> You can't modify global values in RPython anyway.  But more generally,
> yes, the annotator would follow all mutations and propagate security
> levels.

Hum, sorry for having asked too many questions at once.  Obviously, 
i could have said "whatever.num_bids +=1". 
Anyway, it seems that we cannot avoid the "dependent code block" issue 
but you consider the annotator more suited than an object space 
to deal with it, right? 

Also, from your answers i gather that your focus is more on 
getting something at-or-beyond-state-of-the-art first rather
than to provide something directly practical.  Probably makes
sense. 

best, 

    holger


From elmo13 at jippii.fi  Wed Jul 19 15:11:58 2006
From: elmo13 at jippii.fi (=?UTF-8?B?RWxtbyBNw6RudHluZW4=?=)
Date: Wed, 19 Jul 2006 16:11:58 +0300
Subject: [pypy-dev] translation with --profopt fails
Message-ID: <44BE2F9E.4060306@jippii.fi>

Last lines of output (python translate.py --backend=c --profopt='-c
"from richards import *;main(iterations=1)"' --text --batch
targetpypystandalone.py):

"""

[flowgraph] (pypy.rpython.memory.gctransform:733)ll_finalizer
[flowgraph] (pypy.rpython.memory.gctransform:733)ll_finalizer
[flowgraph] (pypy.rpython.memory.gctransform:733)ll_finalizer
XXX: operation v805 = direct_call((<* fn eq__Long_Long>), (StdObjSpace),
arg0_626, v804) cannot raise, but has exception guarding in graph
(?:1)_mm_eq_longS0_W_LongObject_W_BoolObject
XXX: operation v807 = direct_call((<* fn eq__Long_Long>), (StdObjSpace),
arg0_629, v806) cannot raise, but has exception guarding in graph
(?:1)_mm_eq_longS0_W_LongObject_W_IntObject
XXX: operation v809 = direct_call((<* fn eq__Long_Long>), (StdObjSpace),
arg0_631, v808) cannot raise, but has exception guarding in graph
(?:1)_mm_eq_longS0_W_LongObject_W_LongObject
[c]    28000 nodes  [ array: 3816  boehm rtti: 120  func: 2075  struct:
29285 ]
[c]    29000 nodes  [ array: 4045  boehm rtti: 120  func: 2149  struct:
29980 ]
[c]    30000 nodes  [ array: 4129  boehm rtti: 120  func: 2186  struct:
31192 ]
[c]    31000 nodes  [ array: 4257  boehm rtti: 120  func: 2201  struct:
32700 ]
[c]    32000 nodes  [ array: 4419  boehm rtti: 137  func: 2279  struct:
33433 ]
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 335, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/driver.py",
line 509, in proceed
[translation:ERROR]     return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/tool/taskengine.py",
line 108, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/driver.py",
line 197, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/driver.py",
line 301, in task_database_c
[translation:ERROR]     database = cbuilder.build_database()
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c/genc.py",
line 86, in build_database
[translation:ERROR]     db.complete()
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c/database.py",
line 302, in complete
[translation:ERROR]     add_dependencies(node.enum_dependencies())
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c/database.py",
line 288, in add_dependencies
[translation:ERROR]     self.get(value)
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c/database.py",
line 179, in get
[translation:ERROR]     if obj:   # test if the ptr is non-NULL
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/rpython/lltypesystem/lltype.py",
line 886, in __nonzero__
[translation:ERROR]     return self._obj is not None
[translation:ERROR]    File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/rpython/lltypesystem/lltype.py",
line 910, in _getobj
[translation:ERROR]     raise RuntimeError("accessing already garbage
collected %r"
[translation:ERROR]  RuntimeError: accessing already garbage collected
<Array of Char >
"""

Just in case no one noticed this already...

Elmo


From leo.soto at gmail.com  Wed Jul 19 19:23:07 2006
From: leo.soto at gmail.com (Leonardo Soto)
Date: Wed, 19 Jul 2006 13:23:07 -0400
Subject: [pypy-dev] Implenting -m command line option
Message-ID: <5a8cc9e00607191023i75f4e6a6r79eda06a610b8944@mail.gmail.com>

Hi all,

I'm new to pypy, and to get myself started I'm trying to fix #145,
that is, implement the -m flag.

So I did search on the web and python sources looking for what exactly
-m means, and PEP-338[1] has the better explanation:

"If the module is found, and is of type PY_SOURCE or PY_COMPILED, then
the command line is effectively reinterpreted from python <options> -m
<module> <args> to python <options> <filename> <args>. This includes
setting sys.argv[0] correctly (some scripts rely on this - Python's
own regrtest.py is one example)."

But, by the way, PEP-338 is about generalizing -m to fully support
import hooks and executing modules inside packages, including a
reference implementation almost entirely written as a python module
(runpy). It has been included recently on CPython 2.5.

So, here is my question:

What "-m behaviour" should be implemented on pypy?

-- 
Leonardo Soto M.

[1] http://www.python.org/dev/peps/pep-0338/


From mwh at python.net  Wed Jul 19 21:59:43 2006
From: mwh at python.net (Michael Hudson)
Date: Wed, 19 Jul 2006 20:59:43 +0100
Subject: [pypy-dev] Implenting -m command line option
References: <5a8cc9e00607191023i75f4e6a6r79eda06a610b8944@mail.gmail.com>
Message-ID: <2md5c11i33.fsf@starship.python.net>

"Leonardo Soto" <leo.soto at gmail.com> writes:

> Hi all,
>
> I'm new to pypy, and to get myself started I'm trying to fix #145,
> that is, implement the -m flag.
>
> So I did search on the web and python sources looking for what exactly
> -m means, and PEP-338[1] has the better explanation:
>
> "If the module is found, and is of type PY_SOURCE or PY_COMPILED, then
> the command line is effectively reinterpreted from python <options> -m
> <module> <args> to python <options> <filename> <args>. This includes
> setting sys.argv[0] correctly (some scripts rely on this - Python's
> own regrtest.py is one example)."
>
> But, by the way, PEP-338 is about generalizing -m to fully support
> import hooks and executing modules inside packages, including a
> reference implementation almost entirely written as a python module
> (runpy). It has been included recently on CPython 2.5.
>
> So, here is my question:
>
> What "-m behaviour" should be implemented on pypy?

To be honest, either would be great.  The runpy-based solution sounds
easier, so maybe that?

Cheers,
mwh

-- 
41. Some programming languages manage to absorb change, but
    withstand progress.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From anto.cuni at gmail.com  Thu Jul 20 00:53:50 2006
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 20 Jul 2006 00:53:50 +0200
Subject: [pypy-dev] Tomorrow morning...
Message-ID: <44BEB7FE.3060903@gmail.com>

... I will graduate!! :-) :-) :-)

A big, big 'thank you' to all pypy developers, for having welcomed and 
supported me during last months: it has been very nice to work with you 
(and I hope to continue in future).

I still remember my first post on pypy-dev: I was "Looking for a thesis" 
(cit.) and I was rather confused about pypy architecture: it was only 5 
months ago, but it seems years. Then I started coding and I've never 
stopped :-).

For those interested in, the final version of my thesis is here:
http://codespeak.net/svn/user/antocuni/tesi/Implementing%20Python%20in%20.NET.pdf

ciao Anto


From hpk at trillke.net  Thu Jul 20 07:05:45 2006
From: hpk at trillke.net (holger krekel)
Date: Thu, 20 Jul 2006 07:05:45 +0200
Subject: [pypy-dev] Tomorrow morning...
In-Reply-To: <44BEB7FE.3060903@gmail.com>
References: <44BEB7FE.3060903@gmail.com>
Message-ID: <20060720050545.GC27182@solar.trillke.net>

Hi Antonio!

On Thu, Jul 20, 2006 at 00:53 +0200, Antonio Cuni wrote:
> ... I will graduate!! :-) :-) :-)

Woa! Big congrats! :))
 
> A big, big 'thank you' to all pypy developers, for having welcomed and 
> supported me during last months: it has been very nice to work with you 
> (and I hope to continue in future).

same here with you! 

> I still remember my first post on pypy-dev: I was "Looking for a thesis" 
> (cit.) and I was rather confused about pypy architecture: it was only 5 
> months ago, but it seems years. Then I started coding and I've never 
> stopped :-).
> 
> For those interested in, the final version of my thesis is here:
> http://codespeak.net/svn/user/antocuni/tesi/Implementing%20Python%20in%20.NET.pdf

just printed it and will see to read it sometime in the sun :) 

best & see you, 

    holger


From l.oluyede at gmail.com  Thu Jul 20 09:25:16 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Thu, 20 Jul 2006 09:25:16 +0200
Subject: [pypy-dev] Tomorrow morning...
In-Reply-To: <44BEB7FE.3060903@gmail.com>
References: <44BEB7FE.3060903@gmail.com>
Message-ID: <9eebf5740607200025x29d5675flde43353894113e95@mail.gmail.com>

On 7/20/06, Antonio Cuni <anto.cuni at gmail.com> wrote:
> ... I will graduate!! :-) :-) :-)

Congratulations Antonio :-)

-- 
Lawrence
http://www.oluyede.org/blog


From Ben.Young at risk.sungard.com  Thu Jul 20 09:29:07 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 20 Jul 2006 08:29:07 +0100
Subject: [pypy-dev] .NET build failures
Message-ID: <OFFD8947AE.9DE95F27-ON802571B1.00289D40-802571B1.00291DAE@risk.sungard.com>

Hi Antocuni,

I ran the .NET build last night and it got a lot further. The errors I get 
now are:

[translation:ERROR] ***** FAILURE *****
[translation:ERROR]
[translation:ERROR] 
c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(66076) : error -- 
Duplicate label: '__check_block_5'
[translation:ERROR] 
c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(372039) : error -- 
Duplicate label: '__check_block_5'
[translation:ERROR] 
c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(821924) : error -- 
Duplicate label: '__check_block_7'
[translation:ERROR] 
c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(821985) : error -- 
Duplicate label: '__check_block_3'
[translation:ERROR] 
c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(822900) : error -- 
Duplicate label: '__check_block_7'
[translation:ERROR] 
c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(822961) : error -- 
Duplicate label: '__check_block_3'

I also notice that you appear to have identifiers with hundreds of 
underscores appended. Is this expected?

[translation:ERROR] Assembled global method 
memo_get_unique_interplevel_subclass_4______________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________________________________________________________________________________________
____________________________________

Cheers,
Ben


From mwh at python.net  Thu Jul 20 10:05:14 2006
From: mwh at python.net (Michael Hudson)
Date: Thu, 20 Jul 2006 09:05:14 +0100
Subject: [pypy-dev] Tomorrow morning...
References: <44BEB7FE.3060903@gmail.com>
Message-ID: <2m8xmo1z2d.fsf@starship.python.net>

Antonio Cuni <anto.cuni at gmail.com> writes:

> ... I will graduate!! :-) :-) :-)
>
> A big, big 'thank you' to all pypy developers, for having welcomed and 
> supported me during last months: it has been very nice to work with you 
> (and I hope to continue in future).

No, thank you for all you've done!

Cheers,
mwh

-- 
  C++ is a siren song.  It *looks* like a HLL in which you ought to
  be able to write an application, but it really isn't.
                                       -- Alain Picard, comp.lang.lisp



From mwh at python.net  Thu Jul 20 10:16:21 2006
From: mwh at python.net (Michael Hudson)
Date: Thu, 20 Jul 2006 09:16:21 +0100
Subject: [pypy-dev] return of the return of pypy-sync
Message-ID: <2m4pxc1yju.fsf@starship.python.net>

Once again, I'd like to invite all active PyPy developers to a short
(no more than half an hour) sychronisation meeting in #pypy-sync on
freenode at 17:30 UTC+2 today.

Agenda:

 - status

   pre-prepared 3-line reports of what you did last week, what you
   plan to do next week and what is blocking your progress at this
   time.

   this is the area i see as the most important this week: work has
   begun to spread in many directions, and it's a good idea to have an
   idea who is doing what and where they've got to.

 - work out how to resolve any blockers above

 - summer of pypy

   the summer of pypy program is ready to be announced:

   http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html

   there's not too much to discuss, but the "project ideas" page:

   http://codespeak.net/pypy/dist/pypy/doc/independent-project-ideas.html

   could use an update, we can talk about ideas for that.

 - ireland sprint

   again, mostly an information point: our next scheduled sprint in
   Ireland towards the end of August will be announced soon:

   http://codespeak.net/pypy/extradoc/sprintinfo/ireland-2006/announce.html

   so maybe you can start thinking about travel arrangements already.
   it is looking likely that the sprint part will not be attended by a
   large number of locals, so it's likely to be a fairly "core"
   sprint.

See you there!

Cheers,
mwh

-- 
  The "of course, while I have no problem with this at all, it's
  surely too much for a lesser being" flavor of argument always
  rings hollow to me.                       -- Tim Peters, 29 Apr 1998



From Ben.Young at risk.sungard.com  Thu Jul 20 15:08:02 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 20 Jul 2006 14:08:02 +0100
Subject: [pypy-dev] .NET build failures
In-Reply-To: <OFFD8947AE.9DE95F27-ON802571B1.00289D40-802571B1.00291DAE@risk.sungard.com>
Message-ID: <OFD704530D.0D116AD9-ON802571B1.0047CBEC-802571B1.00482551@risk.sungard.com>

To reply to myself :)

The problem appears to be in the _check function in opcodes.py. This 
generates a number for the jump at opcode creation time. Therefore, if an 
opcode is used twice in a block then you get the error.

I guess the solution would be to create a "_check" micro instruction that 
increments the count on render. I'm afraid I haven't got time to actually 
do that myself though.

Cheers,
Ben


pypy-dev-bounces at codespeak.net wrote on 20/07/2006 08:29:07:

> Hi Antocuni,
> 
> I ran the .NET build last night and it got a lot further. The errors I 
get 
> now are:
> 
> [translation:ERROR] ***** FAILURE *****
> [translation:ERROR]
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(66076) : error 
-- 
> Duplicate label: '__check_block_5'
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(372039) : error 
-- 
> Duplicate label: '__check_block_5'
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(821924) : error 
-- 
> Duplicate label: '__check_block_7'
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(821985) : error 
-- 
> Duplicate label: '__check_block_3'
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(822900) : error 
-- 
> Duplicate label: '__check_block_7'
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(822961) : error 
-- 
> Duplicate label: '__check_block_3'
> 
> I also notice that you appear to have identifiers with hundreds of 
> underscores appended. Is this expected?
> 
> [translation:ERROR] Assembled global method 
> 
memo_get_unique_interplevel_subclass_4______________________________________________________________________________
> 
________________________________________________________________________________________________________________________________________________________________
> 
________________________________________________________________________________________________________________________________________________________________
> ____________________________________
> 
> Cheers,
> Ben
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 



From arigo at tunes.org  Thu Jul 20 17:59:12 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 20 Jul 2006 17:59:12 +0200
Subject: [pypy-dev] Tomorrow morning...
In-Reply-To: <44BEB7FE.3060903@gmail.com>
References: <44BEB7FE.3060903@gmail.com>
Message-ID: <20060720155912.GA30518@code0.codespeak.net>

Hi Antonio,

On Thu, Jul 20, 2006 at 12:53:50AM +0200, Antonio Cuni wrote:
> ... I will graduate!! :-) :-) :-)

Congratulation!

> A big, big 'thank you' to all pypy developers, for having welcomed and 
> supported me during last months: it has been very nice to work with you 
> (and I hope to continue in future).

Thanks to you, you are doing quite a great job :-)

Armin


From 2006a at usenet.alexanderweb.de  Fri Jul 21 02:23:10 2006
From: 2006a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Fri, 21 Jul 2006 02:23:10 +0200
Subject: [pypy-dev] [pypy-svn] r30034 - pypy/dist/pypy/objspace/std
References: <20060714105155.DC18A10090@code0.codespeak.net>
	<OFED1A6F46.5AACC121-ON802571AB.003D1B50-802571AB.003D9CD1@risk.sungard.com>
	<nbhefnx5b0oe.dlg@usenet.alexanderweb.de>
Message-ID: <no0jbpy9l9pv$.dlg@usenet.alexanderweb.de>

On Sun, 16 Jul 2006 23:40:01 +0200, I wrote:

>> This looks really good! It shows there are still some major (easy) 
>> performance wins in PyPy.
> 
> Yes, I measured 35% speedup on richards/pystone on "the other branch" (that 
> solely knows this kind of dictionary unlike the current trunk).

OK, newest benchmarks show a stable 32% speedup at richards.py, even
working on the trunk - you just need to enable the strdict flag like this
in the target:

    config.objspace.std.withstrdict = True

This gives the following factors for pypy-c vs. cpython (measured on
pypy2):
Richards: 2.98x as slow
PyStone: 3.5x as slow

Now I am looking forward to have someone building an LLVM/C/Profile guided
opt binary with strdicts enabled :-)

Kind regards,
Alexander



From hpk at trillke.net  Fri Jul 21 13:20:39 2006
From: hpk at trillke.net (holger krekel)
Date: Fri, 21 Jul 2006 13:20:39 +0200
Subject: [pypy-dev] Summer of PyPy: call for proposals
Message-ID: <20060721112039.GC27182@solar.trillke.net>

Hi to all students! 

Happily, we are able to offer students mentoring and full sprint
participant's funding if we receive a proposal outlining an
interesting project related to PyPy and its development tools.  This
follows up on the "Summer of Code" campaign from Google but is
completely independent from it and also works differently (i.e. there
is no success bounty).

See the full call for details: 

    http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html

and here is a list of concrete inspirations for possible work: 

    http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html

There are certainly many more possibilities now that PyPy starts to
get faster, Stackless and custom GCs are integrated, PyPy.NET, Logic
Variables, lazy evaluation and other new features are emerging.  Note
though that PyPy still is pursuing challenging research goals so it
will take some time before it becomes viable in production
environments.

Don't hesitate to drop by on #pypy on freenode or post questions/ideas
to pypy-dev at codespeak net.

A last note: you can propose something at any time, you don't need to
wait for a specific deadline. In fact, our next sprint is nearing in
Ireland, 21st-27th August and if you are quick you may already be
there with us :)

For the PyPy team, 

    Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, ... 



From bea at changemaker.nu  Fri Jul 21 13:40:42 2006
From: bea at changemaker.nu (Beatrice During)
Date: Fri, 21 Jul 2006 13:40:42 +0200
Subject: [pypy-dev] PyPy sprint Limerick 21-27th of August
Message-ID: <44C0BD3A.3060300@changemaker.nu>

Hi there

The PyPy team goes sprinting in Limerick/Ireland  - are
you interested in participating?


Ireland PyPy sprint 21th-27th August 2006
====================================================

The next PyPy sprint will happen in the nice city of
Limerick in Ireland from 21st till 27th August.
(Most people intend to arrive 20th August).

The main focus of the sprint will be on JIT compiler works,
various optimization works, porting extension modules,
infrastructure works like a build tool for PyPy, or
extended (distributed) testing.

It's also open to new topics.  If you are a student
consider to participate in `Summer of PyPy` in order
get funding for your travels and accomodation.

The sprint is being hosted by University of Limerick
(http://www.ul.ie/) - and is arranged in co-operation
with people from our sister project Calibre (www.calibre.ie).
Our contact at the University is P?r ?gerfalk and Eoin
Oconchuir.

For more information on logistics, facilities etc - please
read the full announcement at:

http://codespeak.net/pypy/extradoc/sprintinfo/ireland-2006/announce.html

For the PyPy team

Bea & Holger



From arigo at tunes.org  Fri Jul 21 17:29:39 2006
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 21 Jul 2006 17:29:39 +0200
Subject: [pypy-dev] translation with --profopt fails
In-Reply-To: <44BE2F9E.4060306@jippii.fi>
References: <44BE2F9E.4060306@jippii.fi>
Message-ID: <20060721152939.GB31604@code0.codespeak.net>

Hi Elmo,

On Wed, Jul 19, 2006 at 04:11:58PM +0300, Elmo M??ntynen wrote:
> Last lines of output (python translate.py --backend=c --profopt='-c
> "from richards import *;main(iterations=1)"' --text --batch
> targetpypystandalone.py):

I have no clue how --profopt can affect the translation itself.  You
don't get the same error without the --profopt option?


A bientot,

Armin


From elmo13 at jippii.fi  Fri Jul 21 18:07:38 2006
From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=)
Date: Fri, 21 Jul 2006 19:07:38 +0300
Subject: [pypy-dev] translation with --profopt fails
In-Reply-To: <20060721152939.GB31604@code0.codespeak.net>
References: <44BE2F9E.4060306@jippii.fi>
	<20060721152939.GB31604@code0.codespeak.net>
Message-ID: <44C0FBCA.1090605@jippii.fi>

Armin Rigo wrote:
> Hi Elmo,
>
> On Wed, Jul 19, 2006 at 04:11:58PM +0300, Elmo M??ntynen wrote:
>   
>> Last lines of output (python translate.py --backend=c --profopt='-c
>> "from richards import *;main(iterations=1)"' --text --batch
>> targetpypystandalone.py):
>>     
>
> I have no clue how --profopt can affect the translation itself.  You
> don't get the same error without the --profopt option?
>
>
> A bientot,
>
> Armin
>   
Should've tested that, will do as soon as possible. benchmark.html on
snake shows something that says to me that the profopt executable
doesn't exist, which suggests that it wasn't built.

Elmo



From anto.cuni at gmail.com  Fri Jul 21 21:07:26 2006
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 21 Jul 2006 21:07:26 +0200
Subject: [pypy-dev] .NET build failures
In-Reply-To: <OFFD8947AE.9DE95F27-ON802571B1.00289D40-802571B1.00291DAE@risk.sungard.com>
References: <OFFD8947AE.9DE95F27-ON802571B1.00289D40-802571B1.00291DAE@risk.sungard.com>
Message-ID: <44C125EE.1000900@gmail.com>

Ben.Young at risk.sungard.com wrote:
> Hi Antocuni,

Hi Ben,

> I ran the .NET build last night and it got a lot further. The errors I get 
> now are:
> 
> [translation:ERROR] ***** FAILURE *****
> [translation:ERROR]
> [translation:ERROR] 
> c:\docume~1\ben~1.you\locals~1\temp\usession-16\main.il(66076) : error -- 
> Duplicate label: '__check_block_5'

as you correctly spotted out, the problem was in opcodes._check; I've 
fixed that, thanks for the help.

> I also notice that you appear to have identifiers with hundreds of 
> underscores appended. Is this expected?
> 
> [translation:ERROR] Assembled global method 
> memo_get_unique_interplevel_subclass_4______________________________________________________________________________
> ________________________________________________________________________________________________________________________________________________________________
> ________________________________________________________________________________________________________________________________________________________________
> ____________________________________

Yes, it's expected. Appending underscores it's a way to get unique names 
for methods and classes, though I didn't think to get such a big number 
of underscores. I changed the code and now I append a unique number at 
the end of the name instead of a list of underscores.

ciao Anto



From elmo13 at jippii.fi  Fri Jul 21 22:26:53 2006
From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=)
Date: Fri, 21 Jul 2006 23:26:53 +0300
Subject: [pypy-dev] translation with --profopt fails
In-Reply-To: <44C0FBCA.1090605@jippii.fi>
References: <44BE2F9E.4060306@jippii.fi>	<20060721152939.GB31604@code0.codespeak.net>
	<44C0FBCA.1090605@jippii.fi>
Message-ID: <44C1388D.80003@jippii.fi>

Elmo M?ntynen wrote:
> Armin Rigo wrote:
>   
>> Hi Elmo,
>>
>> On Wed, Jul 19, 2006 at 04:11:58PM +0300, Elmo M??ntynen wrote:
>>   
>>     
>>> Last lines of output (python translate.py --backend=c --profopt='-c
>>> "from richards import *;main(iterations=1)"' --text --batch
>>> targetpypystandalone.py):
>>>     
>>>       
>> I have no clue how --profopt can affect the translation itself.  You
>> don't get the same error without the --profopt option?
>>
>>
>> A bientot,
>>
>> Armin
>>   
>>     
> Should've tested that, will do as soon as possible. benchmark.html on
> snake shows something that says to me that the profopt executable
> doesn't exist, which suggests that it wasn't built.
>
> Elmo
>   
Translation without profopt worked now, though there might have been
something that got changed in between, will report after some more builds.

Elmo


From arigo at tunes.org  Fri Jul 21 22:36:13 2006
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 21 Jul 2006 22:36:13 +0200
Subject: [pypy-dev] Failing stackless tests
Message-ID: <20060721203613.GA2825@code0.codespeak.net>

Hi all, hi Aurelien,

The stackless-related tests are all consistently failing nowadays.  Some
have been failing for 4 days and others for 9 days.  See

   http://snake.cs.uni-duesseldorf.de/pypytest/summary.html

Given that you appear to continue to work on it, but without making
tests pass again, I'm a bit worrying exactly about what you are doing?
That doesn't seem too much in line with the "test-driven development"
approach.


A bientot,

Armin


From scott+pypy-dev at scottdial.com  Sat Jul 22 02:31:57 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Fri, 21 Jul 2006 20:31:57 -0400
Subject: [pypy-dev] Newb questions about ext module writing
Message-ID: <44C171FD.60801@scottdial.com>

Greetings everyone,

I'm relatively new to PyPy. I've been aware of it for a while, but never 
really got into poking around at all. But recently I decided this is a 
project that I would like to get involved in. I think at the time I 
planned on doing something big, like addding a JVM backend or something 
like that.. maybe I'll get back to that eventually. I realized I was 
fairly clueless with the codebase and decided to jump on writing a 
module as a type of introductory project.

The independent-project-ideas went out of its way to mention readline, 
so boom, readline is what I had/have chosen to finish. Things were going 
well until I started having to deal with making callback functions.

I have become stuck on the need to setup_readline with some callback 
functions which are capable of accessing the State of the module. I'm 
pretty lost on how to go about getting the state. Within the 
interp_readline, I do the "space.fromcache(State)" deal to get ahold of 
the State, but how do I do this from within the "c_readline" component. 
Perhaps this is completely the wrong way to go about this? My first 
thought is to import the "getstate" function from interp_readline, but I 
can't find anywhere else this is done in the codebase off-hand.

Furthermore, my callback functions from readline will not be passed any 
sort of "state", so how do I get that from within my callback function? 
Again, I thought a poor kludge was to tag reference to "state" onto the 
function during setup_readline.

def setup_readline(state, w_module):
     ...
     cfunc = CFUNCTYPE(POINTER(c_char_p), c_char_p, c_int, c_int)
     c_rl_attempted_completion_function = cfunc(space.flex_complete)
     space.flex_complete.space = space
     ...

def flex_complete(text, start, end):
     from pypy.module.readline.interp_readline import getstate
     getstate(flex_complete).begidx = start
     getstate(flex_complete).endidx = end
     ...

Some ugly stuff like that.. help? :) Thanks.

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu


From len-l at telus.net  Sat Jul 22 03:24:19 2006
From: len-l at telus.net (Lenard Lindstrom)
Date: Fri, 21 Jul 2006 18:24:19 -0700
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <44C171FD.60801@scottdial.com>
Message-ID: <44C11BD3.25178.4E9F16@len-l.telus.net>

On 21 Jul 2006 at 20:31, Scott Dial wrote:

> Furthermore, my callback functions from readline will not be passed any 
> sort of "state", so how do I get that from within my callback function? 
> Again, I thought a poor kludge was to tag reference to "state" onto the 
> function during setup_readline.
> 
> def setup_readline(state, w_module):
>      ...
>      cfunc = CFUNCTYPE(POINTER(c_char_p), c_char_p, c_int, c_int)
>      c_rl_attempted_completion_function = cfunc(space.flex_complete)
>      space.flex_complete.space = space
>      ...
> 
> def flex_complete(text, start, end):
>      from pypy.module.readline.interp_readline import getstate
>      getstate(flex_complete).begidx = start
>      getstate(flex_complete).endidx = end
>      ...
> 
In ctypes that is easy: use a bound method as the callback. Does that 
even work in rctypes?

Lenard Lindstrom
<len-l at telus.net>




From arigo at tunes.org  Sat Jul 22 12:24:42 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 22 Jul 2006 12:24:42 +0200
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <44C11BD3.25178.4E9F16@len-l.telus.net>
References: <44C171FD.60801@scottdial.com>
	<44C11BD3.25178.4E9F16@len-l.telus.net>
Message-ID: <20060722102442.GA8055@code0.codespeak.net>

Hi,

On Fri, Jul 21, 2006 at 06:24:19PM -0700, Lenard Lindstrom wrote:
> In ctypes that is easy: use a bound method as the callback. Does that 
> even work in rctypes?

I'm afraid not, and it's not obvious how this would be compiled to C
code, given that there is no kind of closure in ANSI C.

I don't see how to do that at all, actually :-(  It's somehow a design
problem of the readline lib, which should in theory always pass an extra
void* "closure" arguments to callbacks...

It might be necessary to use hacks like a single global State instance,
storing the "space" the readline lib was initialized with, instead of
one State per space.


Armin


From scott+pypy-dev at scottdial.com  Sat Jul 22 13:39:27 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Sat, 22 Jul 2006 07:39:27 -0400
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <20060722102442.GA8055@code0.codespeak.net>
References: <44C171FD.60801@scottdial.com>	<44C11BD3.25178.4E9F16@len-l.telus.net>
	<20060722102442.GA8055@code0.codespeak.net>
Message-ID: <44C20E6F.2080608@scottdial.com>

Armin Rigo wrote:
> On Fri, Jul 21, 2006 at 06:24:19PM -0700, Lenard Lindstrom wrote:
>> In ctypes that is easy: use a bound method as the callback. Does that 
>> even work in rctypes?
> 
> I'm afraid not, and it's not obvious how this would be compiled to C
> code, given that there is no kind of closure in ANSI C.
> 
> I don't see how to do that at all, actually :-(  It's somehow a design
> problem of the readline lib, which should in theory always pass an extra
> void* "closure" arguments to callbacks...
> 
> It might be necessary to use hacks like a single global State instance,
> storing the "space" the readline lib was initialized with, instead of
> one State per space.

I am admittedly ignorant of rctypes capabilities, but I can appreciate
the conversion in C code being awkward. Reading around more, I see in
rctypes.implementation where an attempt for CALLBACK_FUNCTYPE was
droppped. Seems like I have bitten off more than I can chew here.

I think you understate the issue by saying it's "a design problem of the
readline lib" because many, many C libraries are going to need callback
functionality. If I have evaluated the project's current state
correctly, then I realize this is more-or-less not the current focus of
development. So, no biggie.

I just figured it would something I could just go through the motions on
to become familiar. I am open to other suggestions. I can comeback to
this when I have a better understanding of everything.

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu



From mahs at telcopartners.com  Sat Jul 22 23:17:46 2006
From: mahs at telcopartners.com (Michael Spencer)
Date: Sat, 22 Jul 2006 14:17:46 -0700
Subject: [pypy-dev] 20th Level IP Communicator?
Message-ID: <e9u4la$54j$1@sea.gmane.org>

http://gigaom.com/2006/07/15/sipphone-brings-voice-to-nokia-770-tablet/#comment-125818

I thought you'd like this comment:

"looks like i?m about to fail my saving throw against ?cool gadget temptation?."



From mahs at telcopartners.com  Sat Jul 22 23:21:52 2006
From: mahs at telcopartners.com (Michael Spencer)
Date: Sat, 22 Jul 2006 14:21:52 -0700
Subject: [pypy-dev] 20th Level IP Communicator?
In-Reply-To: <e9u4la$54j$1@sea.gmane.org>
References: <e9u4la$54j$1@sea.gmane.org>
Message-ID: <e9u4sv$54j$2@sea.gmane.org>

Michael Spencer wrote:
> http://gigaom.com/2006/07/15/sipphone-brings-voice-to-nokia-770-tablet/#comment-125818
> 
> I thought you'd like this comment:
> 
> "looks like i?m about to fail my saving throw against ?cool gadget temptation?."
> 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 
Sorry - sent to the wrong address
Michael



From arigo at tunes.org  Sun Jul 23 09:18:38 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 23 Jul 2006 09:18:38 +0200
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <44C20E6F.2080608@scottdial.com>
References: <44C171FD.60801@scottdial.com>
	<44C11BD3.25178.4E9F16@len-l.telus.net>
	<20060722102442.GA8055@code0.codespeak.net>
	<44C20E6F.2080608@scottdial.com>
Message-ID: <20060723071838.GA29559@code0.codespeak.net>

Hi Scott,

On Sat, Jul 22, 2006 at 07:39:27AM -0400, Scott Dial wrote:
> I am admittedly ignorant of rctypes capabilities, but I can appreciate
> the conversion in C code being awkward. Reading around more, I see in
> rctypes.implementation where an attempt for CALLBACK_FUNCTYPE was
> droppped. Seems like I have bitten off more than I can chew here.

Callbacks by themselves should "almost" work.  We need to re-enable the
commented-out code and try -- this code is lot complexity, some of it
bordering on the hackish side, which turned out not to be needed any
more at some point.  So to ease development of the rest of rctypes I
disabled the tests and commented it out.  But of course callbacks are
clearly needed in the long run.

Was I was complaining about is that there is no C-convertable way to
have e.g. bound methods as callbacks.  You can only use plain functions.
Many C libs have the same "design problem" as readline and don't pass
you a "void*" argument back.  An example of lib that is "correctly
designed" in this respect is the Windows API.

So for now I suppose these cases can be handled by just assuming that
there is only one 'space' around, and storing it in global state.


A bientot,

Armin.


From len-l at telus.net  Sun Jul 23 19:56:06 2006
From: len-l at telus.net (Lenard Lindstrom)
Date: Sun, 23 Jul 2006 10:56:06 -0700
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <20060722102442.GA8055@code0.codespeak.net>
References: <44C11BD3.25178.4E9F16@len-l.telus.net>
Message-ID: <44C355C6.11938.7869AE@len-l.telus.net>

On 22 Jul 2006 at 12:24, Armin Rigo wrote:

> Hi,
> 
> On Fri, Jul 21, 2006 at 06:24:19PM -0700, Lenard Lindstrom wrote:
> > In ctypes that is easy: use a bound method as the callback. Does that 
> > even work in rctypes?
> 
> I'm afraid not, and it's not obvious how this would be compiled to C
> code, given that there is no kind of closure in ANSI C.
> 
I just brought it up as it is one more thing to consider when 
converting a ctypes Python module to rctypes. But could rctypes use 
ffi when the callback is a bound method?

Lenard Lindstrom
<len-l at telus.net>




From l.oluyede at gmail.com  Sun Jul 23 19:58:50 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Sun, 23 Jul 2006 19:58:50 +0200
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <44C355C6.11938.7869AE@len-l.telus.net>
References: <44C11BD3.25178.4E9F16@len-l.telus.net>
	<20060722102442.GA8055@code0.codespeak.net>
	<44C355C6.11938.7869AE@len-l.telus.net>
Message-ID: <9eebf5740607231058w2a2b688bpb5c746c01750ad96@mail.gmail.com>

> I just brought it up as it is one more thing to consider when
> converting a ctypes Python module to rctypes. But could rctypes use
> ffi when the callback is a bound method?

AFAIK one of the reasons of the existence of rctypes and the ext.
compiler is not to use the ffi library...

-- 
Lawrence
http://www.oluyede.org/blog


From elmo13 at jippii.fi  Sun Jul 23 20:43:05 2006
From: elmo13 at jippii.fi (=?windows-1252?Q?Elmo_M=E4ntynen?=)
Date: Sun, 23 Jul 2006 21:43:05 +0300
Subject: [pypy-dev] A bug in cc? (was: "translation with --profopt
	fails")
In-Reply-To: <44C1388D.80003@jippii.fi>
References: <44BE2F9E.4060306@jippii.fi>	<20060721152939.GB31604@code0.codespeak.net>	<44C0FBCA.1090605@jippii.fi>
	<44C1388D.80003@jippii.fi>
Message-ID: <44C3C339.8070809@jippii.fi>

Elmo M?ntynen wrote:
> Elmo M?ntynen wrote:
>   
>> Armin Rigo wrote:
>>   
>>     
>>> Hi Elmo,
>>>
>>> On Wed, Jul 19, 2006 at 04:11:58PM +0300, Elmo M??ntynen wrote:
>>>   
>>>     
>>>       
>>>> Last lines of output (python translate.py --backend=c --profopt='-c
>>>> "from richards import *;main(iterations=1)"' --text --batch
>>>> targetpypystandalone.py):
>>>>     
>>>>       
>>>>         
>>> I have no clue how --profopt can affect the translation itself.  You
>>> don't get the same error without the --profopt option?
>>>
>>>
>>> A bientot,
>>>
>>> Armin
>>>   
>>>     
>>>       
>> Should've tested that, will do as soon as possible. benchmark.html on
>> snake shows something that says to me that the profopt executable
>> doesn't exist, which suggests that it wasn't built.
>>
>> Elmo
>>   
>>     
> Translation without profopt worked now, though there might have been
> something that got changed in between, will report after some more builds.
>   
Now I got an internal compiler error from cc:
"""
[cbuild:profopt] Gathering profile data from:
/tmp/usession-11/testing_1/testing_1 -c "from richards import
*;main(iterations=1)"
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c testing_1.c -o
testing_1.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c structimpl.c -o
structimpl.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c nonfuncnodes.c
-o nonfuncnodes.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_1.c -o nonfuncnodes_1.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_2.c -o nonfuncnodes_2.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_3.c -o nonfuncnodes_3.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_4.c -o nonfuncnodes_4.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_5.c -o nonfuncnodes_5.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_6.c -o nonfuncnodes_6.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_7.c -o nonfuncnodes_7.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_8.c -o nonfuncnodes_8.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_9.c -o nonfuncnodes_9.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_10.c -o nonfuncnodes_10.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_11.c -o nonfuncnodes_11.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_12.c -o nonfuncnodes_12.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_13.c -o nonfuncnodes_13.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_14.c -o nonfuncnodes_14.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_15.c -o nonfuncnodes_15.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_16.c -o nonfuncnodes_16.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_17.c -o nonfuncnodes_17.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_18.c -o nonfuncnodes_18.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_19.c -o nonfuncnodes_19.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c
nonfuncnodes_20.c -o nonfuncnodes_20.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c implement.c -o
implement.o
[cbuild:execute] cc -O2 -pthread -fprofile-use
-I/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c
-I/usr/include/python2.4 -I/tmp/usession-11/testing_1 -c implement_1.c
-o implement_1.o
debug: entry point starting
debug: argv -> /tmp/usession-11/testing_1/testing_1
debug: argv -> -c "from richards import *;main(iterations=1)"
unrecognized option '-c "from richards import *;main(iterations=1)"'
usage: /tmp/usession-11/testing_1/testing_1 [options]
Try `/tmp/usession-11/testing_1/testing_1 -h` for more information.
implement_1.c: In function ?pypy_g_get_len_of_range?:
implement_1.c:137183: internal compiler error: Floating point exception
Please submit a full bug report,
with preprocessed source if appropriate.
See <URL:http://gcc.gnu.org/bugs.html> for instructions.
For Debian GNU/Linux specific bug reporting instructions,
see <URL:file:///usr/share/doc/gcc-4.0/README.Bugs>.

[translation:ERROR] Error:
[translation:ERROR] Traceback (most recent call last):
[translation:ERROR] File "translate.py", line 349, in main
[translation:ERROR] drv.proceed(goals)
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/driver.py",
line 508, in proceed
[translation:ERROR] return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/tool/taskengine.py",
line 108, in _execute
[translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/driver.py",
line 197, in _do
[translation:ERROR] res = func()
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/driver.py",
line 333, in task_compile_c
[translation:ERROR] cbuilder.compile()
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/c/genc.py",
line 229, in compile
[translation:ERROR] compiler.build()
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/tool/cbuild.py",
line 323, in build
[translation:ERROR] self._build()
[translation:ERROR] File
"/home/drayko/Workfolder/Security/Python/cvs/pypy-dist/pypy/translator/tool/cbuild.py",
line 353, in _build
[translation:ERROR] extra_preargs=self.compile_extra)
[translation:ERROR] File "/usr/lib/python2.4/distutils/ccompiler.py",
line 699, in compile
[translation:ERROR] self._compile(obj, src, ext, cc_args,
extra_postargs, pp_opts)
[translation:ERROR] File
"/usr/lib/python2.4/distutils/unixccompiler.py", line 115, in _compile
[translation:ERROR] raise CompileError, msg
[translation:ERROR] CompileError: command 'cc' failed with exit status 1
"""
If this really calls for a bugreport, it would be better if some one
more comfortable with the compiling process and the gcc toolset would do
it. Should I send some of the produced files?

Elmo


From elmo13 at jippii.fi  Sun Jul 23 22:15:47 2006
From: elmo13 at jippii.fi (=?windows-1252?Q?Elmo_M=E4ntynen?=)
Date: Sun, 23 Jul 2006 23:15:47 +0300
Subject: [pypy-dev] A bug in cc?
In-Reply-To: <44C3C339.8070809@jippii.fi>
References: <44BE2F9E.4060306@jippii.fi>	<20060721152939.GB31604@code0.codespeak.net>	<44C0FBCA.1090605@jippii.fi>	<44C1388D.80003@jippii.fi>
	<44C3C339.8070809@jippii.fi>
Message-ID: <44C3D8F3.4080603@jippii.fi>

Elmo M?ntynen wrote:
> Elmo M?ntynen wrote:
>   
>> Elmo M?ntynen wrote:
>>   
>>     
>>> Armin Rigo wrote:
>>>   
>>>     
>>>       
>>>> Hi Elmo,
>>>>
>>>> On Wed, Jul 19, 2006 at 04:11:58PM +0300, Elmo M??ntynen wrote:
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> Last lines of output (python translate.py --backend=c --profopt='-c
>>>>> "from richards import *;main(iterations=1)"' --text --batch
>>>>> targetpypystandalone.py):
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>> I have no clue how --profopt can affect the translation itself.  You
>>>> don't get the same error without the --profopt option?
>>>>
>>>>
>>>> A bientot,
>>>>
>>>> Armin
>>>>   
>>>>     
>>>>       
>>>>         
>>> Should've tested that, will do as soon as possible. benchmark.html on
>>> snake shows something that says to me that the profopt executable
>>> doesn't exist, which suggests that it wasn't built.
>>>
>>> Elmo
>>>   
>>>     
>>>       
>> Translation without profopt worked now, though there might have been
>> something that got changed in between, will report after some more builds.
>>   
>>     
> Now I got an internal compiler error from cc:
> """
> [cbuild:profopt] Gathering profile data from:
> /tmp/usession-11/testing_1/testing_1 -c "from richards import
> *;main(iterations=1)"
>
>   
...
> [translation:ERROR] File
> "/usr/lib/python2.4/distutils/unixccompiler.py", line 115, in _compile
> [translation:ERROR] raise CompileError, msg
> [translation:ERROR] CompileError: command 'cc' failed with exit status 1
> """
> If this really calls for a bugreport, it would be better if some one
> more comfortable with the compiling process and the gcc toolset would do
> it. Should I send some of the produced files?
>   
Just to note that building without profopt works (this time its the same
rev).

Elmo


From len-l at telus.net  Mon Jul 24 06:12:41 2006
From: len-l at telus.net (Lenard Lindstrom)
Date: Sun, 23 Jul 2006 21:12:41 -0700
Subject: [pypy-dev] Newb questions about ext module writing
In-Reply-To: <9eebf5740607231058w2a2b688bpb5c746c01750ad96@mail.gmail.com>
References: <44C355C6.11938.7869AE@len-l.telus.net>
Message-ID: <44C3E649.11747.B7036B@len-l.telus.net>

On 23 Jul 2006 at 19:58, Lawrence Oluyede wrote:

> > I just brought it up as it is one more thing to consider when
> > converting a ctypes Python module to rctypes. But could rctypes use
> > ffi when the callback is a bound method?
> 
> AFAIK one of the reasons of the existence of rctypes and the ext.
> compiler is not to use the ffi library...
> 
Then perhaps the ext. compiler can achieve the same effect as an ffi 
C closure by creating a stub function to call the bound method, which 
is stored in a thread-local variable.

Lenard Lindstrom
<len-l at telus.net>




From aurelien.campeas at logilab.fr  Mon Jul 24 12:21:52 2006
From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=)
Date: Mon, 24 Jul 2006 12:21:52 +0200
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060721203613.GA2825@code0.codespeak.net>
References: <20060721203613.GA2825@code0.codespeak.net>
Message-ID: <20060724102151.GC15904@crater.logilab.fr>

On Fri, Jul 21, 2006 at 10:36:13PM +0200, Armin Rigo wrote:
> Hi all, hi Aurelien,
> 
> The stackless-related tests are all consistently failing nowadays.  Some
> have been failing for 4 days and others for 9 days.  See
> 
>    http://snake.cs.uni-duesseldorf.de/pypytest/summary.html
> 
> Given that you appear to continue to work on it, but without making
> tests pass again, I'm a bit worrying exactly about what you are
> doing?

First, I did some (superficial, nothing related to the translation
aspect itself) changes that allowed me to build a (somewhat
non-crashing) translated pypy-logic using clonable coroutines. These
changes indeed broke some tests, in ways that I don't
understand. Please note that indeed most failing tests in
/modules/_stackless/test are due to run-time failures, after a
successful translation. Michael put in a fix last week which,
strangely, yielded a working version on ... PPC arch only.

IOW, wizards needed for these cases.

I'll have a look at the applevel stuff in /lib/test2 (the 3 tests pass
all independantly but fail when executed in sequence), since I'm more
likely directly responsible for breakage there.

> That doesn't seem too much in line with the "test-driven development"
> approach.
> 

Yes. Otoh, it seems that I'm currently the only consummer of these
features, so I didn't feel the urge to rush to fix them immediately. I
will look at them today.

Regards,
Aur?lien.


From fijal at genesilico.pl  Mon Jul 24 12:35:05 2006
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Mon, 24 Jul 2006 12:35:05 +0200
Subject: [pypy-dev] JS backend updates
Message-ID: <44C4A259.3010309@genesilico.pl>

There were some updates recently about JS backend.

Recently:

- Added some decorator which might be usefull at declaring server-side 
of AJAX communication
- Polished a little bit backend so now it is able to run richards and 
pystone as soon as you cut off zeroing assumption out of ll_alloc_and_set
- Added some documentation
- Bugfixes and some helper functions (ie StringBuilder) so now 
"aaa".upper() works as expected

Coming work:

- Hiding browser differencies
- more examples & docs
- Maybe some stackless stuff?
- Very hard and usefull point: to get out Python-level traceback out of 
exception handlers.
- Some JS optimisations (assuming that JS compilers themselves cannot 
optimise much if anything at all)
- BnB more playable.



From cfbolz at gmx.de  Mon Jul 24 13:51:02 2006
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 24 Jul 2006 13:51:02 +0200
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060724102151.GC15904@crater.logilab.fr>
References: <20060721203613.GA2825@code0.codespeak.net>
	<20060724102151.GC15904@crater.logilab.fr>
Message-ID: <44C4B426.60206@gmx.de>

Hi!

Aur?lien Camp?as wrote:
> On Fri, Jul 21, 2006 at 10:36:13PM +0200, Armin Rigo wrote:
>> Hi all, hi Aurelien,
>>
>> The stackless-related tests are all consistently failing nowadays.  Some
>> have been failing for 4 days and others for 9 days.  See
>>
>>    http://snake.cs.uni-duesseldorf.de/pypytest/summary.html
>>
>> Given that you appear to continue to work on it, but without making
>> tests pass again, I'm a bit worrying exactly about what you are
>> doing?
>
> First, I did some (superficial, nothing related to the translation
> aspect itself) changes that allowed me to build a (somewhat
> non-crashing) translated pypy-logic using clonable coroutines. These
> changes indeed broke some tests, in ways that I don't
> understand. Please note that indeed most failing tests in
> /modules/_stackless/test are due to run-time failures, after a
> successful translation. Michael put in a fix last week which,
> strangely, yielded a working version on ... PPC arch only.
>
> IOW, wizards needed for these cases.
>
> I'll have a look at the applevel stuff in /lib/test2 (the 3 tests pass
> all independantly but fail when executed in sequence), since I'm more
> likely directly responsible for breakage there.

It's just not good to have failing tests checked in. If they fail for
good reasons and will probably work soon again, just skip them with an
appropriate message. Otherwise it becomes impossible for somebody
working in a related area to find out whether his changes were
responsible for the breakage.

>> That doesn't seem too much in line with the "test-driven development"
>> approach.
>>
>
> Yes. Otoh, it seems that I'm currently the only consummer of these
> features, so I didn't feel the urge to rush to fix them immediately. I
> will look at them today.

Well, that's not true. People wanted to try out stackless and couldn't
because it didn't translate. I needed to do benchmarks and couldn't
either. It's just a good general policy to have tranlations working all
the time.

Cheers,

Carl Friedrich


From Ben.Young at risk.sungard.com  Mon Jul 24 14:48:57 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Mon, 24 Jul 2006 13:48:57 +0100
Subject: [pypy-dev] Example of .NET filehandles
Message-ID: <OF4BE4E08A.626924BA-ON802571B5.00462B19-802571B5.004666AF@risk.sungard.com>

Hi Antonio,

Here's the example I mentioned. I don't think it's particularly safe, so 
it may be better to keep a map of ints to filestreams that you have 
opened, and use that if possible, and fall back to this if you can't.

Cheers,
Ben

using System;
using System.Collections.Generic;
using System.Text;

namespace posix
{
    class Program
    {
        static void Main(string[] args)
        {
            System.IO.FileStream stream1 = new System.IO.FileStream
(args[0], System.IO.FileMode.Create);
            int handle = 
stream1.SafeFileHandle.DangerousGetHandle().ToInt32();

            System.IO.FileStream stream2 = new System.IO.FileStream(new 
Microsoft.Win32.SafeHandles.SafeFileHandle(new IntPtr(handle), true), 
System.IO.FileAccess.Write);
            stream2.Write(new byte[]{(byte)'H', (byte)'e', (byte)'l', (
byte)'l', (byte)'o'}, 0, 5);
            stream2.Close();
        }
    }
}


From arigo at tunes.org  Mon Jul 24 15:33:59 2006
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 24 Jul 2006 15:33:59 +0200
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060724102151.GC15904@crater.logilab.fr>
References: <20060721203613.GA2825@code0.codespeak.net>
	<20060724102151.GC15904@crater.logilab.fr>
Message-ID: <20060724133359.GB12961@code0.codespeak.net>

Hi Aurelien,

On Mon, Jul 24, 2006 at 12:21:52PM +0200, Aur?lien Camp?as wrote:
> First, I did some (superficial, nothing related to the translation
> aspect itself) changes that allowed me to build a (somewhat
> non-crashing) translated pypy-logic using clonable coroutines.

I did not look closely, so I cannot comment, but it seems to me that the
tests and the demo we fixed during the post-EuroPython sprint all
translated correctly, so I'm a bit surprized that you had to hack so
much on the existing stackless code only for translatability reasons.

This said, there were not many tests so far.  That's also why I'm not
happy to see half of them fail nowadays.  The "agreed" development
procedure for PyPy is to write more tests when a bug is identified or
existing code doesn't translate in some context, and then make the test
pass without breaking the previous tests (which are all valid, so again
I'm surprized that you can compile anything at all when half of the
basic tests fail).


A bientot,

Armin


From aurelien.campeas at logilab.fr  Mon Jul 24 15:56:27 2006
From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=)
Date: Mon, 24 Jul 2006 15:56:27 +0200
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060724133359.GB12961@code0.codespeak.net>
References: <20060721203613.GA2825@code0.codespeak.net>
	<20060724102151.GC15904@crater.logilab.fr>
	<20060724133359.GB12961@code0.codespeak.net>
Message-ID: <20060724135627.GB23406@crater.logilab.fr>

On Mon, Jul 24, 2006 at 03:33:59PM +0200, Armin Rigo wrote:
> Hi Aurelien,
> 
> On Mon, Jul 24, 2006 at 12:21:52PM +0200, Aur?lien Camp?as wrote:
> > First, I did some (superficial, nothing related to the translation
> > aspect itself) changes that allowed me to build a (somewhat
> > non-crashing) translated pypy-logic using clonable coroutines.
> 
> I did not look closely, so I cannot comment, but it seems to me that the
> tests and the demo we fixed during the post-EuroPython sprint all
> translated correctly, so I'm a bit surprized that you had to hack so
> much on the existing stackless code only for translatability
> reasons.

svn blame currently shows how incredibly little I had to play with the
code (to break it ... and have it fit my needs, and later make it
translatable again).

Apart from some temporary uncautious changes in interp_clonable from
which one can trace back the first breakage, and that were fixed
since, there is something else going on, that happened some time
after. The current breakage is, I think, unrelated to my
misdoings. This fact is just masked by the insufficient granularity of
http://snake.cs.uni-duesseldorf.de/pypytest/summary.html.

I am tracking every change concerning stackless, now.

Then, again, please consider that the current module/_stackless tests
fail at RUNTIME. They translate perfectly well.

> 
> This said, there were not many tests so far.  That's also why I'm not
> happy to see half of them fail nowadays.  The "agreed" development
> procedure for PyPy is to write more tests when a bug is identified or
> existing code doesn't translate in some context, and then make the test
> pass without breaking the previous tests (which are all valid, so again
> I'm surprized that you can compile anything at all when half of the
> basic tests fail).

I am surprised, too. But I can exhibit the binary :)

Regards,
Aur?lien.


From stephan.diehl at gmx.net  Mon Jul 24 20:00:56 2006
From: stephan.diehl at gmx.net (Stephan Diehl)
Date: Mon, 24 Jul 2006 20:00:56 +0200
Subject: [pypy-dev] JS backend updates
In-Reply-To: <44C4A259.3010309@genesilico.pl>
References: <44C4A259.3010309@genesilico.pl>
Message-ID: <44C50AD8.9080004@gmx.net>

Hi Maciek,

Maciek Fijalkowski wrote:
> There were some updates recently about JS backend.
> 
> Recently:
> 
> - Added some decorator which might be usefull at declaring server-side 
> of AJAX communication
> - Polished a little bit backend so now it is able to run richards and 
> pystone as soon as you cut off zeroing assumption out of ll_alloc_and_set
> - Added some documentation
> - Bugfixes and some helper functions (ie StringBuilder) so now 
> "aaa".upper() works as expected

thanks for the added documentation.
In the meantime, I've been busy with my own JS test application (not 
using pypy yet :-(   ). If you have some time on your hand, you could 
check out http://codespeak.net/svn/user/stephan/hacks/webapp
which, if nothing else, contains a working JsonRpc implementation.
Just run 'python examplewebapp.py' and point your browser to localhost:8000.
The only dependency is on 'simplejson' (MochiKit is included)

Anyway, I'd like to see some widget stuff working, so one could maybe 
integrate / expand the pyjamas stuff or write our own.

Cheers

Stephan

> 
> Coming work:
> 
> - Hiding browser differencies
> - more examples & docs
> - Maybe some stackless stuff?
> - Very hard and usefull point: to get out Python-level traceback out of 
> exception handlers.
> - Some JS optimisations (assuming that JS compilers themselves cannot 
> optimise much if anything at all)
> - BnB more playable.
> 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 
> 



From arigo at tunes.org  Mon Jul 24 20:32:35 2006
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 24 Jul 2006 20:32:35 +0200
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060724135627.GB23406@crater.logilab.fr>
References: <20060721203613.GA2825@code0.codespeak.net>
	<20060724102151.GC15904@crater.logilab.fr>
	<20060724133359.GB12961@code0.codespeak.net>
	<20060724135627.GB23406@crater.logilab.fr>
Message-ID: <20060724183235.GA11228@code0.codespeak.net>

Hi Aurelien,

On Mon, Jul 24, 2006 at 03:56:27PM +0200, Aur?lien Camp?as wrote:
> svn blame currently shows how incredibly little I had to play with the
> code

Well, that's hardly an argument, is it?  I know that the code is
obscure, and you're welcome to ask questions here and on IRC.  I also
know from experience that making coroutines translatable can trigger
obscure problems.  It doesn't change the basic issue that breaking tests
without worrying about them is not a good way to go forward...

> The current breakage is, I think, unrelated to my
> misdoings. This fact is just masked by the insufficient granularity of
> http://snake.cs.uni-duesseldorf.de/pypytest/summary.html.

Well, at least one of these tests - test_coroutine_reconstruction - has
been breaking in the same way since your rev 30119.  For the other
tests, knowing exactly what is the cause of the current breakage is
difficult, and that's precisely because the tests were already failing.
This prevents us from easily tracking what breaks what.  For now "naive"
tracking tells me that you broke the tests first...

> Then, again, please consider that the current module/_stackless tests
> fail at RUNTIME. They translate perfectly well.

I don't see how that is an argument, either.

As far as I can tell, the only way out of this kind of mess is to revert
to the first revision before the first failure, and try to re-understand
the diff of that initial breaking revision, debug until we see what the
problem was, and then attempt to fix it in the head.  That's certainly
more work than needed because of the revision-juggling involved.

I'd appreciate it if you could tell us if you're going to do that, or if
there is something out of your scope going on.  I'd like this situation
to be resolved.


A bientot,

Armin


From arigo at tunes.org  Mon Jul 24 22:56:10 2006
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 24 Jul 2006 22:56:10 +0200
Subject: [pypy-dev] A bug in cc? (was: "translation with --profopt
	fails")
In-Reply-To: <44C3C339.8070809@jippii.fi>
References: <44BE2F9E.4060306@jippii.fi>
	<20060721152939.GB31604@code0.codespeak.net>
	<44C0FBCA.1090605@jippii.fi> <44C1388D.80003@jippii.fi>
	<44C3C339.8070809@jippii.fi>
Message-ID: <20060724205610.GA25017@code0.codespeak.net>

Hi Elmo,

On Sun, Jul 23, 2006 at 09:43:05PM +0300, Elmo M?ntynen wrote:
> Now I got an internal compiler error from cc:

Couldn't reproduce it...  I ran

  python translate.py --backend=c --profopt='-c "from richards import
  *;main(iterations=1)"' --text --batch targetpypystandalone.py

but everything went fine.  It's a gcc 3.4.5 (Gentoo 3.4.5-r1,
ssp-3.4.5-1.0, pie-8.7.9).  Either it's a bug in a specific gcc version,
or else maybe (just guessing) the floating-point exception is when gcc
computes something with the profile data, maybe timings, in which case
the bug could be hard to reproduce on a different machine.

In any case, if the bug is still there in the latest gcc, yes, I'd
consider reporting it.  PyPy is good to push many limits of its backends
-- e.g. it gave quite a few LLVM bug reports and I wouldn't be surpized
if mono was next :-)


A bientot,

Armin


From scott+pypy-dev at scottdial.com  Mon Jul 24 23:33:09 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Mon, 24 Jul 2006 17:33:09 -0400
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for win32
Message-ID: <44C53C95.4070002@scottdial.com>

I attempted to translate a pypy-c in windows today. I had to mask out 
including sys/mman.h. Afterwards, I was able to succesfully translate.

Index: ll_osdefs.h
===================================================================
--- ll_osdefs.h	(revision 30390)
+++ ll_osdefs.h	(working copy)
@@ -42,7 +42,9 @@

  #include <sys/types.h>

+#if !defined(MS_WINDOWS)
  #include <sys/mman.h>
+#endif

  #ifdef HAVE_STROPTS_H
  #include <stropts.h>

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu


From anto.cuni at gmail.com  Mon Jul 24 23:41:47 2006
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 24 Jul 2006 23:41:47 +0200
Subject: [pypy-dev] A bug in cc?
In-Reply-To: <20060724205610.GA25017@code0.codespeak.net>
References: <44BE2F9E.4060306@jippii.fi>	<20060721152939.GB31604@code0.codespeak.net>	<44C0FBCA.1090605@jippii.fi>
	<44C1388D.80003@jippii.fi>	<44C3C339.8070809@jippii.fi>
	<20060724205610.GA25017@code0.codespeak.net>
Message-ID: <44C53E9B.3000605@gmail.com>

Armin Rigo wrote:
> In any case, if the bug is still there in the latest gcc, yes, I'd
> consider reporting it.  PyPy is good to push many limits of its backends
> -- e.g. it gave quite a few LLVM bug reports and I wouldn't be surpized
> if mono was next :-)

mono *will be* the next, definitively :-).

ciao Anto


From elmo13 at jippii.fi  Tue Jul 25 00:01:23 2006
From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=)
Date: Tue, 25 Jul 2006 01:01:23 +0300
Subject: [pypy-dev] A bug in cc?
In-Reply-To: <20060724205610.GA25017@code0.codespeak.net>
References: <44BE2F9E.4060306@jippii.fi>
	<20060721152939.GB31604@code0.codespeak.net>
	<44C0FBCA.1090605@jippii.fi> <44C1388D.80003@jippii.fi>
	<44C3C339.8070809@jippii.fi>
	<20060724205610.GA25017@code0.codespeak.net>
Message-ID: <44C54333.5080701@jippii.fi>

Armin Rigo wrote:
> Hi Elmo,
>
> On Sun, Jul 23, 2006 at 09:43:05PM +0300, Elmo M?ntynen wrote:
>   
>> Now I got an internal compiler error from cc:
>>     
>
> Couldn't reproduce it...  I ran
>
>   python translate.py --backend=c --profopt='-c "from richards import
>   *;main(iterations=1)"' --text --batch targetpypystandalone.py
>
> but everything went fine.  It's a gcc 3.4.5 (Gentoo 3.4.5-r1,
> ssp-3.4.5-1.0, pie-8.7.9).  Either it's a bug in a specific gcc version,
> or else maybe (just guessing) the floating-point exception is when gcc
> computes something with the profile data, maybe timings, in which case
> the bug could be hard to reproduce on a different machine.
>
> In any case, if the bug is still there in the latest gcc, yes, I'd
> consider reporting it.  PyPy is good to push many limits of its backends
> -- e.g. it gave quite a few LLVM bug reports and I wouldn't be surpized
> if mono was next :-)
>
>
> A bientot,
>
> Armin
>   
I have 4.0.4 (debian 4.0.3-4). How should we go about testing on the
latest gcc, or are you doing it right now. I can definitely reproduce
the bug with the that certain file(s). Which of the .gcda and .gcno (or
the rest) files should I send?

Elmo


From l.oluyede at gmail.com  Tue Jul 25 06:45:17 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Tue, 25 Jul 2006 06:45:17 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <44C53C95.4070002@scottdial.com>
References: <44C53C95.4070002@scottdial.com>
Message-ID: <9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>

On 7/24/06, Scott Dial <scott+pypy-dev at scottdial.com> wrote:
> I attempted to translate a pypy-c in windows today. I had to mask out
> including sys/mman.h. Afterwards, I was able to succesfully translate.
>
> Index: ll_osdefs.h
> ===================================================================
> --- ll_osdefs.h (revision 30390)
> +++ ll_osdefs.h (working copy)
> @@ -42,7 +42,9 @@
>
>   #include <sys/types.h>
>
> +#if !defined(MS_WINDOWS)
>   #include <sys/mman.h>
> +#endif
>
>   #ifdef HAVE_STROPTS_H
>   #include <stropts.h>

That was my fault. It has been fixed. Thanks

-- 
Lawrence
http://www.oluyede.org/blog


From aurelien.campeas at logilab.fr  Tue Jul 25 11:09:32 2006
From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=)
Date: Tue, 25 Jul 2006 11:09:32 +0200
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060724183235.GA11228@code0.codespeak.net>
References: <20060721203613.GA2825@code0.codespeak.net>
	<20060724102151.GC15904@crater.logilab.fr>
	<20060724133359.GB12961@code0.codespeak.net>
	<20060724135627.GB23406@crater.logilab.fr>
	<20060724183235.GA11228@code0.codespeak.net>
Message-ID: <20060725090932.GA2908@crater.logilab.fr>

On Mon, Jul 24, 2006 at 08:32:35PM +0200, Armin Rigo wrote:
> Well, that's hardly an argument, is it?  I know that the code is
> obscure, and you're welcome to ask questions here and on IRC.  I also

Sometimes people are on vacation. I apologize for the nuisance.

Given the time to learn enough of subversion, I'll try to replay
versions up to what caused these failures.


From arigo at tunes.org  Tue Jul 25 11:35:26 2006
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 25 Jul 2006 11:35:26 +0200
Subject: [pypy-dev] A bug in cc?
In-Reply-To: <44C54333.5080701@jippii.fi>
References: <44BE2F9E.4060306@jippii.fi>
	<20060721152939.GB31604@code0.codespeak.net>
	<44C0FBCA.1090605@jippii.fi> <44C1388D.80003@jippii.fi>
	<44C3C339.8070809@jippii.fi>
	<20060724205610.GA25017@code0.codespeak.net>
	<44C54333.5080701@jippii.fi>
Message-ID: <20060725093526.GA25834@code0.codespeak.net>

Hi Elmo,

On Tue, Jul 25, 2006 at 01:01:23AM +0300, Elmo M?ntynen wrote:
> I have 4.0.4 (debian 4.0.3-4). How should we go about testing on the
> latest gcc, or are you doing it right now. I can definitely reproduce
> the bug with the that certain file(s). Which of the .gcda and .gcno (or
> the rest) files should I send?

I don't know what these files are, so I can't comment, but you should
look or ask around in the gcc community instead.  Also, I'm sure they
would like to see information related to their latest release, so you
should try with that.  And of course you should try to see if it's a
known bug, maybe even if it's been fixed already...


A bientot,

Armin


From arigo at tunes.org  Tue Jul 25 11:38:45 2006
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 25 Jul 2006 11:38:45 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
Message-ID: <20060725093843.GA26167@code0.codespeak.net>

Hi all,

On Tue, Jul 25, 2006 at 06:45:17AM +0200, Lawrence Oluyede wrote:
> > I attempted to translate a pypy-c in windows today. I had to mask out
> > including sys/mman.h. Afterwards, I was able to succesfully translate.
> 
> That was my fault. It has been fixed. Thanks

Now that we have automated test runs and translations on Linux machines,
we could try to think how we could go about doing the same on Windows
machines.  Very few of us actually have Windows machines around, so it
would be important, otherwise things will break more and more (it's
already a good surprize that a PyPy translation still works after just a
quick fix :-)

"""What is not tested is broken."""


A bientot,

Armin


From stephan.diehl at gmx.net  Tue Jul 25 11:55:36 2006
From: stephan.diehl at gmx.net (Stephan Diehl)
Date: Tue, 25 Jul 2006 11:55:36 +0200
Subject: [pypy-dev] pypy development tasks
Message-ID: <44C5EA98.5070700@gmx.net>

Hi all,

starting next week, I'll be able again to continue working on pypy.
As a part-part-part time worker, I find it a bit difficult to stay on 
top on what's going on (and what needs to be done).

These are the things, I would continue on, if nobody has a better idea:

- write a python module that exposes Coroutine interface (using
   greenlets, I suppose). This would be usefull for further development
   of applevel stackless module. As a sideeffect, the stackless module
   could then be used with CPython.
- continue work on applevel stackless module and documentation.

Other than that, if I should rather do things like adding docstrings or 
do other usefull tasks, don't hesitate to tell me :-)

Cheers

Stephan


From scott+pypy-dev at scottdial.com  Tue Jul 25 13:44:45 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Tue, 25 Jul 2006 07:44:45 -0400
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h
	for	win32
In-Reply-To: <20060725093843.GA26167@code0.codespeak.net>
References: <44C53C95.4070002@scottdial.com>	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
Message-ID: <44C6042D.7090806@scottdial.com>

Armin Rigo wrote:
> Now that we have automated test runs and translations on Linux machines,
> we could try to think how we could go about doing the same on Windows
> machines.  Very few of us actually have Windows machines around, so it
> would be important, otherwise things will break more and more (it's
> already a good surprize that a PyPy translation still works after just a
> quick fix :-)
> 

I actually was pretty suprised too.. Presuming this is an important 
platform to test (*grin*), I can provide a daily testing run for 
Windows. I don't know what the code behind your pypytest is, but I 
presume I could run something similar? Let me know.

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu


From l.oluyede at gmail.com  Tue Jul 25 20:51:09 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Tue, 25 Jul 2006 20:51:09 +0200
Subject: [pypy-dev] extension compiler and Python special methods
Message-ID: <9eebf5740607251151y29f5341l43bc92fe28ff592e@mail.gmail.com>

As u can see in rev. #30543 I commited a series of methods to complete
the mmap module. Those are the special methods (__xyz__) implemented
as regular ones. I thought it was best to have the functionality in
anyway. When and if we'll have that support it will be only a matter
of change methods names to gain compatibility with CPython's code.

If you disagree with me let me know, I'll comment out the code or whatever.

-- 
Lawrence
http://www.oluyede.org/blog


From arigo at tunes.org  Wed Jul 26 11:18:25 2006
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 26 Jul 2006 11:18:25 +0200
Subject: [pypy-dev] about bz2 and more
In-Reply-To: <9eebf5740607260150x3be8d4ta9ee5497886d5c87@mail.gmail.com>
References: <9eebf5740607260150x3be8d4ta9ee5497886d5c87@mail.gmail.com>
Message-ID: <20060726091824.GA26783@code0.codespeak.net>

Hi Lawrence,

On Wed, Jul 26, 2006 at 10:50:40AM +0200, Lawrence Oluyede wrote:
> Bz2 is kinda a beast module doing heavy operations with function
> pointers, buffering and so on. The problem I encountered lies in the
> initialization of the bz2 file object. It has support for buffering,
> universal newlines and more. All this stuff is exposed in CPython via
> file()/open() factory functions. Those are not supported at RPython
> level and I can't write the class at app-level.

I think the first thing do to is support a "raw" interface, that gives
access to unbuffered binary data.  Then for PyPy we can simply plug the
existing buffering and universal-newline code, which is already written
at app-level in pypy/lib/_sio.py.

It means that if you write a class with the correct interface, and
expose it to app-level in a mixed-module, then at least in PyPy you can
write the rest at app-level: in bz2.open(), instantiate the "raw" class
from the interp-level part of the module, then import the _sio module
and build the correct stack of buffering and translating classes on top
of your "raw" class.

See _file.py, _inithelper(), for how to build such a stack from standard
mode/buffering arguments.  Maybe abstracting out some of this code in a
helper function in _sio or _file would be useful.

An example of "raw" class is _sio.DiskFile, which uses os.read()
internally.  For bz2 you'd build a class with the same interface, based
on ctypes calls instead.

IMHO this all shows that writing buffering and universal newlines
explicitly and at app-level is a great idea :-) Essentially, with almost
no more effort we could provide users with a PyPy-specific module that
plugs buffering and translating on top of any "raw" reader/writer that
she provides.


A bientot,

Armin


From Ben.Young at risk.sungard.com  Wed Jul 26 13:34:29 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Wed, 26 Jul 2006 12:34:29 +0100
Subject: [pypy-dev] [pypy-svn] r30565 - in pypy/dist/pypy/tool/build: .
	bin builds test
In-Reply-To: <20060726111851.DD6C6100B8@code0.codespeak.net>
Message-ID: <OFA83F9FBD.75F9FF4B-ON802571B7.003F7C6B-802571B7.003F9419@risk.sungard.com>

Hi Guido,

Cool tool! Is this something I would be able to run just in the 
evenings/weekends etc? Will it pick up that fact that I don't have boehm 
automatically?

Cheers,
Ben


pypy-svn-bounces at codespeak.net wrote on 26/07/2006 12:18:51:

> Author: guido
> Date: Wed Jul 26 13:18:25 2006
> New Revision: 30565
> 
> Added:
>    pypy/dist/pypy/tool/build/   (props changed)
>    pypy/dist/pypy/tool/build/README.txt   (contents, props changed)
>    pypy/dist/pypy/tool/build/__init__.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/bin/   (props changed)
>    pypy/dist/pypy/tool/build/bin/client   (contents, props changed)
>    pypy/dist/pypy/tool/build/bin/path.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/bin/server   (contents, props changed)
>    pypy/dist/pypy/tool/build/bin/startcompile   (contents, props 
changed)
>    pypy/dist/pypy/tool/build/builds/
>    pypy/dist/pypy/tool/build/client.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/config.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/conftest.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/execnetconference.py   (contents, props 
changed)
>    pypy/dist/pypy/tool/build/server.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/test/   (props changed)
>    pypy/dist/pypy/tool/build/test/fake.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/test/path.py   (contents, props changed)
>    pypy/dist/pypy/tool/build/test/test.zip   (contents, props changed)
>    pypy/dist/pypy/tool/build/test/test_client.py   (contents, props 
changed)
>    pypy/dist/pypy/tool/build/test/test_pypybuilder.py   (contents, 
> props changed)
>    pypy/dist/pypy/tool/build/test/test_request_storage.py 
> (contents, props changed)
>    pypy/dist/pypy/tool/build/test/test_server.py   (contents, props 
changed)
> Log:
> Added 'pypybuilder', a tool to build a 'build farm' from 
> participating clients,
> the clients register to a server with information about what they can 
build,
> then the server waits for build requests and dispatches to clients. 
Clients
> send back a build when they're done, on which the server sends out 
emails to
> whoever is waiting for the build. When a build is already available, the
> requestor is provided with a path (will be URL in the future) to the 
build,
> if no client is available for a certain request the request is queued 
until
> there is one.
> Worked on this from https://merlinux.de/svn/user/guido/pypybuilder 
before
> checking in here.
> 
> 
> Added: pypy/dist/pypy/tool/build/README.txt
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/README.txt   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,59 @@
> +============
> +PyPyBuilder
> +============
> +
> +What is this?
> +=============
> +
> +PyPyBuilder is an application that allows people to build PyPy 
instances on
> +demand. If you have a nice idle machine connected to the Internet, and 
don't
> +mind us 'borrowing' it every once in a while, you can start up the 
client
> +script (in bin/client) and have the server send compile jobs to your 
machine.
> +If someone requests a build of PyPy that is not already available on 
the PyPy
> +website, and your machine is capable of making such a build, the 
> server may ask
> +your machine to create it. If enough people participate, with diverse 
enough
> +machines, an ad-hoc 'build farm' is created this way.
> +
> +Components
> +==========
> +
> +The application consists of 3 main components: a server component, 
> a client and
> +a small component to start compile jobs, which we'll call 
'startcompile' for
> +now. 
> +
> +The server waits for clients to register, and for compile job requests. 
When
> +clients register, they pass the server information about what 
> compilations they
> +can handle (system info). Then when the 'startcompile' component 
requests a
> +compilation job, the server first checks whether a binary is 
> already available,
> +and if so returns that. 
> +
> +If there isn't one, the server walks through a list of connected 
> clients to see
> +if there is one that can handle the job, and if so tells it to perform 
it. If
> +there's no client to handle the job, it gets queued until there is.
> +
> +Once a build is available, the server will send an email to all 
> email addresses
> +(it could be more than one person asked for the same build at the same 
time!)
> +passed to it by 'startcompile'.
> +
> +Installation
> +============
> +
> +Client
> +------
> +
> +Installing the system should not be required: just run '.
> /bin/client' to start
> +the client. Note that it depends on the `py lib`_.
> +
> +Server
> +------
> +
> +Also for the server there's no real setup required, and again there's a 

> +dependency on the `py lib`_.
> +
> +.. _`py lib`: http://codespeak.net/py
> +
> +More info
> +=========
> +
> +For more information, bug reports, patches, etc., please send an email 
to 
> +guido at merlinux.de.
> 
> Added: pypy/dist/pypy/tool/build/__init__.py
> 
==============================================================================
> 
> Added: pypy/dist/pypy/tool/build/bin/client
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/bin/client   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,58 @@
> +#!/usr/bin/python
> +
> +BUFSIZE = 1024
> +
> +import path
> +import sys
> +import random
> +from pypy.tool.build import config
> +
> +# XXX using random values for testing
> +modules = ['_stackless', '_socket']
> +
> +"""
> +random.shuffle(modules)
> +sysinfo = {
> +    'maxint': random.choice((sys.maxint, (2 ** 63 - 1))),
> +    'use_modules': modules[:random.randrange(len(modules) + 1)],
> +    'byteorder': random.choice(('little', 'big')),
> +}
> +"""
> +
> +sysinfo = {
> +    'maxint': sys.maxint,
> +    'use_modules': ['_stackless', '_socket'],
> +    'byteorder': sys.byteorder,
> +}
> +
> +if __name__ == '__main__':
> +    from py.execnet import SshGateway
> +    from pypy.tool.build.client import init
> +    gw = SshGateway(config.server)
> +    channel = init(gw, sysinfo, path=config.path, port=config.port)
> +    print channel.receive() # welcome message
> +    try:
> +        while 1:
> +            data = channel.receive()
> +            if not isinstance(data, dict): # needs more checks here
> +                raise ValueError(
> +                    'received wrong unexpected data of type %s' % 
> (type(data),)
> +                )
> +            info = data
> +            # XXX we should compile here, using data dict for info
> +            print 'compilation requested for info %r, now faking 
> that' % (info,)
> +            import time; time.sleep(10)
> +
> +            # write the zip to the server in chunks to server
> +            # XXX we're still faking this
> +            zipfp = (path.packagedir / 'test/test.zip').open()
> +            while True:
> +                chunk = zipfp.read(BUFSIZE)
> +                if not chunk:
> +                    break
> +                channel.send(chunk)
> +            channel.send(None) # tell the server we're done
> +            print 'done with compilation, waiting for next'
> +    finally:
> +        channel.close()
> +        gw.exit()
> 
> Added: pypy/dist/pypy/tool/build/bin/path.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/bin/path.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,5 @@
> +import py
> +
> +packagedir = py.magic.autopath().dirpath().dirpath()
> +rootpath = packagedir.dirpath().dirpath().dirpath()
> +py.std.sys.path.append(str(rootpath))
> 
> Added: pypy/dist/pypy/tool/build/bin/server
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/bin/server   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,27 @@
> +#!/usr/bin/python
> +
> +import path
> +from pypy.tool.build import config
> +
> +from py.execnet import SshGateway
> +
> +if __name__ == '__main__':
> +    from py.execnet import SshGateway
> +    from pypy.tool.build.server import init
> +
> +    gw = SshGateway(config.server)
> +    channel = init(gw, port=config.port, path=config.path, 
> +                    projectname=config.projectname,
> +                    buildpath=str(config.buildpath),
> +                    mailhost=config.mailhost,
> +                    mailport=config.mailport,
> +                    mailfrom=config.mailfrom)
> +
> +    try:
> +        while 1:
> +            data = channel.receive()
> +            assert isinstance(data, str)
> +            print data
> +    finally:
> +        channel.close()
> +        gw.exit()
> 
> Added: pypy/dist/pypy/tool/build/bin/startcompile
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/bin/startcompile   Wed Jul 26 13:18:25 
2006
> @@ -0,0 +1,57 @@
> +#!/usr/bin/python
> +
> +import path
> +import sys
> +import random
> +from pypy.tool.build import config
> +
> +initcode = """
> +    import sys
> +    sys.path += %r
> + 
> +    from pypy.tool.build import ppbserver
> +    channel.send(ppbserver.compile(%r, %r))
> +    channel.close()
> +"""
> +def init(gw, sysinfo, email, port=12321):
> +    from pypy.tool.build import execnetconference
> +
> +    conference = execnetconference.conference(gw, port, False)
> +    channel = conference.remote_exec(initcode % (config.path, 
> email, sysinfo))
> +    return channel
> +
> +if __name__ == '__main__':
> +    from py.execnet import SshGateway
> +
> +    from optparse import OptionParser
> +    optparser = OptionParser('%prog [options] email')
> +    for args, kwargs in config.options:
> +        optparser.add_option(*args, **kwargs)
> +    optparser.add_option('-r', '--revision', dest='revision', 
> default='trunk',
> +                        help='SVN revision (defaults to "trunk")')
> + 
> +    (options, args) = optparser.parse_args()
> +
> +    if not args or len(args) != 1:
> +        optparser.error('please provide an email address')
> +
> +    sysinfo = dict([(attr, getattr(options, attr)) for attr in 
> dir(options) if
> +                        not attr.startswith('_') and 
> +                        not callable(getattr(options, attr))])
> + 
> +    print 'going to start compile job with info:'
> +    for k, v in sysinfo.items():
> +        print '%s: %r' % (k, v)
> +    print
> +
> +    gw = SshGateway(config.server)
> +    channel = init(gw, sysinfo, args[0], port=config.port)
> +    ispath, data = channel.receive()
> +    if ispath:
> +        print ('a suitable result is already available, you can find it 
'
> +                'at "%s"' % (data,))
> +    else:
> +        print data
> +        print 'you will be mailed once it\'s ready'
> +    channel.close()
> +    gw.exit()
> 
> Added: pypy/dist/pypy/tool/build/client.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/client.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,71 @@
> +import time
> +import thread
> +
> +class PPBClient(object):
> +    def __init__(self, channel, sysinfo, testing=False):
> +        self.channel = channel
> +        self.sysinfo = sysinfo
> +        self.busy_on = None
> +        self.testing = testing
> +
> +        from pypy.tool.build import ppbserver
> +        self.server = ppbserver
> +        self.server.register(self)
> + 
> +    def sit_and_wait(self):
> +        """connect to the host and wait for commands"""
> +        self.channel.waitclose()
> +        self.channel.close()
> +
> +    def compile(self, info):
> +        """send a compile job to the client side
> +
> +            this waits until the client is done, and assumes the client 
sends
> +            back the whole binary as a single string (XXX this 
> should change ;)
> +        """
> +        self.busy_on = info
> +        self.channel.send(info)
> +        thread.start_new_thread(self.wait_until_done, (info,))
> +
> +    def wait_until_done(self, info):
> +        buildpath = self.server.get_new_buildpath(info)
> + 
> +        fp = buildpath.zipfile.open('w')
> +        if not self.testing:
> +            try:
> +                while True:
> +                    try:
> +                        chunk = self.channel.receive()
> +                    except EOFError:
> +                        # stop compilation, client has disconnected
> +                        return 
> +                    if chunk is None:
> +                        break
> +                    fp.write(chunk)
> +            finally:
> +                fp.close()
> + 
> +        self.server.compilation_done(info, buildpath)
> +        self.busy_on = None
> +
> +initcode = """
> +    import sys
> +    sys.path += %r
> + 
> +    from pypy.tool.build.client import PPBClient
> +
> +    try:
> +        client = PPBClient(channel, %r, %r)
> +        client.sit_and_wait()
> +    finally:
> +        channel.close()
> +"""
> +def init(gw, sysinfo, path=None, port=12321, testing=False):
> +    from pypy.tool.build import execnetconference
> + 
> +    if path is None:
> +        path = []
> +
> +    conference = execnetconference.conference(gw, port, False)
> +    channel = conference.remote_exec(initcode % (path, sysinfo, 
testing))
> +    return channel
> 
> Added: pypy/dist/pypy/tool/build/config.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/config.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,45 @@
> +import py
> +
> +# general settings, used by both server and client
> +server = 'johnnydebris.net'
> +port = 12321
> +path = ['/home/johnny/temp/pypy-dist'] 
> +
> +# option definitions for the startcompile script
> +# for now we have them here, we should probably use pypy's config 
instead 
> +# though...
> +import sys
> +def _use_modules_callback(option, opt_str, value, parser):
> +    parser.values.use_modules = [m.strip() for m in value.split(',') 
> +                                    if m.strip()]
> +
> +def _maxint_callback(option, opt_str, value, parser):
> +    parser.values.maxint = 2 ** (int(value) - 1) - 1
> +
> +options = [
> +    (('-m', '--use-modules'), {'action': 'callback', 'type': 'string',
> +                                'callback': _use_modules_callback,
> +                                'dest': 'use_modules', 'default': [],
> +                                'help': 'select the modules you 
> want to use'}),
> +    (('-i', '--maxint'), {'action': 'callback', 'callback': 
_maxint_callback,
> +                                'default': sys.maxint, 'dest': 
'maxint',
> +                                'type': 'string',
> +                                'help': ('size of an int in bits 
(32/64, '
> +                                            'defaults to 
sys.maxint)')}),
> +    (('-b', '--byteorder'), {'action': 'store', 
> +                                'dest': 'byteorder', 'default': 
> sys.byteorder,
> +                                'nargs': 1,
> +                                'help': ('byte order (little/big, 
defaults '
> +                                            'to sys.byteorder)')}),
> +]
> +
> +# settings for the server
> +projectname = 'pypy'
> +buildpath = '/home/johnny/temp/pypy-dist/pypy/tool/build/builds'
> +mailhost = '127.0.0.1'
> +mailport = 25
> +mailfrom = 'johnny at johnnydebris.net'
> +
> +# settings for the tests
> +testpath = [str(py.magic.autopath().dirpath().dirpath())] 
> +
> 
> Added: pypy/dist/pypy/tool/build/conftest.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/conftest.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,20 @@
> +import py
> +from py.__.documentation.conftest import Directory as Dir, DoctestText, 
\
> +                                            ReSTChecker
> +mypath = py.magic.autopath().dirpath()
> +
> +Option = py.test.Config.Option 
> +option = py.test.Config.addoptions("pypybuilder test options", 
> +        Option('', '--functional',
> +               action="store_true", dest="functional", default=False,
> +               help="run pypybuilder functional tests"
> +        ),
> +) 
> +
> +py.test.pypybuilder_option = option
> +
> +class Directory(Dir):
> +    def run(self):
> +        if self.fspath == mypath:
> +            return ['README.txt', 'test']
> +        return super(Directory, self).run()
> 
> Added: pypy/dist/pypy/tool/build/execnetconference.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/execnetconference.py   Wed Jul 26 13:18:25 
2006
> @@ -0,0 +1,126 @@
> +"""
> +An extension to py.execnet to allow multiple programs to exchange 
information
> +via a common server.  The idea is that all programs first open a 
gateway to
> +the same server (e.g. an SshGateway), and then call the conference() 
function
> +with a local TCP port number.  The first program must pass 
is_server=True and
> +the next ones is_server=False: the first program's remote gateway is 
used as
> +shared server for the next ones.
> +
> +For all programs, the conference() call returns a new gateway that is
> +connected to the Python process of this shared server.  Information can
> +be exchanged by passing data around within this Python process.
> +"""
> +import py
> +from py.__.execnet.register import InstallableGateway
> +
> +
> +def conference(gateway, port, is_server='auto'):
> +    if is_server:    # True or 'auto'
> +        channel = gateway.remote_exec(r"""
> +            import thread
> +            from socket import *
> +            s = socket(AF_INET, SOCK_STREAM)
> +            port = channel.receive()
> +            try:
> +                s.bind(('', port))
> +                s.listen(5)
> +            except error:
> +                channel.send(0)
> +            else:
> +                channel.send(1)
> +
> +                def readall(s, n):
> +                    result = ''
> +                    while len(result) < n:
> +                        t = s.read(n-len(result))
> +                        if not t:
> +                            raise EOFError
> +                        result += t
> +                    return result
> +
> +                def handle_connexion(clientsock, address):
> +                    clientfile = clientsock.makefile('r+b',0)
> +                    source = clientfile.readline().rstrip()
> +                    clientfile.close()
> +                    g = {'clientsock' : clientsock, 'address' : 
address}
> +                    source = eval(source)
> +                    if source:
> +                        g = {'clientsock' : clientsock, 'address' : 
address}
> +                        co = compile(source+'\n', source, 'exec')
> +                        exec co in g
> +
> +                while True:
> +                    conn, addr = s.accept()
> +                    if addr[0] == '127.0.0.1':   # else connexion 
refused
> +                        thread.start_new_thread(handle_connexion, 
> (conn, addr))
> +                    del conn
> +        """)
> +        channel.send(port)
> +        ok = channel.receive()
> +        if ok:
> +            return gateway
> +        if is_server == 'auto':
> +            pass   # fall-through and try as a client
> +        else:
> +            raise IOError("cannot listen on port %d (already in 
> use?)" % port)
> +
> +    if 1:   # client
> +        channel = gateway.remote_exec(r"""
> +            import thread
> +            from socket import *
> +            s = socket(AF_INET, SOCK_STREAM)
> +            port = channel.receive()
> +            s.connect(('', port))
> +            channel.send(1)
> +            def receiver(s, channel):
> +                while True:
> +                    data = s.recv(4096)
> +                    #print >> open('LOG','a'), 'backward', repr(data)
> +                    channel.send(data)
> +                    if not data: break
> +            thread.start_new_thread(receiver, (s, channel))
> +            try:
> +                for data in channel:
> +                    #print >> open('LOG','a'), 'forward', repr(data)
> +                    s.sendall(data)
> +            finally:
> +                s.shutdown(1)
> +        """)
> +        channel.send(port)
> +        ok = channel.receive()
> +        assert ok
> +        return InstallableGateway(ConferenceChannelIO(channel))
> +
> +
> +class ConferenceChannelIO:
> +    server_stmt = """
> +io = SocketIO(clientsock)
> +"""
> +
> +    error = (EOFError,)
> +
> +    def __init__(self, channel):
> +        self.channel = channel
> +        self.buffer = ''
> +
> +    def read(self, numbytes):
> +        #print >> open('LOG', 'a'), 'read %d bytes' % numbytes
> +        while len(self.buffer) < numbytes:
> +            t = self.channel.receive()
> +            if not t:
> +                #print >> open('LOG', 'a'), 'EOFError'
> +                raise EOFError
> +            self.buffer += t
> +        buf, self.buffer = self.buffer[:numbytes], 
self.buffer[numbytes:]
> +        #print >> open('LOG', 'a'), '--->', repr(buf)
> +        return buf
> +
> +    def write(self, data):
> +        #print >> open('LOG', 'a'), 'write(%r)' % (data,)
> +        self.channel.send(data)
> +
> +    def close_read(self):
> +        pass
> +
> +    def close_write(self):
> +        self.channel.close()
> 
> Added: pypy/dist/pypy/tool/build/server.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/server.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,332 @@
> +import random
> +import time
> +import thread
> +import smtplib
> +import py
> +
> +def issubdict(d1, d2):
> +    """sees whether a dict is a 'subset' of another dict
> + 
> +        dictvalues can be immutable data types and list and dicts of 
> +        immutable data types and lists and ... (recursive)
> +    """
> +    for k, v in d1.iteritems():
> +        if not k in d2:
> +            return False
> +        d2v = d2[k]
> +        if isinstance(v, dict):
> +            if not issubdict(v, d2v):
> +                return False
> +        elif isinstance(v, list):
> +            if not set(v).issubset(set(d2v)):
> +                return False
> +        elif v != d2v:
> +            return False
> +    return True
> +
> +# XXX note that all this should be made thread-safe at some point 
(meaning it
> +# currently isn't)!!
> +
> +class RequestStorage(object):
> +    """simple registry that manages information"""
> +    def __init__(self, info_to_path=[]):
> +        self._id_to_info = {} # id -> info dict
> +        self._id_to_emails = {} # id -> requestor email address
> +        self._id_to_path = {} # id -> filepath
> +
> +        self._last_id = 0
> +        self._id_lock = thread.allocate_lock()
> +
> +        self._build_initial(info_to_path)
> +
> +    def request(self, email, info):
> +        """place a request
> +
> +            this either returns a path to the binary (if it's available 

> +            already) or an id for the info
> +        """
> +        self._normalize(info)
> +        infoid = self.get_info_id(info)
> +        path = self._id_to_path.get(infoid)
> +        if path is not None:
> +            return path
> +        self._id_to_emails.setdefault(infoid, []).append(email)
> + 
> +    def get_info_id(self, info):
> +        """retrieve or create an id for an info dict"""
> +        self._id_lock.acquire()
> +        try:
> +            self._normalize(info)
> +            for k, v in self._id_to_info.iteritems():
> +                if v == info:
> +                    return k
> +            self._last_id += 1
> +            id = self._last_id
> +            self._id_to_info[id] = info
> +            return id
> +        finally:
> +            self._id_lock.release()
> +
> +    def add_build(self, info, path):
> +        """store the data for a build and make it available
> +
> +            returns a list of email addresses for the people that 
should be
> +            warned
> +        """
> +        self._normalize(info)
> +        infoid = self.get_info_id(info)
> +        emails = self._id_to_emails.pop(infoid)
> +        self._id_to_path[infoid] = path
> +        return emails
> +
> +    def _build_initial(self, info_to_path):
> +        """fill the dicts with info about files that are already 
built"""
> +        for info, path in info_to_path:
> +            id = self.get_info_id(info)
> +            self._id_to_path[id] = path
> +
> +    def _normalize(self, info):
> +        for k, v in info.iteritems():
> +            if isinstance(v, list):
> +                v.sort()
> +
> +from py.__.path.local.local import LocalPath
> +class BuildPath(LocalPath):
> +    def _info(self):
> +        info = getattr(self, '_info_value', {})
> +        if info:
> +            return info
> +        infopath = self / 'info.txt'
> +        if not infopath.check():
> +            return {}
> +        for line in infopath.readlines():
> +            line = line.strip()
> +            if not line:
> +                continue
> +            chunks = line.split(':')
> +            key = chunks.pop(0)
> +            value = ':'.join(chunks)
> +            info[key] = eval(value)
> +        self._info_value = info
> +        return info
> +
> +    def _set_info(self, info):
> +        self._info_value = info
> +        infopath = self / 'info.txt'
> +        infopath.ensure()
> +        fp = infopath.open('w')
> +        try:
> +            for key, value in info.iteritems():
> +                fp.write('%s: %r\n' % (key, value))
> +        finally:
> +            fp.close()
> + 
> +    info = property(_info, _set_info)
> +
> +    def _zipfile(self):
> +        return py.path.local(self / 'data.zip')
> +
> +    def _set_zipfile(self, iterable):
> +        # XXX not in use right now...
> +        fp = self._zipfile().open('w')
> +        try:
> +            for chunk in iterable:
> +                fp.write(chunk)
> +        finally:
> +            fp.close()
> +
> +    zipfile = property(_zipfile, _set_zipfile)
> +
> +class PPBServer(object):
> +    retry_interval = 10
> + 
> +    def __init__(self, projname, channel, builddir, mailhost=None, 
> +                    mailport=None, mailfrom=None):
> +        self._projname = projname
> +        self._channel = channel
> +        self._builddir = builddir
> +        self._mailhost = mailhost
> +        self._mailport = mailport
> +        self._mailfrom = mailfrom
> + 
> +        self._buildpath = py.path.local(builddir)
> +        self._clients = []
> +        info_to_path = [(p.info, str(p)) for p in 
> +                        self._get_buildpaths(builddir)]
> +        self._requeststorage = RequestStorage(info_to_path)
> +        self._queued = []
> +
> +        self._queuelock = thread.allocate_lock()
> +        self._namelock = thread.allocate_lock()
> + 
> +    def register(self, client):
> +        self._clients.append(client)
> +        self._channel.send('registered %s with info %r' % (
> +                            client, client.sysinfo))
> +        client.channel.send('welcome')
> +
> +    def compile(self, requester_email, info):
> +        """start a compilation
> +
> +            returns a tuple (ispath, data)
> +
> +            if there's already a build available for info, this will 
return
> +            a tuple (True, path), if not, this will return (False, 
message),
> +            where message describes what is happening with the request 
(is
> +            a build made rightaway, or is there no client available?)
> +
> +            in any case, if the first item of the tuple returned is 
False,
> +            an email will be sent once the build is available
> +        """
> +        path = self._requeststorage.request(requester_email, info)
> +        if path is not None:
> +            self._channel.send('already a build for this info 
available')
> +            return (True, path)
> +        for client in self._clients:
> +            if client.busy_on == info:
> +                self._channel.send('build for %r currently in progress' 
% 
> +                                    (info,))
> +                return (False, 'this build is already in progress')
> +        # we don't have a build for this yet, find a client to compile 
it
> +        if self.run(info):
> +            return (False, 'found a suitable client, going to build')
> +        else:
> +            self._queuelock.acquire()
> +            try:
> +                self._queued.append(info)
> +            finally:
> +                self._queuelock.release()
> +            return (False, 'no suitable client found; your request 
> is queued')
> + 
> +    def run(self, info):
> +        """find a suitable client and run the job if possible"""
> +        # XXX shuffle should be replaced by something smarter obviously 
;)
> +        clients = self._clients[:]
> +        random.shuffle(clients)
> +        rev = info.pop('revision', 'trunk')
> +        for client in clients:
> +            # popping out revision here, going to add later... the 
client 
> +            # should be able to retrieve source code for any revision 
(so
> +            # it doesn't need to match a revision field in 
client.sysinfo)
> +            if client.busy_on or not issubdict(info, client.sysinfo):
> +                continue
> +            else:
> +                info['revision'] = rev
> +                self._channel.send(
> +                    'going to send compile job with info %r to %s' % (
> +                        info, client
> +                    )
> +                )
> +                client.compile(info)
> +                return True
> +        info['revision'] = rev
> +        self._channel.send(
> +            'no suitable client available for compilation with info %r' 
% (
> +                info,
> +            )
> +        )
> +
> +    def serve_forever(self):
> +        """this keeps the script from dying, and re-tries jobs"""
> +        self._channel.send('going to serve')
> +        while 1:
> +            time.sleep(self.retry_interval)
> +            self._cleanup_clients()
> +            self._try_queued()
> +
> +    def get_new_buildpath(self, info):
> +        path = BuildPath(str(self._buildpath / 
self._create_filename()))
> +        path.info = info
> +        return path
> +
> +    def compilation_done(self, info, path):
> +        """client is done with compiling and sends data"""
> +        self._channel.send('compilation done for %r, written to %s' % (
> +                                                                info, 
path))
> +        emails = self._requeststorage.add_build(info, path)
> +        for emailaddr in emails:
> +            self._send_email(emailaddr, info, path)
> +
> +    def _cleanup_clients(self):
> +        self._queuelock.acquire()
> +        try:
> +            clients = self._clients[:]
> +            for client in clients:
> +                if client.channel.isclosed():
> +                    if client.busy_on:
> +                        self._queued.append(client.busy_on)
> +                    self._clients.remove(client)
> +        finally:
> +            self._queuelock.release()
> +
> +    def _try_queued(self):
> +        self._queuelock.acquire()
> +        try:
> +            toremove = []
> +            for info in self._queued:
> +                if self.run(info):
> +                    toremove.append(info)
> +            for info in toremove:
> +                self._queued.remove(info)
> +        finally:
> +            self._queuelock.release()
> +
> +    def _get_buildpaths(self, dirpath):
> +        for p in py.path.local(dirpath).listdir():
> +            yield BuildPath(str(p))
> +
> +    _i = 0
> +    def _create_filename(self):
> +        self._namelock.acquire()
> +        try:
> +            today = time.strftime('%Y%m%d')
> +            buildnames = [p.basename for p in 
> +                            py.path.local(self._buildpath).listdir()]
> +            while True:
> +                name = '%s-%s-%s' % (self._projname, today, self._i)
> +                self._i += 1
> +                if name not in buildnames:
> +                    return name
> +        finally:
> +            self._namelock.release()
> +
> +    def _send_email(self, addr, info, path):
> +        self._channel.send('going to send email to %s' % (addr,))
> +        if self._mailhost is not None:
> +            msg = '\r\n'.join([
> +                'From: %s' % (self._mailfrom,),
> +                'To: %s' % (addr,),
> +                'Subject: %s compilation done' % (self._projname,),
> +                '',
> +                'The compilation you requested is done. You can find it 
at',
> +                str(path),
> +            ])
> +            server = smtplib.SMTP(self._mailhost, self._mailport)
> +            server.set_debuglevel(0)
> +            server.sendmail(self._mailfrom, addr, msg)
> +            server.quit()
> +
> +initcode = """
> +    import sys
> +    sys.path += %r
> +
> +    try:
> +        from pypy.tool.build.server import PPBServer
> +        server = PPBServer(%r, channel, %r, %r, %r, %r)
> +
> +        # make the server available to clients as 
pypy.tool.build.ppbserver
> +        from pypy.tool import build
> +        build.ppbserver = server
> +
> +        server.serve_forever()
> +    finally:
> +        channel.close()
> +"""
> +def init(gw, port=12321, path=[], projectname='pypy', buildpath=None,
> +            mailhost=None, mailport=25, mailfrom=None):
> +    from pypy.tool.build import execnetconference
> +    conference = execnetconference.conference(gw, port, True)
> +    channel = conference.remote_exec(initcode % (path, 
projectname,buildpath,
> +                                                    mailhost, mailport,
> +                                                    mailfrom))
> +    return channel
> 
> Added: pypy/dist/pypy/tool/build/test/fake.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/test/fake.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,54 @@
> +from pypy.tool.build.server import BuildPath
> +
> +class FakeChannel(object):
> +    def __init__(self):
> +        self._buffer = []
> +
> +    def send(self, item):
> +        self._buffer.append(item)
> +
> +    def receive(self):
> +        return self._buffer.pop(0)
> +
> +    def close(self):
> +        pass
> +
> +    def waitclose(self):
> +        pass
> +
> +class FakeClient(object):
> +    def __init__(self, info):
> +        self.channel = FakeChannel()
> +        self.sysinfo = info
> +        self.busy_on = None
> +
> +    def compile(self, info):
> +        info.pop('revision')
> +        for k, v in info.items():
> +            self.channel.send('%s: %r' % (k, v))
> +        self.channel.send(None)
> +        self.busy_on = info
> +
> +class FakeServer(object):
> +    def __init__(self, builddirpath):
> +        builddirpath.ensure(dir=True)
> +        self._channel = FakeChannel()
> +        self._builddirpath = builddirpath
> +        self._clients = []
> +        self._done = []
> +
> +    def register(self, client):
> +        self._clients.append(client)
> +
> +    def compilation_done(self, info, data):
> +        self._done.append((info, data))
> + 
> +    i = 0
> +    def get_new_buildpath(self, info):
> +        name = 'build-%s' % (self.i,)
> +        self.i += 1
> +        bp = BuildPath(str(self._builddirpath / name))
> +        bp.info = info
> +        bp.ensure(dir=1)
> +        return bp
> +
> 
> Added: pypy/dist/pypy/tool/build/test/path.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/test/path.py   Wed Jul 26 13:18:25 2006
> @@ -0,0 +1,6 @@
> +import py
> +
> +testpath = py.magic.autopath().dirpath()
> +packagepath = testpath.dirpath()
> +rootpath = packagepath.dirpath().dirpath().dirpath()
> +py.std.sys.path.append(str(rootpath))
> 
> Added: pypy/dist/pypy/tool/build/test/test.zip
> 
==============================================================================
> Binary file. No diff available.
> 
> Added: pypy/dist/pypy/tool/build/test/test_client.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/test/test_client.py   Wed Jul 26 13:18:25 
2006
> @@ -0,0 +1,43 @@
> +import path
> +from pypy.tool.build import client
> +import py
> +import time
> +from fake import FakeChannel, FakeServer
> +
> +class ClientForTests(client.PPBClient):
> +    def __init__(self, *args, **kwargs):
> +        super(ClientForTests, self).__init__(*args, **kwargs)
> +        self._done = []
> + 
> +def setup_module(mod):
> +    mod.temp = temp = py.test.ensuretemp('pypybuilder-client')
> +    mod.svr = svr = FakeServer(temp)
> +
> +    import pypy.tool.build
> +    pypy.tool.build.ppbserver = svr
> +
> +    mod.c1c = c1c = FakeChannel()
> +    mod.c1 = c1 = ClientForTests(c1c, {'foo': 1, 'bar': [1,2]})
> +    svr.register(c1)
> +
> +    mod.c2c = c2c = FakeChannel()
> +    mod.c2 = c2 = ClientForTests(c2c, {'foo': 2, 'bar': [2,3]})
> +    svr.register(c2)
> +
> +def test_compile():
> +    info = {'foo': 1}
> +    c1.compile(info)
> +    c1.channel.receive()
> +    c1.channel.send('foo bar')
> +    c1.channel.send(None)
> +
> +    # meanwhile the client starts a thread that waits until there's 
data 
> +    # available on its own channel, with our FakeChannel it has 
> data rightaway,
> +    # though (the channel out and in are the same, and we just sent 
'info'
> +    # over the out one)
> +    time.sleep(1) 
> + 
> +    done = svr._done.pop()
> + 
> +    assert done[0] == info
> +    assert done[1] == (temp / 'build-0')
> 
> Added: pypy/dist/pypy/tool/build/test/test_pypybuilder.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/test/test_pypybuilder.py   Wed Jul 26 
> 13:18:25 2006
> @@ -0,0 +1,137 @@
> +import path
> +from pypy.tool.build import client, server, execnetconference
> +from pypy.tool.build import config
> +import py
> +
> +# some functional tests (although some of the rest aren't strictly
> +# unit tests either), to run use --functional as an arg to py.test
> +def test_functional_1():
> +    if not py.test.pypybuilder_option.functional:
> +        py.test.skip('skipping functional test, use --functional to run 
it')
> + 
> +    # XXX this one is a bit messy, it's a quick functional test forthe 
whole
> +    # system, but for instance contains time.sleep()s to make sure 
> all threads
> +    # get the time to perform tasks and such... 
> +
> +    sleep_interval = 0.3
> +
> +    # first initialize a server
> +    sgw = py.execnet.PopenGateway()
> +    temppath = py.test.ensuretemp('pypybuilder-functional')
> +    sc = server.init(sgw, port=config.port, path=config.testpath, 
> +                        buildpath=str(temppath))
> +
> +    # give the server some time to wake up
> +    py.std.time.sleep(sleep_interval)
> +
> +    # then two clients, both with different system info
> +    sysinfo1 = {
> +        'foo': 1,
> +        'bar': [1,2],
> +    }
> +    cgw1 = py.execnet.PopenGateway()
> +    cc1 = client.init(cgw1, sysinfo1, port=config.port, testing=True)
> +
> +    sysinfo2 = {
> +        'foo': 2,
> +        'bar': [1],
> +    }
> +    cgw2 = py.execnet.PopenGateway()
> +    cc2 = client.init(cgw2, sysinfo2, port=config.port, testing=True)
> +
> +    # give the clients some time to register themselves
> +    py.std.time.sleep(sleep_interval)
> +
> +    # now we're going to send some compile jobs
> +    code = """
> +        import sys
> +        sys.path += %r
> + 
> +        from pypy.tool.build import ppbserver
> +        channel.send(ppbserver.compile(%r, %r))
> +        channel.close()
> +    """
> +    compgw = py.execnet.PopenGateway()
> +    compconf = execnetconference.conference(compgw, config.port)
> +
> +    # this one should fail because there's no client found for foo = 3
> +    compc = compconf.remote_exec(code % (config.testpath, 
'foo1 at bar.com', 
> +                                            {'foo': 3}))
> + 
> +    # sorry...
> +    py.std.time.sleep(sleep_interval)
> +
> +    ret = compc.receive()
> +    assert not ret[0]
> +    assert ret[1].find('no suitable client found') > -1
> +
> +    # this one should be handled by client 1
> +    compc = compconf.remote_exec(code % (config.testpath, 
'foo2 at bar.com',
> +                                            {'foo': 1, 'bar': [1]}))
> + 
> +    # and another one
> +    py.std.time.sleep(sleep_interval)
> + 
> +    ret = compc.receive()
> +    assert not ret[0]
> +    assert ret[1].find('found a suitable client') > -1
> +
> +    # the messages may take a bit to arrive, too
> +    py.std.time.sleep(sleep_interval)
> +
> +    # client 1 should by now have received the info to build for
> +    cc1.receive() # 'welcome'
> +    ret = cc1.receive() 
> +    assert ret == {'foo': 1, 'bar': [1], 'revision': 'trunk'}
> +
> +    # this should have created a package in the temp dir
> +    assert len(temppath.listdir()) == 1
> +
> +    # now we're going to satisfy the first request by adding a new 
client
> +    sysinfo3 = {'foo': 3}
> +    cgw3 = py.execnet.PopenGateway()
> +    cc3 = client.init(cgw3, sysinfo3, port=config.port, testing=True)
> +
> +    # again a bit of waiting may be desired
> +    py.std.time.sleep(sleep_interval)
> +
> +    # _try_queued() should check whether there are new clients 
available for 
> +    # queued jobs
> +    code = """
> +        import sys, time
> +        sys.path += %r
> + 
> +        from pypy.tool.build import ppbserver
> +        ppbserver._try_queued()
> +        # give the server some time, the clients 'compile' in threads
> +        time.sleep(%s) 
> +        channel.send(ppbserver._requeststorage._id_to_emails)
> +        channel.close()
> +    """
> +    compgw2 = py.execnet.PopenGateway()
> +    compconf2 = execnetconference.conference(compgw2, config.port)
> +
> +    compc2 = compconf2.remote_exec(code % (config.testpath, 
sleep_interval))
> +
> +
> +    # we check whether all emails are now sent, since after adding the 
third
> +    # client, and calling _try_queued(), both jobs should have 
beenprocessed
> +    ret = compc2.receive()
> +    assert ret.values() == []
> +
> +    # this should also have created another package in the temp dir
> +    assert len(temppath.listdir()) == 2
> +
> +    # some cleanup (this should all be in nested try/finallys, blegh)
> +    cc1.close()
> +    cc2.close()
> +    cc3.close()
> +    compc.close()
> +    compc2.close()
> +    sc.close()
> +
> +    cgw1.exit()
> +    cgw2.exit()
> +    compgw.exit()
> +    compgw2.exit()
> +    sgw.exit()
> 
> Added: pypy/dist/pypy/tool/build/test/test_request_storage.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/test/test_request_storage.py   Wed Jul
> 26 13:18:25 2006
> @@ -0,0 +1,64 @@
> +import path
> +import py
> +from pypy.tool.build.server import RequestStorage
> +
> +def test_request_storage():
> +    s = RequestStorage()
> +
> +    assert s._id_to_info == {}
> +    assert s._id_to_emails == {}
> +    assert s._id_to_path == {}
> +
> +    info = {'foo': 1}
> +    infoid = s.get_info_id(info)
> + 
> +    path = s.request('foo at bar.com', info)
> +    assert path is None
> +    assert s._id_to_info == {infoid: info}
> +    assert s._id_to_emails == {infoid: ['foo at bar.com']}
> +    assert s._id_to_path == {}
> +
> +    path = s.request('bar at bar.com', info)
> +    assert path is None
> +    assert s._id_to_info == {infoid: info}
> +    assert s._id_to_emails == {infoid: ['foo at bar.com', 'bar at bar.com']}
> +    assert s._id_to_path == {}
> +
> +    emails = s.add_build(info, 'foobar')
> +    assert emails == ['foo at bar.com', 'bar at bar.com']
> +    assert s._id_to_info == {infoid: info}
> +    assert s._id_to_emails == {}
> +    assert s._id_to_path == {infoid: 'foobar'}
> +
> +    info2 = {'foo': 2, 'bar': [1,2]}
> +    infoid2 = s.get_info_id(info2)
> + 
> +    path = s.request('foo at baz.com', info2)
> +    assert path is None
> +    assert s._id_to_info == {infoid: info, infoid2: info2}
> +    assert s._id_to_emails == {infoid2: ['foo at baz.com']}
> +    assert s._id_to_path == {infoid: 'foobar'}
> +
> +    emails = s.add_build(info2, 'foobaz')
> +    assert emails == ['foo at baz.com']
> +    assert s._id_to_info == {infoid: info, infoid2: info2}
> +    assert s._id_to_emails == {}
> +    assert s._id_to_path == {infoid: 'foobar', infoid2: 'foobaz'}
> +
> +    path = s.request('foo at qux.com', info)
> +    assert path == 'foobar'
> +
> +def test__build_initial():
> +    s = RequestStorage([({'foo': 1}, 'foo'), ({'foo': 2}, 'bar'),])
> +
> +    id1 = s.get_info_id({'foo': 1})
> +    id2 = s.get_info_id({'foo': 2})
> +
> +    assert s._id_to_info == {id1: {'foo': 1}, id2: {'foo': 2}}
> +    assert s._id_to_emails == {}
> +    assert s._id_to_path == {id1: 'foo', id2: 'bar'}
> +
> +def test__normalize():
> +    s = RequestStorage()
> +    assert (s._normalize({'foo': ['bar', 'baz']}) == 
> +            s._normalize({'foo': ['baz', 'bar']}))
> 
> Added: pypy/dist/pypy/tool/build/test/test_server.py
> 
==============================================================================
> --- (empty file)
> +++ pypy/dist/pypy/tool/build/test/test_server.py   Wed Jul 26 13:18:25 
2006
> @@ -0,0 +1,132 @@
> +import path
> +from pypy.tool.build import server
> +import py
> +from fake import FakeChannel, FakeClient
> +from pypy.tool.build.server import RequestStorage
> +from pypy.tool.build.server import BuildPath
> +import time
> +
> +def setup_module(mod):
> +    mod.temppath = temppath = py.test.ensuretemp('pypybuilder-server')
> +    mod.svr = server.PPBServer('pypytest', FakeChannel(), 
str(temppath))
> + 
> +    mod.c1 = FakeClient({'foo': 1, 'bar': [1,2]})
> +    mod.svr.register(mod.c1)
> +
> +    mod.c2 = FakeClient({'foo': 2, 'bar': [2,3]})
> +    mod.svr.register(mod.c2)
> +
> +def test_server_issubdict():
> +    from pypy.tool.build.server import issubdict
> +    assert issubdict({'foo': 1, 'bar': 2}, {'foo': 1, 'bar': 2, 'baz': 
3})
> +    assert not issubdict({'foo': 1, 'bar': 2}, {'foo': 1, 'baz': 3})
> +    assert not issubdict({'foo': 1, 'bar': 3}, {'foo': 1, 'bar': 
2,'baz': 3})
> +    assert issubdict({'foo': [1,2]}, {'foo': [1,2,3]})
> +    assert not issubdict({'foo': [1,2,3]}, {'foo': [1,2]})
> +    assert issubdict({'foo': 1L}, {'foo': 1})
> +    assert issubdict({}, {'foo': 1})
> +    assert issubdict({'foo': [1,2]}, {'foo': [1,2,3,4], 'bar': [1,2]})
> +
> +# XXX: note that the order of the tests matters! the first test reads 
the
> +# information from the channels that was set by the setup_module() 
function,
> +# the rest assumes this information is already read...
> + 
> +def test_register():
> +    assert len(svr._clients) == 2
> +    assert svr._clients[0] == c1
> +    assert svr._clients[1] == c2
> +
> +    assert c1.channel.receive() == 'welcome'
> +    assert c2.channel.receive() == 'welcome'
> +    py.test.raises(IndexError, "c1.channel.receive()")
> +
> +    assert svr._channel.receive().find('registered') > -1
> +    assert svr._channel.receive().find('registered') > -1
> +    py.test.raises(IndexError, 'svr._channel.receive()')
> +
> +def test_compile():
> +    # XXX this relies on the output not changing... quite scary
> +    info = {'foo': 1}
> +    ret = svr.compile('test at domain.com', info)
> +    assert not ret[0]
> +    assert ret[1].find('found a suitable client') > -1
> +    assert svr._channel.receive().find('going to send compile job') > 
-1
> +    assert c1.channel.receive() == 'foo: 1'
> +    assert c1.channel.receive() is None
> +    py.test.raises(IndexError, "c2.channel.receive()")
> +
> +    svr.compile('test at domain.com', {'foo': 3})
> +    assert svr._channel.receive().find('no suitable client available') 
> -1
> +
> +    info = {'bar': [3]}
> +    ret = svr.compile('test at domain.com', info)
> +    assert svr._channel.receive().find('going to send') > -1
> +    assert c2.channel.receive() == 'bar: [3]'
> +    assert c2.channel.receive() is None
> +    py.test.raises(IndexError, "c1.channel.receive()")
> +
> +    info = {'foo': 1}
> +    ret = svr.compile('test at domain.com', info)
> +    assert not ret[0]
> +    assert ret[1].find('this build is already') > -1
> +    assert svr._channel.receive().find('currently in progress') > -1
> +
> +    c1.busy_on = None
> +    bp = BuildPath(str(temppath / 'foo'))
> +    svr.compilation_done(info, bp)
> +    ret = svr.compile('test at domain.com', info)
> +    assert ret[0]
> +    assert isinstance(ret[1], BuildPath)
> +    assert ret[1] == bp
> +    assert svr._channel.receive().find('compilation done for') > -1
> +    for i in range(2):
> +        assert svr._channel.receive().find('going to send email to') > 
-1
> +    assert svr._channel.receive().find('already a build for this info') 
> -1
> + 
> +def test_buildpath():
> +    tempdir = py.test.ensuretemp('pypybuilder-buildpath')
> +    # grmbl... local.__new__ checks for class equality :(
> +    bp = BuildPath(str(tempdir / 'test1')) 
> +    assert not bp.check()
> +    assert bp.info == {}
> +
> +    bp.info = {'foo': 1, 'bar': [1,2]}
> +    assert bp.info == {'foo': 1, 'bar': [1,2]}
> +    assert (sorted((bp / 'info.txt').readlines()) == 
> +            ['bar: [1, 2]\n', 'foo: 1\n'])
> +
> +    assert isinstance(bp.zipfile, py.path.local)
> +    bp.zipfile = ['foo', 'bar', 'baz']
> +    assert bp.zipfile.read() == 'foobarbaz'
> +
> +def test__create_filename():
> +    svr._i = 0 # reset counter
> +    today = time.strftime('%Y%m%d')
> +    name1 = svr._create_filename()
> +    assert name1 == 'pypytest-%s-0' % (today,)
> +    assert svr._create_filename() == ('pypytest-%s-1' % (today,))
> +    bp = BuildPath(str(temppath / ('pypytest-%s-2' % (today,))))
> +    try:
> +        bp.ensure()
> +        assert svr._create_filename() == 'pypytest-%s-3'% (today,)
> +    finally:
> +        bp.remove()
> + 
> +def test_get_new_buildpath():
> +    svr._i = 0
> +    today = time.strftime('%Y%m%d')
> +
> +    path1 = svr.get_new_buildpath({'foo': 'bar'})
> +    try:
> +        assert isinstance(path1, BuildPath)
> +        assert path1.info == {'foo': 'bar'}
> +        assert path1.basename == 'pypytest-%s-0' % (today,)
> +
> +        try:
> +            path2 = svr.get_new_buildpath({'foo': 'baz'})
> +            assert path2.info == {'foo': 'baz'}
> +            assert path2.basename == 'pypytest-%s-1' % (today,)
> +        finally:
> +            path2.remove()
> +    finally:
> +        path1.remove()
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
> 



From guido at merlinux.de  Wed Jul 26 13:47:15 2006
From: guido at merlinux.de (Guido Wesdorp)
Date: Wed, 26 Jul 2006 13:47:15 +0200
Subject: [pypy-dev] [pypy-svn] r30565 - in pypy/dist/pypy/tool/build:
 .	bin builds test
In-Reply-To: <OFA83F9FBD.75F9FF4B-ON802571B7.003F7C6B-802571B7.003F9419@risk.sungard.com>
References: <OFA83F9FBD.75F9FF4B-ON802571B7.003F7C6B-802571B7.003F9419@risk.sungard.com>
Message-ID: <44C75643.6040004@merlinux.de>

Ben.Young at risk.sungard.com wrote:
> Cool tool! Is this something I would be able to run just in the 
> evenings/weekends etc? 
Sure! Do note (I wasn't clear there) that it's a work in progress, and
currently doesn't actually compile yet (it sends out some dummy zip file).

The idea is that a server will at some point be ran on codespeak.net to
which you can connect.

> Will it pick up that fact that I don't have boehm 
> automatically?
>
>   
It should become fairly intelligent, the idea is that the client scans
the system what it's capable of compiling, so I guess that's a yes... ;)
But again, it's a work in progress. Currently in the test data I only
check things like sys.maxint and such, the more advanced stuff will come
later (with the help of someone more initiated into PyPy than I am, I
hope ;).

Cheers,

Guido


From Ben.Young at risk.sungard.com  Wed Jul 26 13:51:59 2006
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Wed, 26 Jul 2006 12:51:59 +0100
Subject: [pypy-dev] [pypy-svn] r30565 - in pypy/dist/pypy/tool/build:
 .	bin builds test
In-Reply-To: <44C75643.6040004@merlinux.de>
Message-ID: <OF2DA340EC.CA4FEE5F-ON802571B7.0040E7DA-802571B7.00412EDB@risk.sungard.com>

Hi Guido,

It's alright, I wasn't about to try it! It will be good to know I can help 
out the project, even though I don't really have the time to do any coding 
as such.

Cheers,
Ben

pypy-dev-bounces at codespeak.net wrote on 26/07/2006 12:47:15:

> Ben.Young at risk.sungard.com wrote:
> > Cool tool! Is this something I would be able to run just in the 
> > evenings/weekends etc? 
> Sure! Do note (I wasn't clear there) that it's a work in progress, and
> currently doesn't actually compile yet (it sends out some dummy zip 
file).
> 
> The idea is that a server will at some point be ran on codespeak.net to
> which you can connect.
> 
> > Will it pick up that fact that I don't have boehm 
> > automatically?
> >
> > 
> It should become fairly intelligent, the idea is that the client scans
> the system what it's capable of compiling, so I guess that's a yes... ;)
> But again, it's a work in progress. Currently in the test data I only
> check things like sys.maxint and such, the more advanced stuff will come
> later (with the help of someone more initiated into PyPy than I am, I
> hope ;).
> 
> Cheers,
> 
> Guido
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 



From guido at merlinux.de  Wed Jul 26 14:01:57 2006
From: guido at merlinux.de (Guido Wesdorp)
Date: Wed, 26 Jul 2006 14:01:57 +0200
Subject: [pypy-dev] [pypy-svn] r30565 - in pypy/dist/pypy/tool/build:
 .	bin builds test
In-Reply-To: <OF2DA340EC.CA4FEE5F-ON802571B7.0040E7DA-802571B7.00412EDB@risk.sungard.com>
References: <OF2DA340EC.CA4FEE5F-ON802571B7.0040E7DA-802571B7.00412EDB@risk.sungard.com>
Message-ID: <44C759B5.6010905@merlinux.de>

Ben.Young at risk.sungard.com wrote:
> It will be good to know I can help 
> out the project, even though I don't really have the time to do any coding 
> as such.
>
>   
Your help will certainly be appreciated! ;)

Keep an eye on this list, I will announce when we can use build clients.

Thanks in advance!

Cheers,

Guido


From jacob at strakt.com  Wed Jul 26 15:57:19 2006
From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=)
Date: Wed, 26 Jul 2006 15:57:19 +0200
Subject: [pypy-dev] Registering Pypy on Freshmeat
Message-ID: <200607261557.20132.jacob@strakt.com>

It just struck me that maybe we should register the Pypy project on Freshmeat. 
It would increase the visibility of the project as well as increase the 
visibility of Python in the more general OpenSource world.

What do people think?

Jacob


From mwh at python.net  Wed Jul 26 17:04:42 2006
From: mwh at python.net (Michael Hudson)
Date: Wed, 26 Jul 2006 16:04:42 +0100
Subject: [pypy-dev] pypy development tasks
References: <44C5EA98.5070700@gmx.net>
Message-ID: <2mslkoxv91.fsf@starship.python.net>

Stephan Diehl <stephan.diehl at gmx.net> writes:

> Hi all,
>
> starting next week, I'll be able again to continue working on pypy.
> As a part-part-part time worker, I find it a bit difficult to stay on 
> top on what's going on (and what needs to be done).
>
> These are the things, I would continue on, if nobody has a better idea:
>
> - write a python module that exposes Coroutine interface (using
>    greenlets, I suppose). This would be usefull for further development
>    of applevel stackless module. As a sideeffect, the stackless module
>    could then be used with CPython.

Erm, don't we have this part already?  Oh, you mean on top of
_CPython_, rather than on top of py.py on top of CPython.  I guess
this would help testing...

> - continue work on applevel stackless module and documentation.

This would be good.  Samuele pointed out some more stackless examples
at:

    http://members.verizon.net/olsongt/stackless/why_stackless.html

It would be nice to add these to the test suite (some of the examples
use pygame, so those need to be changed).

> Other than that, if I should rather do things like adding docstrings or 
> do other usefull tasks, don't hesitate to tell me :-)

We've just had a quick IRC discussion, and we decided that it would be
really nice if you could work on the pre-emptive scheduling mentioned
in the description of WP07; this will require adding a hook to the
interpreter than is called every few hundred bytecodes, but other than
that should mostly be implementable at app level (and the design
stolen from stackless CPython, I guess).

Cheers,
mwh

-- 
  <glyph> I am *not* a PSU agent.
                                                -- from Twisted.Quotes



From leo.soto at gmail.com  Wed Jul 26 23:48:35 2006
From: leo.soto at gmail.com (Leonardo Soto)
Date: Wed, 26 Jul 2006 17:48:35 -0400
Subject: [pypy-dev] Fix for #145 (-m cmdline option)
Message-ID: <5a8cc9e00607261448t7ea3d9bfo9cf567b4124681ed@mail.gmail.com>

Hi,

I've tried to upload the attached patch on roundup, but it gives me a
"Required issue properties title, priority not supplied" error. It
could be a permission error, but the message don't suggest that.

So the patch is here. Comments and corrections are welcomed.

-- 
Leonardo Soto M.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pypy-pep-338.patch
Type: text/x-patch
Size: 28507 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20060726/62259ba3/attachment.bin>

From elmo13 at jippii.fi  Thu Jul 27 00:42:48 2006
From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=)
Date: Thu, 27 Jul 2006 01:42:48 +0300
Subject: [pypy-dev] A bug in cc?
In-Reply-To: <20060725093526.GA25834@code0.codespeak.net>
References: <44BE2F9E.4060306@jippii.fi>
	<20060721152939.GB31604@code0.codespeak.net>
	<44C0FBCA.1090605@jippii.fi> <44C1388D.80003@jippii.fi>
	<44C3C339.8070809@jippii.fi>
	<20060724205610.GA25017@code0.codespeak.net>
	<44C54333.5080701@jippii.fi>
	<20060725093526.GA25834@code0.codespeak.net>
Message-ID: <44C7EFE8.10903@jippii.fi>

Armin Rigo wrote:
> Hi Elmo,
>
> On Tue, Jul 25, 2006 at 01:01:23AM +0300, Elmo M?ntynen wrote:
>   
>> I have 4.0.4 (debian 4.0.3-4). How should we go about testing on the
>> latest gcc, or are you doing it right now. I can definitely reproduce
>> the bug with the that certain file(s). Which of the .gcda and .gcno (or
>> the rest) files should I send?
>>     
>
> I don't know what these files are, so I can't comment, but you should
> look or ask around in the gcc community instead.  Also, I'm sure they
> would like to see information related to their latest release, so you
> should try with that.  And of course you should try to see if it's a
> known bug, maybe even if it's been fixed already...
>
>
> A bientot,
>
> Armin
>   
Just to note: I ran into some obstacles, and regenerating the -fprofile
related file produced a non crashing one. Simply, compiling with
profiling works, and so I will drop this.

Elmo


From stephan.diehl at gmx.net  Thu Jul 27 10:33:04 2006
From: stephan.diehl at gmx.net (Stephan Diehl)
Date: Thu, 27 Jul 2006 10:33:04 +0200
Subject: [pypy-dev] pypy development tasks
In-Reply-To: <2mslkoxv91.fsf@starship.python.net>
References: <44C5EA98.5070700@gmx.net> <2mslkoxv91.fsf@starship.python.net>
Message-ID: <44C87A40.9030908@gmx.net>


> 
> Erm, don't we have this part already?  Oh, you mean on top of
> _CPython_, rather than on top of py.py on top of CPython.  I guess
> this would help testing...

yep, that's it: coroutine on top of cpython.

> 
>> - continue work on applevel stackless module and documentation.
> 
> This would be good.  Samuele pointed out some more stackless examples
> at:
> 
>     http://members.verizon.net/olsongt/stackless/why_stackless.html
> 
> It would be nice to add these to the test suite (some of the examples
> use pygame, so those need to be changed).

sounds like a good idea...

> 
>> Other than that, if I should rather do things like adding docstrings or 
>> do other usefull tasks, don't hesitate to tell me :-)
> 
> We've just had a quick IRC discussion, and we decided that it would be
> really nice if you could work on the pre-emptive scheduling mentioned
> in the description of WP07; this will require adding a hook to the
> interpreter than is called every few hundred bytecodes, but other than
> that should mostly be implementable at app level (and the design
> stolen from stackless CPython, I guess).

I'll look into that.

More on it next week.

Cheers

Stephan
> 
> Cheers,
> mwh
> 



From arigo at tunes.org  Thu Jul 27 11:07:29 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 27 Jul 2006 11:07:29 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h
	for	win32
In-Reply-To: <44C6042D.7090806@scottdial.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<44C6042D.7090806@scottdial.com>
Message-ID: <20060727090729.GA25278@code0.codespeak.net>

Hi Scottt,

On Tue, Jul 25, 2006 at 07:44:45AM -0400, Scott Dial wrote:
> I actually was pretty suprised too.. Presuming this is an important 
> platform to test (*grin*), I can provide a daily testing run for 
> Windows. I don't know what the code behind your pypytest is, but I 
> presume I could run something similar? Let me know.

The scripts I use in the following directory:

    http://codespeak.net/svn/user/arigo/hack/misc/htmlconftest/

While some details might be Unix-specific, it shouldn't be too hard to
adapt them to Windows.  (They produce symlinks, for example.)

Put the scripts in a directory, together with a subdir 'html' (initially
empty) which is visible on the web, and a checkout of PyPy in
'pypy-dist'.  The entry point script is autotest.py.


A bientot,

Armin


From arigo at tunes.org  Thu Jul 27 11:17:07 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 27 Jul 2006 11:17:07 +0200
Subject: [pypy-dev] extension compiler and Python special methods
In-Reply-To: <9eebf5740607251151y29f5341l43bc92fe28ff592e@mail.gmail.com>
References: <9eebf5740607251151y29f5341l43bc92fe28ff592e@mail.gmail.com>
Message-ID: <20060727091707.GA27011@code0.codespeak.net>

Hi Lawrence,

On Tue, Jul 25, 2006 at 08:51:09PM +0200, Lawrence Oluyede wrote:
> As u can see in rev. #30543 I commited a series of methods to complete
> the mmap module. Those are the special methods (__xyz__) implemented
> as regular ones. I thought it was best to have the functionality in
> anyway. When and if we'll have that support it will be only a matter
> of change methods names to gain compatibility with CPython's code.
> 
> If you disagree with me let me know, I'll comment out the code or whatever.

That's fine.  Note also that you can use special method *names* in
RPython code, you just have to call them explicitly.  So for example
it's ok to have code like:

    class W_X:
       def __len__(self):
           return 42

       def get_len(self, space):
           return space.wrap(self.__len__())

Or even:

    class PurePythonBase:
        def __len__(self):
            return 42

    class W_X(PurePythonBase):
        def get_len(self, space):
            return space.wrap(self.__len__())


A bientot,

Armin


From l.oluyede at gmail.com  Thu Jul 27 11:17:12 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Thu, 27 Jul 2006 11:17:12 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <20060725093843.GA26167@code0.codespeak.net>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
Message-ID: <9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>

I think it could also be better to refactor a bit that header. I
placed the header I needed for my extension modules but I realize
that's not the right place for them. We can move out "my" includes in
a separate modules.h and include it in g_prerequisite.h or g_include.h
or whatever.



-- 
Lawrence
http://www.oluyede.org/blog


From l.oluyede at gmail.com  Thu Jul 27 11:20:02 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Thu, 27 Jul 2006 11:20:02 +0200
Subject: [pypy-dev] extension compiler and Python special methods
In-Reply-To: <20060727091707.GA27011@code0.codespeak.net>
References: <9eebf5740607251151y29f5341l43bc92fe28ff592e@mail.gmail.com>
	<20060727091707.GA27011@code0.codespeak.net>
Message-ID: <9eebf5740607270220v75a9c68fqb3a862fc89a004de@mail.gmail.com>

> That's fine.  Note also that you can use special method *names* in
> RPython code, you just have to call them explicitly.

Is it some sort of coding style rule? I think is ok but sounds like a
duplication of code to me now...


-- 
Lawrence
http://www.oluyede.org/blog


From arigo at tunes.org  Thu Jul 27 11:55:08 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 27 Jul 2006 11:55:08 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>
Message-ID: <20060727095508.GB27011@code0.codespeak.net>

Hi Lawrence,

On Thu, Jul 27, 2006 at 11:17:12AM +0200, Lawrence Oluyede wrote:
> I think it could also be better to refactor a bit that header. I
> placed the header I needed for my extension modules but I realize
> that's not the right place for them. We can move out "my" includes in
> a separate modules.h and include it in g_prerequisite.h or g_include.h
> or whatever.

The rctypes-based modules' includes shouldn't go in the static .h files.
They are generated automatically by genc if you use the correct magic:
look for _includes_ in module/readline/c_readline.py.


A bientot,

Armin


From arigo at tunes.org  Thu Jul 27 11:58:25 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 27 Jul 2006 11:58:25 +0200
Subject: [pypy-dev] extension compiler and Python special methods
In-Reply-To: <9eebf5740607270220v75a9c68fqb3a862fc89a004de@mail.gmail.com>
References: <9eebf5740607251151y29f5341l43bc92fe28ff592e@mail.gmail.com>
	<20060727091707.GA27011@code0.codespeak.net>
	<9eebf5740607270220v75a9c68fqb3a862fc89a004de@mail.gmail.com>
Message-ID: <20060727095825.GC27011@code0.codespeak.net>

Hi Lawrence,

On Thu, Jul 27, 2006 at 11:20:02AM +0200, Lawrence Oluyede wrote:
> > That's fine.  Note also that you can use special method *names* in
> > RPython code, you just have to call them explicitly.
> 
> Is it some sort of coding style rule? I think is ok but sounds like a
> duplication of code to me now...

Well, as long as we have to write the space wrappers by hand, but would
also like the "pure" method to be available, there is going to be this
kind of duplication :-(  At some point, the unwrap_spec could be
enhanced to cover much more automatic wrapping and unwrapping; then
we'll be able to use the "pure" method directly, as in:

    class X:
        def __len__(self):
            return 42
        __len__.unwrap_spec = ['self'], 'returns an int'  # or whatever

and directly point to the __len__ method in the TypeDef, getting rid of
all explicit references to 'space' in the simple methods.


A bientot,

Armin.


From l.oluyede at gmail.com  Thu Jul 27 13:06:08 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Thu, 27 Jul 2006 13:06:08 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <20060727095508.GB27011@code0.codespeak.net>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>
	<20060727095508.GB27011@code0.codespeak.net>
Message-ID: <9eebf5740607270406q43d44e31t3b965c07365529f5@mail.gmail.com>

> The rctypes-based modules' includes shouldn't go in the static .h files.
> They are generated automatically by genc if you use the correct magic:
> look for _includes_ in module/readline/c_readline.py.

I definitely missed that! Thanks.

-- 
Lawrence
http://www.oluyede.org/blog


From l.oluyede at gmail.com  Thu Jul 27 13:21:57 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Thu, 27 Jul 2006 13:21:57 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <9eebf5740607270406q43d44e31t3b965c07365529f5@mail.gmail.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>
	<20060727095508.GB27011@code0.codespeak.net>
	<9eebf5740607270406q43d44e31t3b965c07365529f5@mail.gmail.com>
Message-ID: <9eebf5740607270421r227820dax7ffc0f1879aa366@mail.gmail.com>

On 7/27/06, Lawrence Oluyede <l.oluyede at gmail.com> wrote:
> > The rctypes-based modules' includes shouldn't go in the static .h files.
> > They are generated automatically by genc if you use the correct magic:
> > look for _includes_ in module/readline/c_readline.py.
>
> I definitely missed that! Thanks.

That doesn't seem to work. I commented out the includes I added to
make fcntl module compile (like sys/ioctl.h for example) and added
them to the _includes_ and this seem to work correctly only with tests
and the generation of ctypesplatcheck files. The headers are not
injected in the generated fcntl.c nor in any of the file it includes.

-- 
Lawrence
http://www.oluyede.org/blog


From l.oluyede at gmail.com  Thu Jul 27 13:32:25 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Thu, 27 Jul 2006 13:32:25 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <9eebf5740607270406q43d44e31t3b965c07365529f5@mail.gmail.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>
	<20060727095508.GB27011@code0.codespeak.net>
	<9eebf5740607270406q43d44e31t3b965c07365529f5@mail.gmail.com>
Message-ID: <9eebf5740607270432p6791448fm96e0dc26f590261a@mail.gmail.com>

On 7/27/06, Lawrence Oluyede <l.oluyede at gmail.com> wrote:
> > The rctypes-based modules' includes shouldn't go in the static .h files.
> > They are generated automatically by genc if you use the correct magic:
> > look for _includes_ in module/readline/c_readline.py.
>
> I definitely missed that! Thanks.

That doesn't seem to work. I commented out the includes I added to
make fcntl module compile (like sys/ioctl.h for example) and added
them to the _includes_ and this seem to work correctly only with tests
and the generation of ctypesplatcheck files. The headers are not
injected in the generated fcntl.c nor in any of the file it includes.

-- 
Lawrence
http://www.oluyede.org/blog


From fijal at genesilico.pl  Thu Jul 27 15:58:12 2006
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Thu, 27 Jul 2006 15:58:12 +0200
Subject: [pypy-dev] Good news everyone
Message-ID: <44C8C674.7030108@genesilico.pl>

Right now, b'n'b JS client should be almost playable. Speed and key 
granularity is still an issue, but there should be no problems with 
zindex and all other problems. Enjoy.




From radix at twistedmatrix.com  Thu Jul 27 17:08:01 2006
From: radix at twistedmatrix.com (Christopher Armstrong)
Date: Thu, 27 Jul 2006 11:08:01 -0400
Subject: [pypy-dev] Failing stackless tests
In-Reply-To: <20060721203613.GA2825@code0.codespeak.net>
References: <20060721203613.GA2825@code0.codespeak.net>
Message-ID: <60ed19d40607270808k6fa6d24di61ba290264acadb@mail.gmail.com>

On 7/21/06, Armin Rigo <arigo at tunes.org> wrote:
> Hi all, hi Aurelien,
>
> The stackless-related tests are all consistently failing nowadays.  Some
> have been failing for 4 days and others for 9 days.  See
>
>    http://snake.cs.uni-duesseldorf.de/pypytest/summary.html
>
> Given that you appear to continue to work on it, but without making
> tests pass again, I'm a bit worrying exactly about what you are doing?
> That doesn't seem too much in line with the "test-driven development"
> approach.

I'll point out that Twisted has been avoiding issues such as these in
the past 8 months or so by using a process that involves always
branching the repository for developments and always getting a review
before merging to trunk. The administrative overhead has definitely
paid off in much, much higher code and test coverage.

-- 
Christopher Armstrong
International Man of Twistery
http://radix.twistedmatrix.com/
http://twistedmatrix.com/
http://canonical.com/


From 2006a at usenet.alexanderweb.de  Thu Jul 27 19:09:14 2006
From: 2006a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Thu, 27 Jul 2006 19:09:14 +0200
Subject: [pypy-dev] Combining strdicts and LLVM-C-Prof ...
Message-ID: <1ol7ygzoxm0fx.dlg@usenet.alexanderweb.de>

Hi,

today I made some tests of the strdict code on snake:

date                           size codesize    executable                        richards             pystone
Thu Jul 27 20:14:33 2006          -        -    python 2.4.3                        795ms (   1.0x)    40000 (   1.0x)
Thu Jul 27 20:08:02 2006   10425736  2475162    pypy-llvm-30641-x86                2902ms (   3.6x)     9191 (   4.4x)
Thu Jul 27 20:05:53 2006   10474191  2028604    pypy-llvm-30641-c-prof             1795ms (   2.3x)    14619 (   2.7x)
Thu Jul 27 19:58:41 2006   10574247  2128805    pypy-llvm-30641-c                  2193ms (   2.8x)    11441 (   3.5x)
Thu Jul 27 17:37:21 2006   10162160  2208132    pypy-c-30634-profopt               2380ms (   3.0x)    12285 (   3.3x)
Thu Jul 27 16:56:55 2006   10382148  2428236    pypy-c-30634                       2590ms (   3.3x)    10846 (   3.7x)

2.3x is really neat - 1.0x is not far :-)

Kind regards,
Alexander



From scott+pypy-dev at scottdial.com  Fri Jul 28 19:41:11 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Fri, 28 Jul 2006 13:41:11 -0400
Subject: [pypy-dev] small patch for
	translator/c/src/ll_osdefs.h	for	win32
In-Reply-To: <20060727090729.GA25278@code0.codespeak.net>
References: <44C53C95.4070002@scottdial.com>	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>	<20060725093843.GA26167@code0.codespeak.net>	<44C6042D.7090806@scottdial.com>
	<20060727090729.GA25278@code0.codespeak.net>
Message-ID: <44CA4C37.3060709@scottdial.com>

Armin Rigo wrote:
> While some details might be Unix-specific, it shouldn't be too hard to
> adapt them to Windows.  (They produce symlinks, for example.)

I was able to adapt the script just fine. And I ran it once without 
problems (with the script, not pypy ;). But, I realized I made an error 
and forget to setup the build enviroment for compiling. After doing so, 
I find that I get a stalled in module\_stackless\test

Nevertheless, in the future the results will be at 
http://scottdial.com/pypytest/

I will look into patching the misbehaving tests.

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu


From scott+pypy-dev at scottdial.com  Sat Jul 29 06:48:46 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Sat, 29 Jul 2006 00:48:46 -0400
Subject: [pypy-dev] MSVC "too many initializer" problem
Message-ID: <44CAE8AE.4070006@scottdial.com>

Right now, if I were to enter the translatorshell and try to compile any 
of the snippets I will get an the following error:

testing_1.c(2318) : warning C4047: 'initializing' : 'char *' differs in 
levels of indirection from 'int'
testing_1.c(2318) : error C2078: too many initializers

Which refers to this line (for this example):

static globalfunctiondef_t globalfunctiondefs[] = {
	{&pypy_gfunc_sieve_of_eratosthenes, "pypy_gfunc_sieve_of_eratosthenes", 
{"sieve_of_eratosthenes", 
(PyCFunction)pypy_g_pyfn_sieve_of_eratosthenes, 
METH_VARARGS|METH_KEYWORDS, {
83,105,101,118,101,32,111,102,32,69,114,97,116,111,115,116,104,101,110,101,
115,10,32,32,32,32,10,32,32,32,32,84,104,105,115,32,111,110,101,32,
105,115,32,102,114,111,109,32,97,110,32,105,110,102,97,109,111,117,115,32,
98,101,110,99,104,109,97,114,107,44,32,34,84,104,101,32,71,114,101,97,
116,32,67,111,109,112,117,116,101,114,10,32,32,32,32,76,97,110,103,117,
97,103,101,32,83,104,111,111,116,111,117,116,34,46,10,10,32,32,32,32,
85,82,76,32,105,115,58,32,104,116,116,112,58,47,47,119,119,119,46,98,
97,103,108,101,121,46,111,114,103,47,126,100,111,117,103,47,115,104,111,111,
116,111,117,116,47,10,32,32,32,32}}},
	{ NULL }	/* Sentinel */
};

I'm not too familiar with structure initialization in MSVC's compiler, 
but it seems to barf on that one. If anyone has a quick answer, then I 
am all ears, otherwise I'll have to do some research.

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu


From arigo at tunes.org  Sat Jul 29 09:05:49 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 29 Jul 2006 09:05:49 +0200
Subject: [pypy-dev] small patch for
	translator/c/src/ll_osdefs.h	for	win32
In-Reply-To: <44CA4C37.3060709@scottdial.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<44C6042D.7090806@scottdial.com>
	<20060727090729.GA25278@code0.codespeak.net>
	<44CA4C37.3060709@scottdial.com>
Message-ID: <20060729070549.GA22919@code0.codespeak.net>

Hi Scott,

On Fri, Jul 28, 2006 at 01:41:11PM -0400, Scott Dial wrote:
> Nevertheless, in the future the results will be at 
> http://scottdial.com/pypytest/

Thanks a lot!  Can we link to this page from the "Status" section of the
documentation index?


A bientot,

Armin


From arigo at tunes.org  Sat Jul 29 09:08:36 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 29 Jul 2006 09:08:36 +0200
Subject: [pypy-dev] small patch for translator/c/src/ll_osdefs.h for
	win32
In-Reply-To: <9eebf5740607270432p6791448fm96e0dc26f590261a@mail.gmail.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<9eebf5740607270217q38087a07idab3ff95ee8e6@mail.gmail.com>
	<20060727095508.GB27011@code0.codespeak.net>
	<9eebf5740607270406q43d44e31t3b965c07365529f5@mail.gmail.com>
	<9eebf5740607270432p6791448fm96e0dc26f590261a@mail.gmail.com>
Message-ID: <20060729070836.GB22919@code0.codespeak.net>

Hi Lawrence,

On Thu, Jul 27, 2006 at 01:32:25PM +0200, Lawrence Oluyede wrote:
> That doesn't seem to work. I commented out the includes I added to
> make fcntl module compile (like sys/ioctl.h for example) and added
> them to the _includes_ and this seem to work correctly only with tests
> and the generation of ctypesplatcheck files. The headers are not
> injected in the generated fcntl.c nor in any of the file it includes.

Strange: it works with the readline module.  If you run 'compilemodule.py
readline' you can check that /tmp/usession-*/readline/common_header.h
contains '#include <readline/readline.h>'.


A bientot,

Armin.


From arigo at tunes.org  Sat Jul 29 09:21:52 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 29 Jul 2006 09:21:52 +0200
Subject: [pypy-dev] MSVC "too many initializer" problem
In-Reply-To: <44CAE8AE.4070006@scottdial.com>
References: <44CAE8AE.4070006@scottdial.com>
Message-ID: <20060729072152.GC22919@code0.codespeak.net>

Hi Scott,

On Sat, Jul 29, 2006 at 12:48:46AM -0400, Scott Dial wrote:
> Right now, if I were to enter the translatorshell and try to compile any 
> of the snippets I will get an the following error:
> 
> testing_1.c(2318) : warning C4047: 'initializing' : 'char *' differs in 
> levels of indirection from 'int'
> testing_1.c(2318) : error C2078: too many initializers

Indeed, the line in question is not valid C at all.  In gcc it just
produces a lot of warnings and invalid data, but not an actual error :-/

This list of numbers is actually means to be the docstring of the
function, but in this broken syntax gcc uses the first number only and
interprets that number as a char*.  So we get (char*)83 as the C-level
docstring.  You bet that accessing it via the __doc__ attribute of the
built-in function gives a nice segfault...  What is not tested is
broken :-(

I'll try to fix this.


A bientot,

Armin.


From l.oluyede at gmail.com  Sat Jul 29 15:58:45 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Sat, 29 Jul 2006 15:58:45 +0200
Subject: [pypy-dev] make _ssl module compile
Message-ID: <9eebf5740607290658w2dba6c96n38e5ab6d61b5c41b@mail.gmail.com>

I'm trying to feed the _ssl module (feature complete) to the ext.
compiler but I get a strange error I've never had before:

[translation:ERROR] Error:
[translation:ERROR]  AttributeError: 'CallEntry' object has no
attribute 'get_repr'
[translation] start debugger...
> /Users/rhymes/scc/pypy/pypy/rpython/rexternalobj.py(20)rtyper_makerepr()
-> return entry.get_repr(rtyper, self)
(Pdb) self
SomeCTypesObject(knowntype=CFunctionType, ownsmemory=False)
(Pdb) entry
<pypy.rpython.rctypes.afunc.CallEntry object at 0x202fd10>
(Pdb) locals()
{'entry': <pypy.rpython.rctypes.afunc.CallEntry object at 0x202fd10>,
'rtyper': <pypy.rpython.rtyper.RPythonTyper object at 0x172ef50>,
'self': SomeCTypesObject(knowntype=CFunctionType, ownsmemory=False)}
(Pdb) dir(entry)
['__class__', '__delattr__', '__dict__', '__doc__', '__eq__',
'__getattribute__', '__hash__', '__init__', '__metaclass__',
'__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__str__', '__weakref__',
'compute_annotation', 'compute_result_annotation', 'instance',
'specialize_call', 'type']

I took a look into rctypes.afunc and in fact there's no get_repr
(because there's no rfunc.py). What problem is that?

-- 
Lawrence
http://www.oluyede.org/blog


From scott+pypy-dev at scottdial.com  Sat Jul 29 17:27:49 2006
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Sat, 29 Jul 2006 11:27:49 -0400
Subject: [pypy-dev] small patch
	for	translator/c/src/ll_osdefs.h	for	win32
In-Reply-To: <20060729070549.GA22919@code0.codespeak.net>
References: <44C53C95.4070002@scottdial.com>	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>	<20060725093843.GA26167@code0.codespeak.net>	<44C6042D.7090806@scottdial.com>	<20060727090729.GA25278@code0.codespeak.net>	<44CA4C37.3060709@scottdial.com>
	<20060729070549.GA22919@code0.codespeak.net>
Message-ID: <44CB7E75.5010805@scottdial.com>

Armin Rigo wrote:
> On Fri, Jul 28, 2006 at 01:41:11PM -0400, Scott Dial wrote:
>> Nevertheless, in the future the results will be at 
>> http://scottdial.com/pypytest/
> 
> Thanks a lot!  Can we link to this page from the "Status" section of the
> documentation index?
> 

You sure can. I'll do my best to keep it available as much as possible. 
I'll be moving in 3 weeks and there'll be some days of downtime while I 
get my internet hooked up there, but otherwise I intend to keep it 
updated and available.

I'm not sure what time is the best for it to run daily. I've currently 
set it run daily at midnight GMT. Although I may have to change that 
depending on my sleeping behavior.. when it runs probably isn't that big 
of a concern as long it gets run, yah?

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu


From leo.soto at gmail.com  Sun Jul 30 03:11:14 2006
From: leo.soto at gmail.com (Leonardo Soto)
Date: Sat, 29 Jul 2006 21:11:14 -0400
Subject: [pypy-dev] gateway.applevel and kwargs
Message-ID: <5a8cc9e00607291811k3a0d0902w6e1cc7bccd170308@mail.gmail.com>

Hi all,

I've recently filled the issue 244 (dict.update() doesn't support
keywords argument), and tried to solve it in a naive way, just
modifying the applevel  definition to:

    def update(d, o, **kwargs):
        if hasattr(o, 'keys'):
            for k in o.keys():
                d[k] = o[k]
        else:
            for k,v in o:
                d[k] = v
        for k in kwargs:
            d[k] = kwargs[k]

But it simply doesn't work:

test_dictobject.py[34] .......................FF.........

__________________________________________________________________________________________________________________________________________
__________________________________________ entrypoint:
AppTest_DictObject().test_update_kwargs
___________________________________________

    def test_update_kwargs(self):
        d = {}
E       d.update(foo='bar', baz=1)
>       (application-level) TypeError: update() got 2 unexpected keyword arguments

[/home/leo/src/pypy-dist/pypy/objspace/std/test/None</home/leo/src/pypy-dist/pypy/interpreter/gateway.py:754>:3]
__________________________________________________________________________________________________________________________________________
______________________________________ entrypoint:
AppTest_DictObject().test_update_dict_and_kwargs
______________________________________

    def test_update_dict_and_kwargs(self):
        d = {}
E       d.update({'foo': 'bar'}, baz=1)
>       (application-level) TypeError: update() got an unexpected keyword argument 'baz'

[/home/leo/src/pypy-dist/pypy/objspace/std/test/None</home/leo/src/pypy-dist/pypy/interpreter/gateway.py:754>:3]

Why?. Should it work or is the lack of kwargs support a known limitation?

-- 
Leonardo Soto M.


From arigo at tunes.org  Sun Jul 30 09:10:27 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 30 Jul 2006 09:10:27 +0200
Subject: [pypy-dev] small patch
	for	translator/c/src/ll_osdefs.h	for	win32
In-Reply-To: <44CB7E75.5010805@scottdial.com>
References: <44C53C95.4070002@scottdial.com>
	<9eebf5740607242145j5fe29b20t2b10de9c28747b33@mail.gmail.com>
	<20060725093843.GA26167@code0.codespeak.net>
	<44C6042D.7090806@scottdial.com>
	<20060727090729.GA25278@code0.codespeak.net>
	<44CA4C37.3060709@scottdial.com>
	<20060729070549.GA22919@code0.codespeak.net>
	<44CB7E75.5010805@scottdial.com>
Message-ID: <20060730071027.GA21606@code0.codespeak.net>

Hi Scott,

On Sat, Jul 29, 2006 at 11:27:49AM -0400, Scott Dial wrote:
> You sure can.

Thanks!

> I'm not sure what time is the best for it to run daily. I've currently 
> set it run daily at midnight GMT. Although I may have to change that 
> depending on my sleeping behavior.. when it runs probably isn't that big 
> of a concern as long it gets run, yah?

Indeed.  On snake it's in the early morning, just to minimize the risk
that someone else is doing something.


A bientot,

Armin