From nvetoshkin at naumen.ru  Tue Sep  2 20:40:58 2008
From: nvetoshkin at naumen.ru (Vetoshkin Nikita)
Date: Wed, 03 Sep 2008 00:40:58 +0600
Subject: [pypy-dev] translate.py failes
In-Reply-To: <20080829155511.GA9583@code0.codespeak.net>
References: <48B6C0B6.6060802@naumen.ru>
	<20080828153110.GA8912@code0.codespeak.net>
	<48B6D3AB.4050801@naumen.ru>
	<20080829155511.GA9583@code0.codespeak.net>
Message-ID: <48BD88BA.3010900@naumen.ru>

Thanks a lot, Armin.
I know how os.fork() works and about copy-on-write too, so as soon as I 
got time, I'll try to fix it.
My interest in translating and playing then with pypy was exactly 
py.execnet =)

Armin Rigo wrote:
> Hi,
>
> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>> Is there a direction, where I could digg to help? Something to begin with.
>> I don't have enough experience but I'd like to take part (even very
>> small) in this project.
>
> Sure.  Do you understand the issue?  It's related to the way a Unix
> process starts a new subprocess: os.fork() followed by os.execv() in the
> child process.  The issue is that os.fork() fails if the parent process'
> total memory is more than half of your RAM+swap, because it tries to
> create a copy of the process, and the copy would also need more than
> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
> at all, but creates "shared pages" of memory so that memory pages are
> duplicated only when one of the two processes really modifies it; but
> still os.fork() has to mark all the memory as *potentially* used, and
> raises MemoryError if there isn't enough.)
>
> A solution might be to split translate.py in two processes: translate.py
> would start a subprocess when it starts, and then do all the work
> (consuming a lot of RAM); but when it needs to start new processes, e.g.
> call gcc, it would instead ask the subprocess to start the new
> processes.  More precisely, the code in pypy.translator.tool.cbuild
> would stop using distutils directly, and instead ask the subprocess to
> use distutils.  It looks even easy to do with the help of a
> py.execnet.PopenGateway() created when translate.py starts (see the py
> lib documentation at http://codespeak.net/py/).
>
>
> A bientot,
>
> Armin

-- 
---
? ?????????,
???????? ??????

??????? ????????? ?????? IP-?????????
????????? ?????
???????? Naumen


From seob at gmx.ch  Wed Sep  3 11:31:01 2008
From: seob at gmx.ch (Severin)
Date: Wed, 3 Sep 2008 11:31:01 +0200
Subject: [pypy-dev] master thesis with pypy
In-Reply-To: <5b0248170808260921j7c2e4bb4hb97980d7e10d8818@mail.gmail.com>
References: <a3ea99f80808260835g71de645q3152c6c0c0a458de@mail.gmail.com>
	<5b0248170808260921j7c2e4bb4hb97980d7e10d8818@mail.gmail.com>
Message-ID: <a3ea99f80809030231w1549872ale4675fc3490b533b@mail.gmail.com>

safethread sound like something interesting. are there already any safthread
ideas in pypy?
In general, working on GILless ideas sounds more interesting than doing
something on multiprocess framworks.

It's a master thesis, that means there is room for experimental stuff, but
in the end half a year is quite short, so the room is limited again :)

was there any attempt to remove the GIL in pypy so far?

--
Severin



On Tue, Aug 26, 2008 at 6:21 PM, Seo Sanghyeon <sanxiyn at gmail.com> wrote:

> 2008/8/27 Severin <seob at gmx.ch>:
> > We started to talk about what we could do and what would be interesting
> for
> > us. In some way we wanted to 'improve multiprocessor capabilities' in
> pypy.
> > We had some ideas so far, but we are not yet fixed on something.
> >
> > I was wondering if somebody of you has something that is of particular
> > interest to the pypy community. Some ideas that might be interesting to
> look
> > at for me. Any kind of input or idea is welcome...
>
> A random idea. Implementing python-safethread monitors for PyPy?
> http://code.google.com/p/python-safethread/wiki/Monitors
>
> --
> Seo Sanghyeon
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20080903/4d7ab7a7/attachment.html>

From fijall at gmail.com  Wed Sep  3 11:44:29 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 3 Sep 2008 11:44:29 +0200
Subject: [pypy-dev] master thesis with pypy
In-Reply-To: <a3ea99f80809030231w1549872ale4675fc3490b533b@mail.gmail.com>
References: <a3ea99f80808260835g71de645q3152c6c0c0a458de@mail.gmail.com>
	<5b0248170808260921j7c2e4bb4hb97980d7e10d8818@mail.gmail.com>
	<a3ea99f80809030231w1549872ale4675fc3490b533b@mail.gmail.com>
Message-ID: <693bc9ab0809030244i10c50ae0t8bd1dc18da4a90d1@mail.gmail.com>

On Wed, Sep 3, 2008 at 11:31 AM, Severin <seob at gmx.ch> wrote:
> safethread sound like something interesting. are there already any safthread
> ideas in pypy?
> In general, working on GILless ideas sounds more interesting than doing
> something on multiprocess framworks.
> It's a master thesis, that means there is room for experimental stuff, but
> in the end half a year is quite short, so the room is limited again :)
> was there any attempt to remove the GIL in pypy so far?
> --
> Severin
>

None that I know of at least. I think any serious GIL-removal proposal
requires some deeper thinking about what exactly are the expected
semantics.

Cheers,
fijal


From rhamph at gmail.com  Wed Sep  3 20:15:37 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 3 Sep 2008 12:15:37 -0600
Subject: [pypy-dev] master thesis with pypy
In-Reply-To: <693bc9ab0809030244i10c50ae0t8bd1dc18da4a90d1@mail.gmail.com>
References: <a3ea99f80808260835g71de645q3152c6c0c0a458de@mail.gmail.com>
	<5b0248170808260921j7c2e4bb4hb97980d7e10d8818@mail.gmail.com>
	<a3ea99f80809030231w1549872ale4675fc3490b533b@mail.gmail.com>
	<693bc9ab0809030244i10c50ae0t8bd1dc18da4a90d1@mail.gmail.com>
Message-ID: <aac2c7cb0809031115qfa522c9n85b7f0ade5134406@mail.gmail.com>

On Wed, Sep 3, 2008 at 3:44 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Wed, Sep 3, 2008 at 11:31 AM, Severin <seob at gmx.ch> wrote:
>> safethread sound like something interesting. are there already any safthread
>> ideas in pypy?
>> In general, working on GILless ideas sounds more interesting than doing
>> something on multiprocess framworks.
>> It's a master thesis, that means there is room for experimental stuff, but
>> in the end half a year is quite short, so the room is limited again :)
>> was there any attempt to remove the GIL in pypy so far?
>> --
>> Severin
>>
>
> None that I know of at least. I think any serious GIL-removal proposal
> requires some deeper thinking about what exactly are the expected
> semantics.

Oi, how did I miss this thread before...

Python-safethread's design of course includes most of this deep thinking.


-- 
Adam Olsen, aka Rhamphoryncus


From nvetoshkin at naumen.ru  Wed Sep  3 21:43:31 2008
From: nvetoshkin at naumen.ru (Vetoshkin Nikita)
Date: Thu, 04 Sep 2008 01:43:31 +0600
Subject: [pypy-dev] translate.py failes
In-Reply-To: <20080829155511.GA9583@code0.codespeak.net>
References: <48B6C0B6.6060802@naumen.ru>
	<20080828153110.GA8912@code0.codespeak.net>
	<48B6D3AB.4050801@naumen.ru>
	<20080829155511.GA9583@code0.codespeak.net>
Message-ID: <48BEE8E3.6010503@naumen.ru>

Playing with py.execnet failes too. Full traceback attached.

P.S. Seens like a hard start with my trials in PyPy =)

Armin Rigo wrote:
> Hi,
>
> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>> Is there a direction, where I could digg to help? Something to begin with.
>> I don't have enough experience but I'd like to take part (even very
>> small) in this project.
>
> Sure.  Do you understand the issue?  It's related to the way a Unix
> process starts a new subprocess: os.fork() followed by os.execv() in the
> child process.  The issue is that os.fork() fails if the parent process'
> total memory is more than half of your RAM+swap, because it tries to
> create a copy of the process, and the copy would also need more than
> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
> at all, but creates "shared pages" of memory so that memory pages are
> duplicated only when one of the two processes really modifies it; but
> still os.fork() has to mark all the memory as *potentially* used, and
> raises MemoryError if there isn't enough.)
>
> A solution might be to split translate.py in two processes: translate.py
> would start a subprocess when it starts, and then do all the work
> (consuming a lot of RAM); but when it needs to start new processes, e.g.
> call gcc, it would instead ask the subprocess to start the new
> processes.  More precisely, the code in pypy.translator.tool.cbuild
> would stop using distutils directly, and instead ask the subprocess to
> use distutils.  It looks even easy to do with the help of a
> py.execnet.PopenGateway() created when translate.py starts (see the py
> lib documentation at http://codespeak.net/py/).
>
>
> A bientot,
>
> Armin

-- 
Nikita
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: execnet_fail.txt
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20080904/54587572/attachment.txt>

From pedronis at openend.se  Wed Sep  3 21:52:05 2008
From: pedronis at openend.se (Samuele Pedroni)
Date: Wed, 03 Sep 2008 21:52:05 +0200
Subject: [pypy-dev] meeting summary Re: Pypy IRC meeting fri 17:00 UTC on
 #pypy (freenode.net)
In-Reply-To: <48B51D17.9020201@openend.se>
References: <48B51D17.9020201@openend.se>
Message-ID: <48BEEAE5.1050606@openend.se>

Samuele Pedroni wrote:
> Hi all,
>
> since EuroPython there's been some thinking and planning about doing a 
> new release of PyPy 1.1 before the end of the year with a focus related 
> to the recent work on compatibility  with applications and CPython and 
> stability.
>
> We think that an important step in the process is to improve and 
> consolidate our testing infrastructure with the goal of making it easy 
> to spot regressions.
>
> After some people availability checking on #pypy we agreed on scheduling 
> a meeting:
>
> this Friday 29 Aug at 17:00 UTC (19:00 CEST)
> on #pypy IRC channel (freenode.net)
>   
I have checked in a brief summary of the meeting as:

http://codespeak.net/svn/pypy/extradoc/planning/1.1/testing_infrastructure.txt

regards.


From fijall at gmail.com  Wed Sep  3 21:52:37 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 3 Sep 2008 21:52:37 +0200
Subject: [pypy-dev] translate.py failes
In-Reply-To: <48BEE8E3.6010503@naumen.ru>
References: <48B6C0B6.6060802@naumen.ru>
	<20080828153110.GA8912@code0.codespeak.net>
	<48B6D3AB.4050801@naumen.ru>
	<20080829155511.GA9583@code0.codespeak.net>
	<48BEE8E3.6010503@naumen.ru>
Message-ID: <693bc9ab0809031252j57c4610bh77e37c5db1143359@mail.gmail.com>

I'm not completele sure what you're trying to do. Can you explain in a
bit more detail? Also feel free to drop by on IRC for a live
discussion.

cheers,
fijal

On Wed, Sep 3, 2008 at 9:43 PM, Vetoshkin Nikita <nvetoshkin at naumen.ru> wrote:
> Playing with py.execnet failes too. Full traceback attached.
>
> P.S. Seens like a hard start with my trials in PyPy =)
>
> Armin Rigo wrote:
>>
>> Hi,
>>
>> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>>>
>>> Is there a direction, where I could digg to help? Something to begin
>>> with.
>>> I don't have enough experience but I'd like to take part (even very
>>> small) in this project.
>>
>> Sure.  Do you understand the issue?  It's related to the way a Unix
>> process starts a new subprocess: os.fork() followed by os.execv() in the
>> child process.  The issue is that os.fork() fails if the parent process'
>> total memory is more than half of your RAM+swap, because it tries to
>> create a copy of the process, and the copy would also need more than
>> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
>> at all, but creates "shared pages" of memory so that memory pages are
>> duplicated only when one of the two processes really modifies it; but
>> still os.fork() has to mark all the memory as *potentially* used, and
>> raises MemoryError if there isn't enough.)
>>
>> A solution might be to split translate.py in two processes: translate.py
>> would start a subprocess when it starts, and then do all the work
>> (consuming a lot of RAM); but when it needs to start new processes, e.g.
>> call gcc, it would instead ask the subprocess to start the new
>> processes.  More precisely, the code in pypy.translator.tool.cbuild
>> would stop using distutils directly, and instead ask the subprocess to
>> use distutils.  It looks even easy to do with the help of a
>> py.execnet.PopenGateway() created when translate.py starts (see the py
>> lib documentation at http://codespeak.net/py/).
>>
>>
>> A bientot,
>>
>> Armin
>
> --
> Nikita
>
>>>>> gw = py.execnet.PopenGateway()
> faking <type '_socket.socket'>
> faking <class 'socket.timeout'>
> Traceback (most recent call last):
>  File "./pypy/bin/py.py", line 152, in <module>
>    sys.exit(main_(sys.argv))
>  File "./pypy/bin/py.py", line 138, in main_
>    con.interact(banner)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line
> 133, in interact
>    code.InteractiveConsole.interact(self, banner)
>  File "/usr/lib64/python2.5/code.py", line 239, in interact
>    more = self.push(line)
>  File "/usr/lib64/python2.5/code.py", line 261, in push
>    more = self.runsource(source, self.filename)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line
> 192, in runsource
>    main.run_toplevel(self.space, doit, verbose=self.verbose)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in
> run_toplevel
>    f()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line
> 184, in doit
>    code.exec_code(self.space, self.w_globals, self.w_globals)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in
> exec_code
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
> run
>    return self.execute_frame()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
> in execute_frame
>    executioncontext)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
> in dispatch
>    next_instr = self.handle_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
> in handle_bytecode
>    next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
> in dispatch_bytecode
>    res = getattr(self, methodname)(oparg, next_instr)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695,
> in LOAD_ATTR
>    w_value = f.space.getattr(w_obj, w_attributename)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669,
> in getattr
>    return DescrOperation.getattr(self, w_obj, w_name)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 134, in getattr
>    return space._handle_getattribute(w_descr, w_obj, w_name)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 147, in _handle_getattribute
>    return space.get_and_call_function(w_descr, w_obj, w_name)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 89, in get_and_call_function
>    return descr.funccall(w_obj, *args_w)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56,
> in funccall
>    return code.fastcall_2(self.space, self, args_w[0], args_w[1])
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in
> fastcall_2
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
> run
>    return self.execute_frame()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
> in execute_frame
>    executioncontext)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
> in dispatch
>    next_instr = self.handle_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
> in handle_bytecode
>    next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
> in dispatch_bytecode
>    res = getattr(self, methodname)(oparg, next_instr)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881,
> in CALL_FUNCTION
>    w_result = f.space.call_valuestack(w_function, nargs, f)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
> 736, in call_valuestack
>    return w_func.funccall_valuestack(nargs, frame)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81,
> in funccall_valuestack
>    frame.peekvalue(0))
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in
> fastcall_2
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
> run
>    return self.execute_frame()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
> in execute_frame
>    executioncontext)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
> in dispatch
>    next_instr = self.handle_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
> in handle_bytecode
>    next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
> in dispatch_bytecode
>    res = getattr(self, methodname)(oparg, next_instr)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881,
> in CALL_FUNCTION
>    w_result = f.space.call_valuestack(w_function, nargs, f)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
> 736, in call_valuestack
>    return w_func.funccall_valuestack(nargs, frame)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81,
> in funccall_valuestack
>    frame.peekvalue(0))
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in
> fastcall_2
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
> run
>    return self.execute_frame()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
> in execute_frame
>    executioncontext)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
> in dispatch
>    next_instr = self.handle_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
> in handle_bytecode
>    next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
> in dispatch_bytecode
>    res = getattr(self, methodname)(oparg, next_instr)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881,
> in CALL_FUNCTION
>    w_result = f.space.call_valuestack(w_function, nargs, f)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
> 736, in call_valuestack
>    return w_func.funccall_valuestack(nargs, frame)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88,
> in funccall_valuestack
>    frame.peekvalue(0))
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663,
> in fastcall_4
>    w_result = self.fastfunc_4(space, w1, w2, w3, w4)
>  File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in
> fastfunc_importhook_4
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 180, in importhook
>    w_mod = absolute_import(space, modulename, 0, w_fromlist,
> tentative=0)
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 192, in absolute_import
>    w_fromlist, tentative)
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 211, in _absolute_import
>    tentative=tentative)
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 272, in load_part
>    w(partname))
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 90, in try_import_mod
>    magic, timestamp, stream.readall())
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 499, in load_compiled_module
>    code_w.exec_code(space, w_dic, w_dic)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in
> exec_code
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
> run
>    return self.execute_frame()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
> in execute_frame
>    executioncontext)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
> in dispatch
>    next_instr = self.handle_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
> in handle_bytecode
>    next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
> in dispatch_bytecode
>    res = getattr(self, methodname)(oparg, next_instr)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760,
> in IMPORT_NAME
>    w_locals, w_fromlist)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
> 712, in call_function
>    return w_func.funccall(*args_w)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62,
> in funccall
>    args_w[1], args_w[2], args_w[3])
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663,
> in fastcall_4
>    w_result = self.fastfunc_4(space, w1, w2, w3, w4)
>  File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in
> fastfunc_importhook_4
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 180, in importhook
>    w_mod = absolute_import(space, modulename, 0, w_fromlist,
> tentative=0)
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 192, in absolute_import
>    w_fromlist, tentative)
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 211, in _absolute_import
>    tentative=tentative)
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 272, in load_part
>    w(partname))
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 90, in try_import_mod
>    magic, timestamp, stream.readall())
>  File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
> line 499, in load_compiled_module
>    code_w.exec_code(space, w_dic, w_dic)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in
> exec_code
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
> run
>    return self.execute_frame()
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
> in execute_frame
>    executioncontext)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
> in dispatch
>    next_instr = self.handle_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
> in handle_bytecode
>    next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
> in dispatch_bytecode
>    res = getattr(self, methodname)(oparg, next_instr)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769,
> in IMPORT_STAR
>    import_all_from(f.space, w_module, w_locals)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839,
> in appcaller
>    return space.call_args(w_func, args)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 98, in call_args
>    return w_obj.call_args(args)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37,
> in call_args
>    return self.code.funcrun(self, args)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483,
> in funcrun
>    return BuiltinCode.funcrun_obj(self, func, None, args)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491,
> in funcrun_obj
>    w_result = activation._run(space, scope_w)
>  File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in
> _run_UWS_ObjSpace_W_Root_W_Root
>  File
> "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py",
> line 59, in import_all_from
>    w_2 = space.getattr(w_module, gs___dict__)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669,
> in getattr
>    return DescrOperation.getattr(self, w_obj, w_name)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 134, in getattr
>    return space._handle_getattribute(w_descr, w_obj, w_name)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 140, in _handle_getattribute
>    return space.get_and_call_function(w_descr, w_obj, w_name)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
> 89, in get_and_call_function
>    return descr.funccall(w_obj, *args_w)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69,
> in funccall
>    return self.call_args(Arguments(self.space, list(args_w)))
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37,
> in call_args
>    return self.code.funcrun(self, args)
>  File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in
> funcrun
>    return frame.run()
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in
> run
>    return self.space.wrap(result)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475,
> in wrap
>    items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()]
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522,
> in wrap
>    return fake_object(self, x)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in
> fake_object
>    ft = fake_type(x)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in
> fake_type
>    faked_type = really_build_fake_type(cpy_type)
>  File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in
> really_build_fake_type
>    assert cpy_type.__base__ is basestring
> AssertionError
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From nvetoshkin at naumen.ru  Wed Sep  3 21:59:39 2008
From: nvetoshkin at naumen.ru (Vetoshkin Nikita)
Date: Thu, 04 Sep 2008 01:59:39 +0600
Subject: [pypy-dev] translate.py failes
In-Reply-To: <693bc9ab0809031252j57c4610bh77e37c5db1143359@mail.gmail.com>
References: <48B6C0B6.6060802@naumen.ru>	
	<20080828153110.GA8912@code0.codespeak.net>	
	<48B6D3AB.4050801@naumen.ru>	
	<20080829155511.GA9583@code0.codespeak.net>	
	<48BEE8E3.6010503@naumen.ru>
	<693bc9ab0809031252j57c4610bh77e37c5db1143359@mail.gmail.com>
Message-ID: <48BEECAB.5020004@naumen.ru>

Oh, sorry.
Here are the details:
launching pypy:
./pypy/bin/py.py
then doing this:
import py
gw = py.execnet.PopenGateway()
and getting that long long traceback.
Machine:
Gentoo Linux x64
Python 2.5.2

Would be glad to provide any useful information.
I'm not very familiar with IRC, what is the server and channel to 
communicate on pypy related issues?


Maciej Fijalkowski wrote:
> I'm not completele sure what you're trying to do. Can you explain in a
> bit more detail? Also feel free to drop by on IRC for a live
> discussion.
>
> cheers,
> fijal
>
> On Wed, Sep 3, 2008 at 9:43 PM, Vetoshkin Nikita<nvetoshkin at naumen.ru>  wrote:
>> Playing with py.execnet failes too. Full traceback attached.
>>
>> P.S. Seens like a hard start with my trials in PyPy =)
>>
>> Armin Rigo wrote:
>>> Hi,
>>>
>>> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>>>> Is there a direction, where I could digg to help? Something to begin
>>>> with.
>>>> I don't have enough experience but I'd like to take part (even very
>>>> small) in this project.
>>> Sure.  Do you understand the issue?  It's related to the way a Unix
>>> process starts a new subprocess: os.fork() followed by os.execv() in the
>>> child process.  The issue is that os.fork() fails if the parent process'
>>> total memory is more than half of your RAM+swap, because it tries to
>>> create a copy of the process, and the copy would also need more than
>>> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
>>> at all, but creates "shared pages" of memory so that memory pages are
>>> duplicated only when one of the two processes really modifies it; but
>>> still os.fork() has to mark all the memory as *potentially* used, and
>>> raises MemoryError if there isn't enough.)
>>>
>>> A solution might be to split translate.py in two processes: translate.py
>>> would start a subprocess when it starts, and then do all the work
>>> (consuming a lot of RAM); but when it needs to start new processes, e.g.
>>> call gcc, it would instead ask the subprocess to start the new
>>> processes.  More precisely, the code in pypy.translator.tool.cbuild
>>> would stop using distutils directly, and instead ask the subprocess to
>>> use distutils.  It looks even easy to do with the help of a
>>> py.execnet.PopenGateway() created when translate.py starts (see the py
>>> lib documentation at http://codespeak.net/py/).
>>>
>>>
>>> A bientot,
>>>
>>> Armin
>> --
>> Nikita
>>
>>>>>> gw = py.execnet.PopenGateway()
>> faking<type '_socket.socket'>
>> faking<class 'socket.timeout'>
>> Traceback (most recent call last):
>>   File "./pypy/bin/py.py", line 152, in<module>
>>     sys.exit(main_(sys.argv))
>>   File "./pypy/bin/py.py", line 138, in main_
>>     con.interact(banner)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line
>> 133, in interact
>>     code.InteractiveConsole.interact(self, banner)
>>   File "/usr/lib64/python2.5/code.py", line 239, in interact
>>     more = self.push(line)
>>   File "/usr/lib64/python2.5/code.py", line 261, in push
>>     more = self.runsource(source, self.filename)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line
>> 192, in runsource
>>     main.run_toplevel(self.space, doit, verbose=self.verbose)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in
>> run_toplevel
>>     f()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line
>> 184, in doit
>>     code.exec_code(self.space, self.w_globals, self.w_globals)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in
>> exec_code
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
>> run
>>     return self.execute_frame()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
>> in execute_frame
>>     executioncontext)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
>> in dispatch
>>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
>> in handle_bytecode
>>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
>> in dispatch_bytecode
>>     res = getattr(self, methodname)(oparg, next_instr)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695,
>> in LOAD_ATTR
>>     w_value = f.space.getattr(w_obj, w_attributename)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669,
>> in getattr
>>     return DescrOperation.getattr(self, w_obj, w_name)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 134, in getattr
>>     return space._handle_getattribute(w_descr, w_obj, w_name)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 147, in _handle_getattribute
>>     return space.get_and_call_function(w_descr, w_obj, w_name)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 89, in get_and_call_function
>>     return descr.funccall(w_obj, *args_w)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56,
>> in funccall
>>     return code.fastcall_2(self.space, self, args_w[0], args_w[1])
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in
>> fastcall_2
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
>> run
>>     return self.execute_frame()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
>> in execute_frame
>>     executioncontext)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
>> in dispatch
>>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
>> in handle_bytecode
>>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
>> in dispatch_bytecode
>>     res = getattr(self, methodname)(oparg, next_instr)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881,
>> in CALL_FUNCTION
>>     w_result = f.space.call_valuestack(w_function, nargs, f)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
>> 736, in call_valuestack
>>     return w_func.funccall_valuestack(nargs, frame)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81,
>> in funccall_valuestack
>>     frame.peekvalue(0))
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in
>> fastcall_2
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
>> run
>>     return self.execute_frame()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
>> in execute_frame
>>     executioncontext)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
>> in dispatch
>>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
>> in handle_bytecode
>>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
>> in dispatch_bytecode
>>     res = getattr(self, methodname)(oparg, next_instr)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881,
>> in CALL_FUNCTION
>>     w_result = f.space.call_valuestack(w_function, nargs, f)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
>> 736, in call_valuestack
>>     return w_func.funccall_valuestack(nargs, frame)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81,
>> in funccall_valuestack
>>     frame.peekvalue(0))
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in
>> fastcall_2
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
>> run
>>     return self.execute_frame()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
>> in execute_frame
>>     executioncontext)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
>> in dispatch
>>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
>> in handle_bytecode
>>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
>> in dispatch_bytecode
>>     res = getattr(self, methodname)(oparg, next_instr)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881,
>> in CALL_FUNCTION
>>     w_result = f.space.call_valuestack(w_function, nargs, f)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
>> 736, in call_valuestack
>>     return w_func.funccall_valuestack(nargs, frame)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88,
>> in funccall_valuestack
>>     frame.peekvalue(0))
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663,
>> in fastcall_4
>>     w_result = self.fastfunc_4(space, w1, w2, w3, w4)
>>   File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in
>> fastfunc_importhook_4
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 180, in importhook
>>     w_mod = absolute_import(space, modulename, 0, w_fromlist,
>> tentative=0)
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 192, in absolute_import
>>     w_fromlist, tentative)
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 211, in _absolute_import
>>     tentative=tentative)
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 272, in load_part
>>     w(partname))
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 90, in try_import_mod
>>     magic, timestamp, stream.readall())
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 499, in load_compiled_module
>>     code_w.exec_code(space, w_dic, w_dic)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in
>> exec_code
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
>> run
>>     return self.execute_frame()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
>> in execute_frame
>>     executioncontext)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
>> in dispatch
>>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
>> in handle_bytecode
>>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
>> in dispatch_bytecode
>>     res = getattr(self, methodname)(oparg, next_instr)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760,
>> in IMPORT_NAME
>>     w_locals, w_fromlist)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line
>> 712, in call_function
>>     return w_func.funccall(*args_w)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62,
>> in funccall
>>     args_w[1], args_w[2], args_w[3])
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663,
>> in fastcall_4
>>     w_result = self.fastfunc_4(space, w1, w2, w3, w4)
>>   File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in
>> fastfunc_importhook_4
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 180, in importhook
>>     w_mod = absolute_import(space, modulename, 0, w_fromlist,
>> tentative=0)
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 192, in absolute_import
>>     w_fromlist, tentative)
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 211, in _absolute_import
>>     tentative=tentative)
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 272, in load_part
>>     w(partname))
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 90, in try_import_mod
>>     magic, timestamp, stream.readall())
>>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py",
>> line 499, in load_compiled_module
>>     code_w.exec_code(space, w_dic, w_dic)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in
>> exec_code
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in
>> run
>>     return self.execute_frame()
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117,
>> in execute_frame
>>     executioncontext)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79,
>> in dispatch
>>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89,
>> in handle_bytecode
>>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235,
>> in dispatch_bytecode
>>     res = getattr(self, methodname)(oparg, next_instr)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769,
>> in IMPORT_STAR
>>     import_all_from(f.space, w_module, w_locals)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839,
>> in appcaller
>>     return space.call_args(w_func, args)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 98, in call_args
>>     return w_obj.call_args(args)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37,
>> in call_args
>>     return self.code.funcrun(self, args)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483,
>> in funcrun
>>     return BuiltinCode.funcrun_obj(self, func, None, args)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491,
>> in funcrun_obj
>>     w_result = activation._run(space, scope_w)
>>   File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in
>> _run_UWS_ObjSpace_W_Root_W_Root
>>   File
>> "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py",
>> line 59, in import_all_from
>>     w_2 = space.getattr(w_module, gs___dict__)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669,
>> in getattr
>>     return DescrOperation.getattr(self, w_obj, w_name)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 134, in getattr
>>     return space._handle_getattribute(w_descr, w_obj, w_name)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 140, in _handle_getattribute
>>     return space.get_and_call_function(w_descr, w_obj, w_name)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line
>> 89, in get_and_call_function
>>     return descr.funccall(w_obj, *args_w)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69,
>> in funccall
>>     return self.call_args(Arguments(self.space, list(args_w)))
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37,
>> in call_args
>>     return self.code.funcrun(self, args)
>>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in
>> funcrun
>>     return frame.run()
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in
>> run
>>     return self.space.wrap(result)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475,
>> in wrap
>>     items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()]
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522,
>> in wrap
>>     return fake_object(self, x)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in
>> fake_object
>>     ft = fake_type(x)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in
>> fake_type
>>     faked_type = really_build_fake_type(cpy_type)
>>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in
>> really_build_fake_type
>>     assert cpy_type.__base__ is basestring
>> AssertionError
>>
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>

-- 
Nikita


From cfbolz at gmx.de  Wed Sep  3 22:01:24 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 03 Sep 2008 22:01:24 +0200
Subject: [pypy-dev] translate.py failes
In-Reply-To: <48BEE8E3.6010503@naumen.ru>
References: <48B6C0B6.6060802@naumen.ru>	<20080828153110.GA8912@code0.codespeak.net>	<48B6D3AB.4050801@naumen.ru>	<20080829155511.GA9583@code0.codespeak.net>
	<48BEE8E3.6010503@naumen.ru>
Message-ID: <48BEED14.6030507@gmx.de>

Vetoshkin Nikita wrote:
> Playing with py.execnet failes too. Full traceback attached.

It seems you are running py.execnet on py.py. Why? py.execnet is rather
independent on PyPy and works fine on CPython.

Cheers,

Carl Friedrich


From cfbolz at gmx.de  Wed Sep  3 22:29:14 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 03 Sep 2008 22:29:14 +0200
Subject: [pypy-dev] translate.py failes
In-Reply-To: <48BEEE73.7010809@naumen.ru>
References: <48B6C0B6.6060802@naumen.ru>	<20080828153110.GA8912@code0.codespeak.net>	<48B6D3AB.4050801@naumen.ru>	<20080829155511.GA9583@code0.codespeak.net>
	<48BEE8E3.6010503@naumen.ru> <48BEED14.6030507@gmx.de>
	<48BEEE73.7010809@naumen.ru>
Message-ID: <48BEF39A.9030406@gmx.de>

Vetoshkin Nikita wrote:
> Thanks!
> I thought it should work on top of PyPy doesn't it?

It probably should, but on py.py it is bound to be slow and I am not
sure it works that on the remote site a py.py is started correctly.
Anyway, if you work on py.execnet for helping the translation process
then you don't need to run it on py.py anyway, but on CPython.

Cheers,

Carl Friedrich


From calen.pennington at gmail.com  Thu Sep  4 04:21:53 2008
From: calen.pennington at gmail.com (Calen Pennington)
Date: Wed, 3 Sep 2008 22:21:53 -0400
Subject: [pypy-dev] Possible starting points
Message-ID: <2d702fee0809031921i66eb06d0t962a7fc2fcca5ded@mail.gmail.com>

Hi, everybody,

I've been intrigued by pypy for a while =, but haven't really been
able to find a good place to start contributing on my own, so I was
wondering if anyone had suggestions. As far as what I'm interested in,
I'd really like to see pypy be more accessible to mainstream python
users, and to be closer to an easy drop in replacement for CPython
(although it seems unlikely that it would become the standard python
distribution any time soon). I know c extensions are one blocker to
that, as well as feature completeness (although i don't really know
the details of where pypy stands with regards to 2.5 (or 2.6/3.0, for
that matter)).

That said, I'm also interested in language design/implementation in
general, so I wouldn't be averse to working on other parts of the
interpreter/translator chain either. So, any thoughts as to what
reasonable starting points would be?

Thanks

-Cale


From holger at merlinux.eu  Thu Sep  4 09:34:08 2008
From: holger at merlinux.eu (holger krekel)
Date: Thu, 4 Sep 2008 09:34:08 +0200
Subject: [pypy-dev] translate.py failes
In-Reply-To: <48BEE8E3.6010503@naumen.ru>
References: <48B6C0B6.6060802@naumen.ru>
	<20080828153110.GA8912@code0.codespeak.net>
	<48B6D3AB.4050801@naumen.ru>
	<20080829155511.GA9583@code0.codespeak.net>
	<48BEE8E3.6010503@naumen.ru>
Message-ID: <20080904073408.GZ9619@trillke.net>

Hi Vetoshkin, 

i think for translate.py you rather need to run py.execnet over
CPython or through a translated pypy-c - both of which 
should work fine.  

I think you ran it through py.py which means starting
a PyPy interpreter on top of CPython.  Then running
py.execnet.PopenGateway() would need to create another
process that also runs through py.py - would be very
slow - not sure it's worth trying to make it work.  

So i suggest you continue your experiments with 
plain CPython or the translated pypy-c.

best & thanks, 

holger

On Thu, Sep 04, 2008 at 01:43 +0600, Vetoshkin Nikita wrote:
> Playing with py.execnet failes too. Full traceback attached.
>
> P.S. Seens like a hard start with my trials in PyPy =)
>
> Armin Rigo wrote:
>> Hi,
>>
>> On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote:
>>> Is there a direction, where I could digg to help? Something to begin with.
>>> I don't have enough experience but I'd like to take part (even very
>>> small) in this project.
>>
>> Sure.  Do you understand the issue?  It's related to the way a Unix
>> process starts a new subprocess: os.fork() followed by os.execv() in the
>> child process.  The issue is that os.fork() fails if the parent process'
>> total memory is more than half of your RAM+swap, because it tries to
>> create a copy of the process, and the copy would also need more than
>> half of your RAM+swap.  (In reality, os.fork() doesn't copy the memory
>> at all, but creates "shared pages" of memory so that memory pages are
>> duplicated only when one of the two processes really modifies it; but
>> still os.fork() has to mark all the memory as *potentially* used, and
>> raises MemoryError if there isn't enough.)
>>
>> A solution might be to split translate.py in two processes: translate.py
>> would start a subprocess when it starts, and then do all the work
>> (consuming a lot of RAM); but when it needs to start new processes, e.g.
>> call gcc, it would instead ask the subprocess to start the new
>> processes.  More precisely, the code in pypy.translator.tool.cbuild
>> would stop using distutils directly, and instead ask the subprocess to
>> use distutils.  It looks even easy to do with the help of a
>> py.execnet.PopenGateway() created when translate.py starts (see the py
>> lib documentation at http://codespeak.net/py/).
>>
>>
>> A bientot,
>>
>> Armin
>
> -- 
> Nikita

> >>>> gw = py.execnet.PopenGateway()
> faking <type '_socket.socket'>
> faking <class 'socket.timeout'>
> Traceback (most recent call last):
>   File "./pypy/bin/py.py", line 152, in <module>
>     sys.exit(main_(sys.argv))
>   File "./pypy/bin/py.py", line 138, in main_
>     con.interact(banner)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 
> 133, in interact
>     code.InteractiveConsole.interact(self, banner)
>   File "/usr/lib64/python2.5/code.py", line 239, in interact
>     more = self.push(line)
>   File "/usr/lib64/python2.5/code.py", line 261, in push
>     more = self.runsource(source, self.filename)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 
> 192, in runsource
>     main.run_toplevel(self.space, doit, verbose=self.verbose)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/main.py", line 103, in 
> run_toplevel
>     f()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/interactive.py", line 
> 184, in doit
>     code.exec_code(self.space, self.w_globals, self.w_globals)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in 
> exec_code
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in 
> run
>     return self.execute_frame()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, 
> in execute_frame
>     executioncontext)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, 
> in dispatch
>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, 
> in handle_bytecode
>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, 
> in dispatch_bytecode
>     res = getattr(self, methodname)(oparg, next_instr)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 695, 
> in LOAD_ATTR
>     w_value = f.space.getattr(w_obj, w_attributename)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, 
> in getattr
>     return DescrOperation.getattr(self, w_obj, w_name)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 134, in getattr
>     return space._handle_getattribute(w_descr, w_obj, w_name)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 147, in _handle_getattribute
>     return space.get_and_call_function(w_descr, w_obj, w_name)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 89, in get_and_call_function
>     return descr.funccall(w_obj, *args_w)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 56, 
> in funccall
>     return code.fastcall_2(self.space, self, args_w[0], args_w[1])
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in 
> fastcall_2
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in 
> run
>     return self.execute_frame()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, 
> in execute_frame
>     executioncontext)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, 
> in dispatch
>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, 
> in handle_bytecode
>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, 
> in dispatch_bytecode
>     res = getattr(self, methodname)(oparg, next_instr)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, 
> in CALL_FUNCTION
>     w_result = f.space.call_valuestack(w_function, nargs, f)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 
> 736, in call_valuestack
>     return w_func.funccall_valuestack(nargs, frame)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, 
> in funccall_valuestack
>     frame.peekvalue(0))
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in 
> fastcall_2
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in 
> run
>     return self.execute_frame()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, 
> in execute_frame
>     executioncontext)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, 
> in dispatch
>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, 
> in handle_bytecode
>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, 
> in dispatch_bytecode
>     res = getattr(self, methodname)(oparg, next_instr)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, 
> in CALL_FUNCTION
>     w_result = f.space.call_valuestack(w_function, nargs, f)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 
> 736, in call_valuestack
>     return w_func.funccall_valuestack(nargs, frame)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 81, 
> in funccall_valuestack
>     frame.peekvalue(0))
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pycode.py", line 191, in 
> fastcall_2
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in 
> run
>     return self.execute_frame()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, 
> in execute_frame
>     executioncontext)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, 
> in dispatch
>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, 
> in handle_bytecode
>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, 
> in dispatch_bytecode
>     res = getattr(self, methodname)(oparg, next_instr)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 881, 
> in CALL_FUNCTION
>     w_result = f.space.call_valuestack(w_function, nargs, f)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 
> 736, in call_valuestack
>     return w_func.funccall_valuestack(nargs, frame)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 88, 
> in funccall_valuestack
>     frame.peekvalue(0))
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, 
> in fastcall_4
>     w_result = self.fastfunc_4(space, w1, w2, w3, w4)
>   File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in 
> fastfunc_importhook_4
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 180, in importhook
>     w_mod = absolute_import(space, modulename, 0, w_fromlist, 
> tentative=0)
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 192, in absolute_import
>     w_fromlist, tentative)
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 211, in _absolute_import
>     tentative=tentative)
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 272, in load_part
>     w(partname))
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 90, in try_import_mod
>     magic, timestamp, stream.readall())
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 499, in load_compiled_module
>     code_w.exec_code(space, w_dic, w_dic)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in 
> exec_code
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in 
> run
>     return self.execute_frame()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, 
> in execute_frame
>     executioncontext)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, 
> in dispatch
>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, 
> in handle_bytecode
>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, 
> in dispatch_bytecode
>     res = getattr(self, methodname)(oparg, next_instr)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 760, 
> in IMPORT_NAME
>     w_locals, w_fromlist)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/baseobjspace.py", line 
> 712, in call_function
>     return w_func.funccall(*args_w)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 62, 
> in funccall
>     args_w[1], args_w[2], args_w[3])
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 663, 
> in fastcall_4
>     w_result = self.fastfunc_4(space, w1, w2, w3, w4)
>   File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in 
> fastfunc_importhook_4
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 180, in importhook
>     w_mod = absolute_import(space, modulename, 0, w_fromlist, 
> tentative=0)
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 192, in absolute_import
>     w_fromlist, tentative)
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 211, in _absolute_import
>     tentative=tentative)
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 272, in load_part
>     w(partname))
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 90, in try_import_mod
>     magic, timestamp, stream.readall())
>   File "/var/tmp/pypy-svn/dist/pypy/module/__builtin__/importing.py", 
> line 499, in load_compiled_module
>     code_w.exec_code(space, w_dic, w_dic)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 27, in 
> exec_code
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 91, in 
> run
>     return self.execute_frame()
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyframe.py", line 117, 
> in execute_frame
>     executioncontext)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 79, 
> in dispatch
>     next_instr = self.handle_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 89, 
> in handle_bytecode
>     next_instr = self.dispatch_bytecode(co_code, next_instr, ec)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 235, 
> in dispatch_bytecode
>     res = getattr(self, methodname)(oparg, next_instr)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/pyopcode.py", line 769, 
> in IMPORT_STAR
>     import_all_from(f.space, w_module, w_locals)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 839, 
> in appcaller
>     return space.call_args(w_func, args)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 98, in call_args
>     return w_obj.call_args(args)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, 
> in call_args
>     return self.code.funcrun(self, args)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 483, 
> in funcrun
>     return BuiltinCode.funcrun_obj(self, func, None, args)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/gateway.py", line 491, 
> in funcrun_obj
>     w_result = activation._run(space, scope_w)
>   File "</var/tmp/pypy-svn/dist/py/code/source.py:213>", line 3, in 
> _run_UWS_ObjSpace_W_Root_W_Root
>   File 
> "/var/tmp/pypy-svn/dist/pypy/_cache/pyopcode_4922ecf55c7db5d81d0c4a63f92bdd44.py", 
> line 59, in import_all_from
>     w_2 = space.getattr(w_module, gs___dict__)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 669, 
> in getattr
>     return DescrOperation.getattr(self, w_obj, w_name)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 134, in getattr
>     return space._handle_getattribute(w_descr, w_obj, w_name)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 140, in _handle_getattribute
>     return space.get_and_call_function(w_descr, w_obj, w_name)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/descroperation.py", line 
> 89, in get_and_call_function
>     return descr.funccall(w_obj, *args_w)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 69, 
> in funccall
>     return self.call_args(Arguments(self.space, list(args_w)))
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/function.py", line 37, 
> in call_args
>     return self.code.funcrun(self, args)
>   File "/var/tmp/pypy-svn/dist/pypy/interpreter/eval.py", line 61, in 
> funcrun
>     return frame.run()
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 171, in 
> run
>     return self.space.wrap(result)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 475, 
> in wrap
>     items_w = [(self.wrap(k), self.wrap(v)) for (k, v) in x.iteritems()]
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/objspace.py", line 522, 
> in wrap
>     return fake_object(self, x)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 20, in 
> fake_object
>     ft = fake_type(x)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 58, in 
> fake_type
>     faked_type = really_build_fake_type(cpy_type)
>   File "/var/tmp/pypy-svn/dist/pypy/objspace/std/fake.py", line 108, in 
> really_build_fake_type
>     assert cpy_type.__base__ is basestring
> AssertionError

> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 


From cfbolz at gmx.de  Thu Sep  4 09:43:08 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 4 Sep 2008 09:43:08 +0200
Subject: [pypy-dev] translate.py failes
In-Reply-To: <20080904073408.GZ9619@trillke.net>
References: <48B6C0B6.6060802@naumen.ru>
	<20080828153110.GA8912@code0.codespeak.net>
	<48B6D3AB.4050801@naumen.ru>
	<20080829155511.GA9583@code0.codespeak.net>
	<48BEE8E3.6010503@naumen.ru> <20080904073408.GZ9619@trillke.net>
Message-ID: <348899050809040043l2cb67084i3dd5960a1e1d8c78@mail.gmail.com>

Hi Holger,

2008/9/4 holger krekel <holger at merlinux.eu>:
> i think for translate.py you rather need to run py.execnet over
> CPython or through a translated pypy-c - both of which
> should work fine.
>
> I think you ran it through py.py which means starting
> a PyPy interpreter on top of CPython.  Then running
> py.execnet.PopenGateway() would need to create another
> process that also runs through py.py - would be very
> slow - not sure it's worth trying to make it work.
>
> So i suggest you continue your experiments with
> plain CPython or the translated pypy-c.

Well, suggesting to use pypy-c isn't really helpful, since his
original problem was that PyPy doesn't translate :-).

Cheers,

Carl Friedrich


From holger at merlinux.eu  Thu Sep  4 09:54:47 2008
From: holger at merlinux.eu (holger krekel)
Date: Thu, 4 Sep 2008 09:54:47 +0200
Subject: [pypy-dev] translate.py failes
In-Reply-To: <348899050809040043l2cb67084i3dd5960a1e1d8c78@mail.gmail.com>
References: <48B6C0B6.6060802@naumen.ru>
	<20080828153110.GA8912@code0.codespeak.net>
	<48B6D3AB.4050801@naumen.ru>
	<20080829155511.GA9583@code0.codespeak.net>
	<48BEE8E3.6010503@naumen.ru> <20080904073408.GZ9619@trillke.net>
	<348899050809040043l2cb67084i3dd5960a1e1d8c78@mail.gmail.com>
Message-ID: <20080904075447.GA9619@trillke.net>

On Thu, Sep 04, 2008 at 09:43 +0200, Carl Friedrich Bolz wrote:
> Hi Holger,
> 
> 2008/9/4 holger krekel <holger at merlinux.eu>:
> > i think for translate.py you rather need to run py.execnet over
> > CPython or through a translated pypy-c - both of which
> > should work fine.
> >
> > I think you ran it through py.py which means starting
> > a PyPy interpreter on top of CPython.  Then running
> > py.execnet.PopenGateway() would need to create another
> > process that also runs through py.py - would be very
> > slow - not sure it's worth trying to make it work.
> >
> > So i suggest you continue your experiments with
> > plain CPython or the translated pypy-c.
> 
> Well, suggesting to use pypy-c isn't really helpful, since his
> original problem was that PyPy doesn't translate :-).

good point - actually i just wanted to make the point that the
translation tool chain (and py.execnet) should 
work fine on top of pypy-c, even if py.py fails. 

FWIW, I didn't see the other answers before 
i posted - using mutt over a high latency line
i no fun, btw. 
holger


From arigo at tunes.org  Thu Sep  4 14:23:43 2008
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 4 Sep 2008 14:23:43 +0200
Subject: [pypy-dev] master thesis with pypy
In-Reply-To: <aac2c7cb0809031115qfa522c9n85b7f0ade5134406@mail.gmail.com>
References: <a3ea99f80808260835g71de645q3152c6c0c0a458de@mail.gmail.com>
	<5b0248170808260921j7c2e4bb4hb97980d7e10d8818@mail.gmail.com>
	<a3ea99f80809030231w1549872ale4675fc3490b533b@mail.gmail.com>
	<693bc9ab0809030244i10c50ae0t8bd1dc18da4a90d1@mail.gmail.com>
	<aac2c7cb0809031115qfa522c9n85b7f0ade5134406@mail.gmail.com>
Message-ID: <20080904122343.GA26189@code0.codespeak.net>

Hi,

On Wed, Sep 03, 2008 at 12:15:37PM -0600, Adam Olsen wrote:
> Python-safethread's design of course includes most of this deep thinking.

Jython includes another possible outcome of deep thinking about this.
In general there are various possible approaches.


A bientot,

Armin.


From cfbolz at gmx.de  Thu Sep  4 14:45:44 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 04 Sep 2008 14:45:44 +0200
Subject: [pypy-dev] Possible starting points
In-Reply-To: <2d702fee0809031921i66eb06d0t962a7fc2fcca5ded@mail.gmail.com>
References: <2d702fee0809031921i66eb06d0t962a7fc2fcca5ded@mail.gmail.com>
Message-ID: <48BFD878.6040509@gmx.de>

Hi and welcome!

Calen Pennington wrote:
> I've been intrigued by pypy for a while =, but haven't really been
> able to find a good place to start contributing on my own, so I was
> wondering if anyone had suggestions. As far as what I'm interested in,
> I'd really like to see pypy be more accessible to mainstream python
> users, and to be closer to an easy drop in replacement for CPython
> (although it seems unlikely that it would become the standard python
> distribution any time soon). I know c extensions are one blocker to
> that, as well as feature completeness (although i don't really know
> the details of where pypy stands with regards to 2.5 (or 2.6/3.0, for
> that matter)).

2.5 should be mostly implemented, there are still a couple of bugs left
and the branch is still not merged, but we should get there soon.

Something that would be quite useful is to port some of the remaining
extension modules of CPython to be either pure Python or RPython (where
Python is usually preferred). Some of the easy candidates are csv, maybe
 curses (using ctypes).

[snip]

Cheers,

Carl Friedrich


From holger at merlinux.eu  Fri Sep  5 14:31:06 2008
From: holger at merlinux.eu (holger krekel)
Date: Fri, 5 Sep 2008 14:31:06 +0200
Subject: [pypy-dev] next pypy sprint 5-13th october
Message-ID: <20080905123105.GD9619@trillke.net>

========================================================
D??sseldorf PyPy sprint 5-13th October, 2008
========================================================

The PyPy team is heading for a new sprint meetup, this time 
aiming at a 1.1 release and to work on integrating PyPy better 
with small devices.   We are also open to other sprint topics!

The sprint is going to take place at the Computer Science
department of Heinrich-Heine Universit??t D??sseldorf (Germany).  

Infos in a nutshell: 

arrival day: Sunday, 5th October 

departure day: Monday, 13th October 

sprint days: 6-12th october with breaks 

Location: Seminar room of the computer science department, situated 
in the building 25.12 of the university campus, second
floor. For travel instructions see 
http://stups.cs.uni-duesseldorf.de/anreise/esbahn.php

------------------------------
Accomodation
------------------------------

We may be able to fit some people into private flats 
and otherwise there are several hotels, one recommended 
one is the http://www.hotel-hillesheim.de/ where we are
investigating regarding a cheaper group rate. 
Please register ASAP if you want to benefit. 

There are also some hotels that are used by visitors to the university and are
quite close to the uni. None of us have been at them, so we have no experience
of how good they are.

1. Hotel Bl??ttler
http://www.hotel-blaettler.com/

2. Hotel Mooren
http://www.hotel-haus-mooren.de/

3. Hotel Europa
can't seem to find a website

4. Schloss Mickeln
This one is the guest house of the university in a very beautiful castle.
Supposed to be extremely nice, but also expensive.
http://www.uni-duesseldorf.de/home/gaeste/schlossmickeln


-----------------------
Funding your travel 
-----------------------

We have a limited budget for funding travels.  If you want to
participate but need travel funding please register and send
a note to the pypy-sprint mailing list.  We'll see what we can
do to help.

------------
Registration
------------

If you'd like to come, please subscribe to the `pypy-sprint mailing list`_
and drop a note about your interests and post any questions.  More
organisational information will be sent to that list.

Please register by adding yourself on the following list (via svn):

  http://codespeak.net/svn/pypy/extradoc/sprintinfo/october-2008/people.txt

or announceyourself on the pypy-sprint mailing list if you do not 
yet have check-in rights:

  http://codespeak.net/mailman/listinfo/pypy-sprint

---------------------------------------
Preparation (if you feel it is needed):
---------------------------------------

* read the `getting-started`_ pages on http://codespeak.net/pypy

* for inspiration, overview and technical status you are welcome to
  read `the technical reports available and other relevant documentation`_

* please direct any technical and/or development oriented questions to
  pypy-dev at codespeak.net and any sprint organizing/logistical
  questions to pypy-sprint at codespeak.net

We are looking forward to meet you at the D??sseldorf PyPy sprint!

The PyPy team 

.. See also ..

.. _getting-started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html
.. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint
.. _`the technical reports available and other relevant documentation`: http://codespeak.net/pypy/dist/pypy/doc/index.html
.. _`EuroPython schedule`: http://europython.org/timetable

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 


From fijall at gmail.com  Fri Sep  5 15:25:19 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 5 Sep 2008 15:25:19 +0200
Subject: [pypy-dev] next pypy sprint 5-13th october
In-Reply-To: <20080905123105.GD9619@trillke.net>
References: <20080905123105.GD9619@trillke.net>
Message-ID: <693bc9ab0809050625m2008a9e0i51ae4b57102f8b90@mail.gmail.com>

> .. See also ..

> .. _`EuroPython schedule`: http://europython.org/timetable
      ^^^^^^^^^^^^^^^

This may be less necessary than one might expect :)


From holger at merlinux.eu  Wed Sep 10 13:48:59 2008
From: holger at merlinux.eu (holger krekel)
Date: Wed, 10 Sep 2008 13:48:59 +0200
Subject: [pypy-dev] build machines/doc directory
Message-ID: <20080910114859.GF9619@trillke.net>

Hi all, 

i've created a new doc directory to start documenting
build machines, test, benchmark environments. 
svnlocation is https://codespeak.net/svn/pypy/build/doc

I added an initial 'ssh_config' file which contains 
the hosts available to pypy developers.  Please help complete. 
I also wrote a small script in svn/py/trunk/contrib/sysinfo.py 
to quickly collect remote system information.  Combine the two
and run "sysinfo.py -f ssh_config" and you'll get an overview
of what we have (you can also use "sysinfo.py HOST1 HOST2 ..."
explicitely)

    BIGDOG2  fqdn: bigdog2
    BIGDOG2  sys.platform: linux2
    BIGDOG2  sys.version_info: (2, 5, 2, 'final', 0)
    BIGDOG2  Memory: 8191804 Swap: 5116660
    BIGDOG2  number of cpus: 4
    BIGDOG2  cpu model AMD Phenom(tm) 9950 Quad-Core Processor

    WYVERN  fqdn: wyvern.cs.uni-duesseldorf.de
    WYVERN  sys.platform: linux2
    WYVERN  sys.version_info: (2, 4, 4, 'final', 0)
    WYVERN  Memory: 5187968 Swap: 8032492
    WYVERN  number of cpus: 8
    WYVERN  cpu model Intel(R) Xeon(TM) CPU 3.20GHz

    COBRA  fqdn: cobra.cs.uni-duesseldorf.de
    COBRA  sys.platform: linux2
    COBRA  sys.version_info: (2, 4, 4, 'final', 0)
    COBRA  Memory: 2059980 Swap: 5148136
    COBRA  number of cpus: 8
    COBRA  cpu model Intel(R) Xeon(TM) CPU 3.00GHz

    TUATARA  fqdn: tuatara.cs.uni-duesseldorf.de
    TUATARA  sys.platform: darwin
    TUATARA  sys.version_info: (2, 3, 5, 'final', 0)

    T20  fqdn: jython01.
    T20  sys.platform: sunos5
    T20  sys.version_info: (2, 4, 4, 'final', 0)

Feel free to extend the sysinfo script - i guess having
more information about windows/OSX/Solaris platforms and about
available python versions / executables would be nice. 
Usually one can obtain this info through files or command line 
options so it shouldn't be hard to get this information 
automatically and add it to the sysinfo script. 

best, 
holger
-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 


From jacob at openend.se  Fri Sep 12 00:13:04 2008
From: jacob at openend.se (Jacob =?utf-8?q?Hall=C3=A9n?=)
Date: Fri, 12 Sep 2008 00:13:04 +0200
Subject: [pypy-dev] Switching to a distributed version control system
Message-ID: <200809120013.04799.jacob@openend.se>

I think that it would be a suitable point in time to switch to a new version 
control system right after the 1.1 release.

The first question to ask is of course why we should switch at all.

While the distributed version control systems allow a workflow where people 
maintan their own repositories and there is a designated role of integrator, 
I don't think we need such a workflow at this point in time. It may very well 
be the model to use in the future, when we have a production usable system, 
but right now this feature has no direct appeal.

The compelling reason to switch is in my opinion the superior support for 
branches that the DVCS's provide. Creating a branch is a very cheap operation 
and merging it to the trunk or whatever branch is far superior to what SVN 
provides. I think this feature would change the way we are working and 
improve our productivity by a significant factor.

There are a few other arguments in favour of a switch. People working through 
GPRS and off-line would have an easier time handling branches and updates. It 
would be possible to do sprints without a working internet access.

There are, in my opinion, 3 viable choices of DVCS for PyPy:

- git

- hg (mercurial)

- bzr

I think they would all be an improvement over SVN and they all have their 
strengths and weaknesses. In favour of bzr and hg is the fact that they are 
written in Python, with core parts in C. Git is all C. Git currently requires 
a cygwin environment to run on Windows, hg and bzr appear to have native 
windows versions. Git  is the fastest of the lot with hg in second place. Bzr 
is still a fair bit slower, though this is being worked on. Hg is really good 
at keeping the repositories small, with git in second place. Speaking for bzr 
is the fact that we have Michael Hudson in the PyPy community, and he seems 
to be a guru on bzr by now. Hg seems to be a little more tedious in its 
command set than the other two. Git used to be rather obscure, but is these 
days very straight forward to use. Git and bzr have very good visualization 
tools for showing the splitting and merging of branches. Git seems to be best 
at showing exactly what changed between 2 versions of the code (even 2 
versions that are not on the same subtree).

The strongest argument in favour of git seems to me to be the rebase feature, 
which allows one to make a branch for a new feature, work on the branch and 
then update the base of the branch to branch off at a later point in time. I 
haven't identified this feature in hg and bzr, but then I haven't read all 
the documentation in detail.

The one feature of svn that we would miss is the inclusion of foreign version 
controlled trees, like we do with the pylib tree. We would have to do this in 
a different way than before, since none of the systems have this feature. I'm 
not sure it makes sense to have the close svn coupling between the projects 
any more, in any case.

The effort of learning any of the systems seems to be quite insignificant. 
getting up to the level of svn is a matter of 15 minutes and learning the 
whole range of commands in a tool is not a big effort.

There is of course the hooks that send mail and blurb in IRC, but all 3 
systems seem to have at least as powerful hooks as svn.

Jacob


From micahel at gmail.com  Fri Sep 12 00:55:36 2008
From: micahel at gmail.com (Michael Hudson)
Date: Fri, 12 Sep 2008 10:55:36 +1200
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <200809120013.04799.jacob@openend.se>
References: <200809120013.04799.jacob@openend.se>
Message-ID: <aa0dfd860809111555w3ab32bf5t9e1730c12cac8f45@mail.gmail.com>

2008/9/12 Jacob Hall?n <jacob at openend.se>:
> I think that it would be a suitable point in time to switch to a new version
> control system right after the 1.1 release.
>
> The first question to ask is of course why we should switch at all.
>
> While the distributed version control systems allow a workflow where people
> maintan their own repositories and there is a designated role of integrator,
> I don't think we need such a workflow at this point in time. It may very well
> be the model to use in the future, when we have a production usable system,
> but right now this feature has no direct appeal.

I would imagine there would still be a 'trunk' branch somewhere that
would be where releases get made from and so on.

> The compelling reason to switch is in my opinion the superior support for
> branches that the DVCS's provide. Creating a branch is a very cheap operation
> and merging it to the trunk or whatever branch is far superior to what SVN
> provides. I think this feature would change the way we are working and
> improve our productivity by a significant factor.
>
> There are a few other arguments in favour of a switch. People working through
> GPRS and off-line would have an easier time handling branches and updates. It
> would be possible to do sprints without a working internet access.
>
> There are, in my opinion, 3 viable choices of DVCS for PyPy:
>
> - git
>
> - hg (mercurial)
>
> - bzr

I'd agree with this list, afaik, there's nothing else out there to
take seriously at present.  Darcs with it's cherry picking and lack of
enforcing a linear history would be interesting, but I'd be a bit wary
of using it seriously, tbh.

> I think they would all be an improvement over SVN and they all have their
> strengths and weaknesses. In favour of bzr and hg is the fact that they are
> written in Python, with core parts in C. Git is all C. Git currently requires
> a cygwin environment to run on Windows, hg and bzr appear to have native
> windows versions. Git  is the fastest of the lot with hg in second place. Bzr
> is still a fair bit slower, though this is being worked on.

I can say here that I work all day every day with bzr on a codebase
(Launchpad) that's actually a pretty similar size to pypy -- 5000
files in 300 directories, a few tens of thousands of revisions -- and
performance is mostly fine.

> Hg is really good
> at keeping the repositories small, with git in second place.

It's worth mentioning how very bad svn is here: svn keeps a separate
copy of every file in the working tree!  I think that a copy of the
tree containing the entire history with any of bzr, hg or git will
only be a little larger than a svn checkout.

> Speaking for bzr
> is the fact that we have Michael Hudson in the PyPy community, and he seems
> to be a guru on bzr by now.

Right, I'd be more than happy to "consult" on a bzr migration.

Another thing for bzr is that people can put branches up on Launchpad,
even if the main branches lived on codespeak and required ssh access
to commit too.

I don't want to pimp bzr too hard, as I actually have very little
experience with the other systems.

> Hg seems to be a little more tedious in its
> command set than the other two. Git used to be rather obscure, but is these
> days very straight forward to use. Git and bzr have very good visualization
> tools for showing the splitting and merging of branches. Git seems to be best
> at showing exactly what changed between 2 versions of the code (even 2
> versions that are not on the same subtree).

I'm not sure what you mean by this last point?

> The strongest argument in favour of git seems to me to be the rebase feature,
> which allows one to make a branch for a new feature, work on the branch and
> then update the base of the branch to branch off at a later point in time. I
> haven't identified this feature in hg and bzr, but then I haven't read all
> the documentation in detail.

This point is somewhat controversial in DVCS circles, and is viewed as
a non-feature in the Bazaar camp at least.  The issue is that if you
make a branch, then publish it, then rebase it onto a later version of
trunk, anyone who has based work on the earlier work is left high and
dry.

> The one feature of svn that we would miss is the inclusion of foreign version
> controlled trees, like we do with the pylib tree. We would have to do this in
> a different way than before, since none of the systems have this feature. I'm
> not sure it makes sense to have the close svn coupling between the projects
> any more, in any case.

Right.  bzr has some experimental support for 'nested trees', which
would replace this functionality, but realistically it's not going to
be ready for prime time any time soon.

> The effort of learning any of the systems seems to be quite insignificant.
> getting up to the level of svn is a matter of 15 minutes and learning the
> whole range of commands in a tool is not a big effort.

And I would say that bzr (and I suspect this applies to the others) is
actually a lot nicer to use than svn.  It seems to have a much more
'humane' interface, compared to svn's, which always makes me think of
barbed wire and pointy metal things for some reason.

> There is of course the hooks that send mail and blurb in IRC, but all 3
> systems seem to have at least as powerful hooks as svn.

Right.

There is also the issue of how to do a conversion.  codespeak's svn
repository is large and old and kind of scary in some quarters, and
the variety of ways that merges have been done in pypy's past means
that an import that preserves the merge information seems pretty
unlikely for any tool.  I'll try to get a bzr-svn import of pypy going
again, though.

Cheers,
mwh


From santagada at gmail.com  Fri Sep 12 01:23:10 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 11 Sep 2008 20:23:10 -0300
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <aa0dfd860809111555w3ab32bf5t9e1730c12cac8f45@mail.gmail.com>
References: <200809120013.04799.jacob@openend.se>
	<aa0dfd860809111555w3ab32bf5t9e1730c12cac8f45@mail.gmail.com>
Message-ID: <818F2865-B24E-4ECB-947E-D1BDB37B8949@gmail.com>


On Sep 11, 2008, at 7:55 PM, Michael Hudson wrote:

> 2008/9/12 Jacob Hall?n <jacob at openend.se>:

>> There are, in my opinion, 3 viable choices of DVCS for PyPy:
>>
>> - git
>>
>> - hg (mercurial)
>>
>> - bzr
>
> I'd agree with this list, afaik, there's nothing else out there to
> take seriously at present.  Darcs with it's cherry picking and lack of
> enforcing a linear history would be interesting, but I'd be a bit wary
> of using it seriously, tbh.

I think we could remove git for the discussion, as its lack of windows  
support (and no mac packaging besides ports) makes it a very high  
barrier of entry for now and future windows developers.

Isn't cfbolz also somewhat involved in bzr in the past?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20080911/8c65a885/attachment.html>

From sanxiyn at gmail.com  Fri Sep 12 01:56:05 2008
From: sanxiyn at gmail.com (Seo Sanghyeon)
Date: Fri, 12 Sep 2008 08:56:05 +0900
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <200809120013.04799.jacob@openend.se>
References: <200809120013.04799.jacob@openend.se>
Message-ID: <5b0248170809111656r43b24122t83d8ecaf4f1655d4@mail.gmail.com>

2008/9/12 Jacob Hall?n <jacob at openend.se>:
> The one feature of svn that we would miss is the inclusion of foreign version
> controlled trees, like we do with the pylib tree. We would have to do this in
> a different way than before, since none of the systems have this feature. I'm
> not sure it makes sense to have the close svn coupling between the projects
> any more, in any case.

Mercurial has "forest" extension which allows nested Mercurial repositories.
http://www.selenic.com/mercurial/wiki/index.cgi/ForestExtension

It is an external extension, but it is used by high profile projects like Sun's
OpenJDK.

-- 
Seo Sanghyeon

From ndbecker2 at gmail.com  Fri Sep 12 17:14:57 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 12 Sep 2008 11:14:57 -0400
Subject: [pypy-dev] Switching to a distributed version control system
References: <200809120013.04799.jacob@openend.se>
	<aa0dfd860809111555w3ab32bf5t9e1730c12cac8f45@mail.gmail.com>
	<818F2865-B24E-4ECB-947E-D1BDB37B8949@gmail.com>
Message-ID: <gae11h$ue0$1@ger.gmane.org>

hg is getting rebase, it was implemented under SOC, IIRC.



From dirkjan at ochtman.nl  Sat Sep 13 14:53:28 2008
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sat, 13 Sep 2008 14:53:28 +0200
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <200809120013.04799.jacob@openend.se>
References: <200809120013.04799.jacob@openend.se>
Message-ID: <48CBB7C8.1000601@ochtman.nl>

Hi there,

I'm a Mercurial committer who also listens in on #pypy some time. I 
remember we had a discussion about this subject with Armin and Michael a 
few weeks ago, where it was stated that the conversion process would 
likely be scary and that SVN was working alright for this project...

Jacob Hall?n wrote:
 > to be a guru on bzr by now. Hg seems to be a little more tedious in 
its command set than the other two. Git used to be rather obscure, but 
is these

I don't know why hg seems tedious. hg's commands have been designed to 
be rather a lot like SVN where this is possible.

 > days very straight forward to use. Git and bzr have very good 
visualization tools for showing the splitting and merging of branches. 
Git seems to be best at showing exactly what changed between 2 versions 
of the code (even 2 versions that are not on the same subtree).

hg has also has fairly good visualization tools. We have a command-line 
graph in the form of the glog extension, and in the upcoming release we 
will feature a canvas-based graph in the included web interface:

http://hg.xavamedia.nl/mercurial/crew/graph/

In addition, there are a TortoiseHG tool for Windows which also provides 
some integration/GUI on Linux (PyGTK-based).

 > The strongest argument in favour of git seems to me to be the rebase 
feature, which allows one to make a branch for a new feature, work on 
the branch and then update the base of the branch to branch off at a 
later point in time. I haven't identified this feature in hg and bzr, 
but then I haven't read all the documentation in detail.

In the next release, hg will come with a rebase extension. hg also comes 
with the very powerful mq extension, which allows developers to keep 
versioned patch queues around, which can be detached from the repo and 
distributed separately (and which allows for easily updating, splitting 
and merging of in-development patches/changesets).

 > The one feature of svn that we would miss is the inclusion of foreign 
version controlled trees, like we do with the pylib tree. We would have 
to do this in a different way than before, since none of the systems 
have this feature. I'm not sure it makes sense to have the close svn 
coupling between the projects any more, in any case.

Mercurial has the forest extension, which does this for OpenJDK, and 
NetBeans, I think. In fact, I think Mercurial is of the DVCS's the one 
with the most large projects: OpenJDK, NetBeans and Mozilla all use hg.

 > There is of course the hooks that send mail and blurb in IRC, but all 
3 systems seem to have at least as powerful hooks as svn.

In addition, hg has a very easy-to-use extension model, allowing Python 
developers to easily extend the functionality available in the tool.

In fairness, Mercurial's support for branches is a little different than 
most people are used to; either people can use separate clones for 
branches (which use hardlinks, so they aren't too expensive), or 
changesets get committed to a branch name, but since history in hg is 
immutable (I think this is largely true for any DVCS, but hg can be a 
little more strict about it), this means branches cannot currently be 
deleted. Branches that have been merged back into another branch can be 
kept out of command output, though.

Anyway, I'd like to help out with the conversion and infrastructure, 
should PyPy chose hg.

Cheers,

Dirkjan


From arigo at tunes.org  Sat Sep 13 16:29:36 2008
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 13 Sep 2008 16:29:36 +0200
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <200809120013.04799.jacob@openend.se>
References: <200809120013.04799.jacob@openend.se>
Message-ID: <20080913142936.GA29969@code0.codespeak.net>

Hi,

On Fri, Sep 12, 2008 at 12:13:04AM +0200, Jacob Hall?n wrote:
> I think that it would be a suitable point in time to switch to a new version 
> control system right after the 1.1 release.

I haven't read this thread in detail, but I should quickly mention that
subversion works quite well enough from my point of view.

An obstacle to switching is the amount of integration efforts that was
done, mainly on codespeak, between the repository and various other
facilities.  It is ok to discuss alternatives, but we need to keep in
mind that someone will have to stand up and redo all that work on
codespeak.


A bientot,

Armin.


From exarkun at divmod.com  Sat Sep 13 17:03:57 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sat, 13 Sep 2008 11:03:57 -0400
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <20080913142936.GA29969@code0.codespeak.net>
Message-ID: <20080913150357.29191.554380514.divmod.quotient.25524@ohm>

On Sat, 13 Sep 2008 16:29:36 +0200, Armin Rigo <arigo at tunes.org> wrote:
>Hi,
>
>On Fri, Sep 12, 2008 at 12:13:04AM +0200, Jacob Hall?n wrote:
>> I think that it would be a suitable point in time to switch to a new version
>> control system right after the 1.1 release.
>
>I haven't read this thread in detail, but I should quickly mention that
>subversion works quite well enough from my point of view.
>
>An obstacle to switching is the amount of integration efforts that was
>done, mainly on codespeak, between the repository and various other
>facilities.  It is ok to discuss alternatives, but we need to keep in
>mind that someone will have to stand up and redo all that work on
>codespeak.
>

In order to try to simplify discussions about changing tools, Twisted
has tried to start building up a centralized list of the important
features we use (still something of a work in progress):

  http://twistedmatrix.com/trac/wiki/WorkflowRequirements

If someone writes up something similar for PyPy then it might be easier
to talk about what tools can be replaced, what the consequences are, etc.

Jean-Paul


From sdouche at gmail.com  Sat Sep 13 21:32:06 2008
From: sdouche at gmail.com (Sebastien Douche)
Date: Sat, 13 Sep 2008 21:32:06 +0200
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <200809120013.04799.jacob@openend.se>
References: <200809120013.04799.jacob@openend.se>
Message-ID: <5e1183fa0809131232h7ce094ebrbfa64d82e24c3dc4@mail.gmail.com>

On Fri, Sep 12, 2008 at 00:13, Jacob Hall?n <jacob at openend.se> wrote:
> There are, in my opinion, 3 viable choices of DVCS for PyPy:
>
> - git
>
> - hg (mercurial)
>
> - bzr

Hi there :)

> I think they would all be an improvement over SVN and they all have their
> strengths and weaknesses. In favour of bzr and hg is the fact that they are
> written in Python, with core parts in C. Git is all C. Git currently requires
> a cygwin environment to run on Windows, hg and bzr appear to have native
> windows versions. Git  is the fastest of the lot with hg in second place. Bzr
> is still a fair bit slower, though this is being worked on. Hg is really good
> at keeping the repositories small, with git in second place.

Excellent quick guide : http://www.infoq.com/articles/dvcs-guide

> The strongest argument in favour of git seems to me to be the rebase feature,
> which allows one to make a branch for a new feature, work on the branch and
> then update the base of the branch to branch off at a later point in time. I
> haven't identified this feature in hg and bzr, but then I haven't read all
> the documentation in detail.

Don't miss "extensions" (but these are officials, include in core):

- Bisec
http://www.selenic.com/mercurial/wiki/index.cgi/BisectExtension

- Graph log (show grah of the tree, "ascii art")
http://www.selenic.com/mercurial/wiki/index.cgi/GraphlogExtension

- Hgk (same above, but with tcl/tk, copy of git's one)
http://www.selenic.com/mercurial/wiki/index.cgi/HgkExtension

- Mq (wonderfull Hg version of "quilt', must have!)
http://www.selenic.com/mercurial/wiki/index.cgi/MqExtension

-  Rebase (warning: available only to the next stable version)
http://www.selenic.com/mercurial/wiki/index.cgi/RebaseExtension

- Transplant ("transplant" patches from another branch or repositor)
http://www.selenic.com/mercurial/wiki/index.cgi/TransplantExtension

> The one feature of svn that we would miss is the inclusion of foreign version
> controlled trees, like we do with the pylib tree.

Do you means externals? The features provided by externals in
Subversion are not currently available in Mercurial,
NestedRepositories is a proposal to integrate this extension into
Mercurial:
http://www.selenic.com/mercurial/wiki/index.cgi/NestedRepositories



-- 
Seb


From micahel at gmail.com  Sun Sep 14 13:07:16 2008
From: micahel at gmail.com (Michael Hudson)
Date: Sun, 14 Sep 2008 23:07:16 +1200
Subject: [pypy-dev] please read my OSDC paper!
Message-ID: <aa0dfd860809140407i4f92e73bjc86ab6873b85df70@mail.gmail.com>

Hi,

As some of you know, I'm speaking about PyPy and OSDC in Sydney in December.

As the deadline for submitting the associated paper is tomorrow, it
seems like time to get some feedback :)

You can see the paper (in reST format) here:

http://codespeak.net/pypy/extradoc/talk/osdc2008/paper.txt

If anyone has any comments, I'd be grateful for feedback.  I'm going
to be asleep for most of the time between now on the deadline, so
please either commit comments into the file or reply to this mail and
I'll take them into account before submitting tomorrow.

Cheers,
mwh


From pedronis at openend.se  Thu Sep 18 14:23:57 2008
From: pedronis at openend.se (Samuele Pedroni)
Date: Thu, 18 Sep 2008 14:23:57 +0200
Subject: [pypy-dev] work on testing infrastructure using py.lib trunk and
	buildbot
In-Reply-To: <48BEEAE5.1050606@openend.se>
References: <48B51D17.9020201@openend.se> <48BEEAE5.1050606@openend.se>
Message-ID: <48D2485D.2090607@openend.se>

Hi all,

Samuele Pedroni wrote:
> I have checked in a brief summary of the meeting as:
>
> http://codespeak.net/svn/pypy/extradoc/planning/1.1/testing_infrastructure.txt
>
>   
some work has been started to implement the strategy described in the 
summary to improve our testing infrastructure and consolidate it using 
buildbot for all our testing needs.

The experimental buildbot master setup we are playing with lives in:

http://codespeak.net/svn/pypy/build/bot2/

this setup is going to be used for a  buildmaster running on wyvern.

As described in the summary the main idea is to use the new event model 
on py.lib trunk to produce structured logfiles that can be used from the 
buildbot master to produce consolidated test result pages. Code doing 
that lives at:

- http://codespeak.net/svn/py/trunk/contrib/filelog/

The integration between pypy and py.lib trunk py.test has been worked on in:

-  http://codespeak.net/svn/pypy/branch/pypy-pytrunk/ (this points 
through an external to the filelog code for it to be readily available)

so far only the major issues have been tackled and we are still using 
the compatibility layer
present in the py.test, medium term we want to switch away from using 
the deprecated
methods.

On the buildbot side:

- there's the start of a status page that looks like the current one we 
use (the htmlconftest based one) but can merge results coming from 
multiple builders

- and definitions of simple builds to run some of pypy own tests

I have checked in a TODO file with what is likely needed for people 
interested in helping or following what is going on:

http://codespeak.net/svn/pypy/build/bot2/TODO

Samuele.


From ondrej at certik.cz  Fri Sep 19 16:32:38 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Fri, 19 Sep 2008 16:32:38 +0200
Subject: [pypy-dev] Switching to a distributed version control system
In-Reply-To: <5e1183fa0809131232h7ce094ebrbfa64d82e24c3dc4@mail.gmail.com>
References: <200809120013.04799.jacob@openend.se>
	<5e1183fa0809131232h7ce094ebrbfa64d82e24c3dc4@mail.gmail.com>
Message-ID: <85b5c3130809190732y1d80a11s3ab1d893650a704e@mail.gmail.com>

On Sat, Sep 13, 2008 at 9:32 PM, Sebastien Douche <sdouche at gmail.com> wrote:
> On Fri, Sep 12, 2008 at 00:13, Jacob Hall?n <jacob at openend.se> wrote:
>> There are, in my opinion, 3 viable choices of DVCS for PyPy:
>>
>> - git
>>
>> - hg (mercurial)
>>
>> - bzr
>
> Hi there :)
>
>> I think they would all be an improvement over SVN and they all have their
>> strengths and weaknesses. In favour of bzr and hg is the fact that they are
>> written in Python, with core parts in C. Git is all C. Git currently requires
>> a cygwin environment to run on Windows, hg and bzr appear to have native
>> windows versions. Git  is the fastest of the lot with hg in second place. Bzr
>> is still a fair bit slower, though this is being worked on. Hg is really good
>> at keeping the repositories small, with git in second place.
>
> Excellent quick guide : http://www.infoq.com/articles/dvcs-guide
>
>> The strongest argument in favour of git seems to me to be the rebase feature,
>> which allows one to make a branch for a new feature, work on the branch and
>> then update the base of the branch to branch off at a later point in time. I
>> haven't identified this feature in hg and bzr, but then I haven't read all
>> the documentation in detail.
>
> Don't miss "extensions" (but these are officials, include in core):
>
> - Bisec
> http://www.selenic.com/mercurial/wiki/index.cgi/BisectExtension
>
> - Graph log (show grah of the tree, "ascii art")
> http://www.selenic.com/mercurial/wiki/index.cgi/GraphlogExtension
>
> - Hgk (same above, but with tcl/tk, copy of git's one)
> http://www.selenic.com/mercurial/wiki/index.cgi/HgkExtension
>
> - Mq (wonderfull Hg version of "quilt', must have!)
> http://www.selenic.com/mercurial/wiki/index.cgi/MqExtension
>
> -  Rebase (warning: available only to the next stable version)
> http://www.selenic.com/mercurial/wiki/index.cgi/RebaseExtension
>
> - Transplant ("transplant" patches from another branch or repositor)
> http://www.selenic.com/mercurial/wiki/index.cgi/TransplantExtension
>
>> The one feature of svn that we would miss is the inclusion of foreign version
>> controlled trees, like we do with the pylib tree.
>
> Do you means externals? The features provided by externals in
> Subversion are not currently available in Mercurial,
> NestedRepositories is a proposal to integrate this extension into
> Mercurial:
> http://www.selenic.com/mercurial/wiki/index.cgi/NestedRepositories


And here is a translation table to git if you later decide (like us:)
that git is better for you:

http://wiki.sympy.org/wiki/Git_hg_rosetta_stone

Also git has the excellent "git svn", so you can use git with pypy
right now and commit using svn (git svn dcommit).

Ondrej

From fijall at gmail.com  Mon Sep 22 17:48:46 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 22 Sep 2008 17:48:46 +0200
Subject: [pypy-dev] 2.5 and 2.4 compatibility
Message-ID: <693bc9ab0809220848s1131bd40i14b36e1016d255f8@mail.gmail.com>

I'm working on 2.5 compatibility branch right now. Due to incompatible
changes, I suggest that we say we don't provide --pyversion option any
more and simply compile 2.5 compatible interpreter. I don't really see
benefits of providing 2.4 right now (after a brief discussion with
Armin).

If anyone objects, please do that now.

Cheers,
fijal