From me at manueljacob.de  Sun Jan  1 23:06:05 2017
From: me at manueljacob.de (Manuel Jacob)
Date: Mon, 02 Jan 2017 05:06:05 +0100
Subject: [pypy-dev] cannot use "id" in Rpython
In-Reply-To: <1128804479.3847063.1483121449862@mail.yahoo.com>
References: <1704861386.3369487.1483039114642.ref@mail.yahoo.com>
 <1704861386.3369487.1483039114642@mail.yahoo.com>
 <CAO41-mMhp0QQtS+ZdckAQhf0sOH6Q5gqKMGWxippiJA2ttuytw@mail.gmail.com>
 <1A8EE0A6-9316-45BF-BE10-66C80055B178@yahoo.com>
 <b9ee8e713e108b9757c6033eecc91a5d@manueljacob.de>
 <21737E83-B661-4961-9F9E-2A15B9D1F34F@yahoo.com>
 <1128804479.3847063.1483121449862@mail.yahoo.com>
Message-ID: <b2df1510b2d104e5dac2843adc3cfef8@manueljacob.de>

Hi Shubha,

I forgot to mention that the code line starting with "fptr_type = " is 
valid Python code, but not valid RPython code.  More details on this in 
the documentation [1].  You should be able to fix the error by moving 
the line to the module toplevel.

-Manuel

[1] http://rpython.readthedocs.io/en/latest/getting-started.html

On 2016-12-30 19:10, Shubha Ramani wrote:
> I don't know the signature of the function coming into execute_frame
> ahead of time. Therefore as you suggested Manuel,I am leaving the
> argument is an empty list []. This doesn't seem to work though. See
> the assertion error I'm getting below.All other examples of this
> technique I see being used are passing in specific argument types to
> define a succinctfunction signature, i.e.?in
> test_warmstate.py:?ENTER_JIT =
> lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Float, ? ? ? ? ? ? ?
> ? ? ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? lltype.Signed],
> lltype.Bool)) ?
> 
> The following causes an assertion error:
> ?fun_name = _safe(pycode.co_name) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
> ? ? ? ? ? ? ? ? ? ??fptr_type = lltype.Ptr(lltype.FuncType([],
> lltype.Void)) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??fptr =
> llhelper(fptr_type, fun_name) ? ??
> [translation] start debugger...>
> /opt/shubha_vtune_pypy/rpython/rtyper/lltypesystem/lltype.py(776)constPtr()->
> assert T.is_constant()
> 
> 
>     On Friday, December 30, 2016 8:59 AM, Shubha Ramani
> <shubharamani at yahoo.com> wrote:
> 
> 
>  This is very helpful. Thank you Manuel !
> I will try your suggestion.
> 
> Shubha
> 
>> On Dec 30, 2016, at 8:04 AM, Manuel Jacob <me at manueljacob.de> wrote:
>> 
>> The id of an object is not necessarily related in any way to the 
>> address of an object in Python.
>> 
>> In RPython, you can convert a function to a low-level function pointer 
>> with llhelper() from rpython.rtyper.annlowlevel.? You can call it like 
>> this:
>> 
>> fptr_type = lltype.Ptr(lltype.FuncType([], lltype.Void))
>> fptr = llhelper(fptr_type, func)
>> 
>> Depending on what you want to do with the function address exactly, 
>> you might need to cast the function pointer to a RPython "Address" 
>> with cast_ptr_to_adr() from rpython.rtyper.lltypesystem.llmemory.
>> 
>>> On 2016-12-30 14:42, Shubha Ramani via pypy-dev wrote:
>>> It's not a bad idea for me. I'm hooking up to a tool made by intel
>>> called "vtune" which requires the function name, function size and
>>> function address from the original python script code. I got the 
>>> first
>>> two from Python reflection (the way VmProf does it) but I need the
>>> third (original script function? address). Is __repr__ supported in
>>> Rpython ? The string returned by __repr__ contains the Python address
>>> at least in Python. If id() doesn't work but __repr__ does then I can
>>> use it in the overloaded execute_frame and that should be fine.
>>> Shubha
>>>> On Dec 29, 2016, at 4:29 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>> What exactly are you trying to do. Using id to get the address of a 
>>>> function is usually a bad idea...
>>>> --
>>>> Ryan (????)
>>>> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> 
>>>> everyone else
>>>> http://kirbyfan64.github.io/
>>>>> On Dec 29, 2016 1:19 PM, "Shubha Ramani via pypy-dev" 
>>>>> <pypy-dev at python.org> wrote:
>>>>> What can I use then ? I need the address of the function ?
>>>>> [translation:ERROR] Exception: cannot use id() in RPython; see 
>>>>> objectmodel.compute_xxx()
>>>>> Processing block:
>>>>> block at 19 is a <class 'rpython.flowspace.flowcontext.SpamBlock'>
>>>>> in (pypy.module._vtune.interp_vtune:28)_get_full_name
>>>>> containing the following operations:
>>>>> ? ? ? buf_0 = simple_call((type RStringIO))
>>>>> ? ? ? v943 = getattr(buf_0, ('write'))
>>>>> ? ? ? v944 = getattr(pycode_0, ('co_code'))
>>>>> ? ? ? v945 = simple_call(v943, v944)
>>>>> ? ? ? v946 = getattr(buf_0, ('getsize'))
>>>>> ? ? ? size_code_0 = simple_call(v946)
>>>>> ? ? ? v947 = getattr(pycode_0, ('co_name'))
>>>>> ? ? ? id_func_0 = id(v947)
>>>>> ? ? ? v948 = getattr(pycode_0, ('co_name'))
>>>>> ? ? ? v949 = simple_call((function _safe), v948)
>>>>> ? ? ? v950 = getattr(pycode_0, ('co_firstlineno'))
>>>>> ? ? ? v951 = getattr(pycode_0, ('co_filename'))
>>>>> ? ? ? v952 = simple_call((function _safe), v951)
>>>>> ? ? ? v953 = newtuple(v949, v950, v952, size_code_0, id_func_0)
>>>>> ? ? ? v954 = mod(('vtune:%s:%d:%s:%d:%d'), v953)
>>>>> --end--
>>>>> [translation] start debugger...
>>>>> > /opt/shubha_vtune_pypy/rpython/annotator/unaryop.py(188)id()
>>>>> -> raise Exception("cannot use id() in RPython; "
>>>>> (Pdb+)
>>>>> _______________________________________________
>>>>> pypy-dev mailing list
>>>>> pypy-dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev

From shubharamani at yahoo.com  Mon Jan  2 09:44:41 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Mon, 2 Jan 2017 06:44:41 -0800
Subject: [pypy-dev] cannot use "id" in Rpython
In-Reply-To: <b2df1510b2d104e5dac2843adc3cfef8@manueljacob.de>
References: <1704861386.3369487.1483039114642.ref@mail.yahoo.com>
 <1704861386.3369487.1483039114642@mail.yahoo.com>
 <CAO41-mMhp0QQtS+ZdckAQhf0sOH6Q5gqKMGWxippiJA2ttuytw@mail.gmail.com>
 <1A8EE0A6-9316-45BF-BE10-66C80055B178@yahoo.com>
 <b9ee8e713e108b9757c6033eecc91a5d@manueljacob.de>
 <21737E83-B661-4961-9F9E-2A15B9D1F34F@yahoo.com>
 <1128804479.3847063.1483121449862@mail.yahoo.com>
 <b2df1510b2d104e5dac2843adc3cfef8@manueljacob.de>
Message-ID: <7B7E7261-9541-4644-B48B-A5DA9B7F2CF2@yahoo.com>

Thank you Manuel. 

Sent from Shubha Ramani's iPhone 7

> On Jan 1, 2017, at 8:06 PM, Manuel Jacob <me at manueljacob.de> wrote:
> 
> Hi Shubha,
> 
> I forgot to mention that the code line starting with "fptr_type = " is valid Python code, but not valid RPython code.  More details on this in the documentation [1].  You should be able to fix the error by moving the line to the module toplevel.
> 
> -Manuel
> 
> [1] http://rpython.readthedocs.io/en/latest/getting-started.html
> 
>> On 2016-12-30 19:10, Shubha Ramani wrote:
>> I don't know the signature of the function coming into execute_frame
>> ahead of time. Therefore as you suggested Manuel,I am leaving the
>> argument is an empty list []. This doesn't seem to work though. See
>> the assertion error I'm getting below.All other examples of this
>> technique I see being used are passing in specific argument types to
>> define a succinctfunction signature, i.e. in
>> test_warmstate.py: ENTER_JIT =
>> lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Float,              
>>                                                  lltype.Signed],
>> lltype.Bool))  
>> The following causes an assertion error:
>>  fun_name = _safe(pycode.co_name)                                    
>>                     fptr_type = lltype.Ptr(lltype.FuncType([],
>> lltype.Void))                                 fptr =
>> llhelper(fptr_type, fun_name)     
>> [translation] start debugger...>
>> /opt/shubha_vtune_pypy/rpython/rtyper/lltypesystem/lltype.py(776)constPtr()->
>> assert T.is_constant()
>>    On Friday, December 30, 2016 8:59 AM, Shubha Ramani
>> <shubharamani at yahoo.com> wrote:
>> This is very helpful. Thank you Manuel !
>> I will try your suggestion.
>> Shubha
>>> On Dec 30, 2016, at 8:04 AM, Manuel Jacob <me at manueljacob.de> wrote:
>>> The id of an object is not necessarily related in any way to the address of an object in Python.
>>> In RPython, you can convert a function to a low-level function pointer with llhelper() from rpython.rtyper.annlowlevel.  You can call it like this:
>>> fptr_type = lltype.Ptr(lltype.FuncType([], lltype.Void))
>>> fptr = llhelper(fptr_type, func)
>>> Depending on what you want to do with the function address exactly, you might need to cast the function pointer to a RPython "Address" with cast_ptr_to_adr() from rpython.rtyper.lltypesystem.llmemory.
>>>> On 2016-12-30 14:42, Shubha Ramani via pypy-dev wrote:
>>>> It's not a bad idea for me. I'm hooking up to a tool made by intel
>>>> called "vtune" which requires the function name, function size and
>>>> function address from the original python script code. I got the first
>>>> two from Python reflection (the way VmProf does it) but I need the
>>>> third (original script function  address). Is __repr__ supported in
>>>> Rpython ? The string returned by __repr__ contains the Python address
>>>> at least in Python. If id() doesn't work but __repr__ does then I can
>>>> use it in the overloaded execute_frame and that should be fine.
>>>> Shubha
>>>>> On Dec 29, 2016, at 4:29 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>>> What exactly are you trying to do. Using id to get the address of a function is usually a bad idea...
>>>>> --
>>>>> Ryan (????)
>>>>> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
>>>>> http://kirbyfan64.github.io/
>>>>>> On Dec 29, 2016 1:19 PM, "Shubha Ramani via pypy-dev" <pypy-dev at python.org> wrote:
>>>>>> What can I use then ? I need the address of the function ?
>>>>>> [translation:ERROR] Exception: cannot use id() in RPython; see objectmodel.compute_xxx()
>>>>>> Processing block:
>>>>>> block at 19 is a <class 'rpython.flowspace.flowcontext.SpamBlock'>
>>>>>> in (pypy.module._vtune.interp_vtune:28)_get_full_name
>>>>>> containing the following operations:
>>>>>>       buf_0 = simple_call((type RStringIO))
>>>>>>       v943 = getattr(buf_0, ('write'))
>>>>>>       v944 = getattr(pycode_0, ('co_code'))
>>>>>>       v945 = simple_call(v943, v944)
>>>>>>       v946 = getattr(buf_0, ('getsize'))
>>>>>>       size_code_0 = simple_call(v946)
>>>>>>       v947 = getattr(pycode_0, ('co_name'))
>>>>>>       id_func_0 = id(v947)
>>>>>>       v948 = getattr(pycode_0, ('co_name'))
>>>>>>       v949 = simple_call((function _safe), v948)
>>>>>>       v950 = getattr(pycode_0, ('co_firstlineno'))
>>>>>>       v951 = getattr(pycode_0, ('co_filename'))
>>>>>>       v952 = simple_call((function _safe), v951)
>>>>>>       v953 = newtuple(v949, v950, v952, size_code_0, id_func_0)
>>>>>>       v954 = mod(('vtune:%s:%d:%s:%d:%d'), v953)
>>>>>> --end--
>>>>>> [translation] start debugger...
>>>>>> > /opt/shubha_vtune_pypy/rpython/annotator/unaryop.py(188)id()
>>>>>> -> raise Exception("cannot use id() in RPython; "
>>>>>> (Pdb+)
>>>>>> _______________________________________________
>>>>>> pypy-dev mailing list
>>>>>> pypy-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>> _______________________________________________
>>>> pypy-dev mailing list
>>>> pypy-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pypy-dev


From shubharamani at yahoo.com  Mon Jan  2 09:47:11 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Mon, 2 Jan 2017 06:47:11 -0800
Subject: [pypy-dev] How do you pass in data from pypy interpreter to backend
 ?
Message-ID: <5D79E870-6D3F-4F3D-99E3-EF4BF013D5E9@yahoo.com>

I've really been struggling with this. For instance pycode from pyframe. I would like to pass along a data structure containing pycode tidbits all the way to the background but cannot figure out a way.

Any clues would be appreciated !

Shubha

From shubharamani at yahoo.com  Mon Jan  2 09:48:57 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Mon, 2 Jan 2017 06:48:57 -0800
Subject: [pypy-dev] How do you pass in data from pypy interpreter to
 backend ?
In-Reply-To: <5D79E870-6D3F-4F3D-99E3-EF4BF013D5E9@yahoo.com>
References: <5D79E870-6D3F-4F3D-99E3-EF4BF013D5E9@yahoo.com>
Message-ID: <24A487B5-20BD-4BC1-9E31-37E53CA7B614@yahoo.com>

Not "all the way to the Background. Typo. Meant to say all the way to the backend".
How do I pass pyframe pycode bits to the backend at runtime ? 

Sent from Shubha Ramani's iPhone 7

> On Jan 2, 2017, at 6:47 AM, Shubha Ramani <shubharamani at yahoo.com> wrote:
> 
> I've really been struggling with this. For instance pycode from pyframe. I would like to pass along a data structure containing pycode tidbits all the way to the background but cannot figure out a way.
> 
> Any clues would be appreciated !
> 
> Shubha


From shubharamani at yahoo.com  Wed Jan  4 14:33:20 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Wed, 4 Jan 2017 19:33:20 +0000 (UTC)
Subject: [pypy-dev] who calls handle_builtin_call of jtransform.py ?
References: <846803444.6207930.1483558400697.ref@mail.yahoo.com>
Message-ID: <846803444.6207930.1483558400697@mail.yahoo.com>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170104/53737199/attachment.html>

From armin.rigo at gmail.com  Wed Jan  4 16:59:59 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Wed, 4 Jan 2017 22:59:59 +0100
Subject: [pypy-dev] who calls handle_builtin_call of jtransform.py ?
In-Reply-To: <846803444.6207930.1483558400697@mail.yahoo.com>
References: <846803444.6207930.1483558400697.ref@mail.yahoo.com>
 <846803444.6207930.1483558400697@mail.yahoo.com>
Message-ID: <CAMSv6X2Tv0phOkviTckUohsoTakQ9U5N+Bgb6331U2KC8uG5+w@mail.gmail.com>

Hi Shubha,

Standard Python answer: add this line in the function:

    import pdb; pdb.set_trace()

Then run the tests (pytest.py rpython/jit/codewriter/).


A bient?t,

Armin.

From shubharamani at yahoo.com  Wed Jan  4 17:29:04 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Wed, 4 Jan 2017 22:29:04 +0000 (UTC)
Subject: [pypy-dev] How do you pass in data from pypy interpreter to
 backend ?
In-Reply-To: <24A487B5-20BD-4BC1-9E31-37E53CA7B614@yahoo.com>
References: <5D79E870-6D3F-4F3D-99E3-EF4BF013D5E9@yahoo.com>
 <24A487B5-20BD-4BC1-9E31-37E53CA7B614@yahoo.com>
Message-ID: <341189045.27719.1483568944053@mail.yahoo.com>

I figured this out - thanks to something Manuel Jacob said -"You should be able to fix the error by moving?the line to the module toplevel."
You can add a global variable to a module top level easily and pass it along to the backend.
Thank You Manuel !
Shubha 

    On Monday, January 2, 2017 6:48 AM, Shubha Ramani <shubharamani at yahoo.com> wrote:
 

 Not "all the way to the Background. Typo. Meant to say all the way to the backend".
How do I pass pyframe pycode bits to the backend at runtime ? 

Sent from Shubha Ramani's iPhone 7

> On Jan 2, 2017, at 6:47 AM, Shubha Ramani <shubharamani at yahoo.com> wrote:
> 
> I've really been struggling with this. For instance pycode from pyframe. I would like to pass along a data structure containing pycode tidbits all the way to the background but cannot figure out a way.
> 
> Any clues would be appreciated !
> 
> Shubha

   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170104/6bd9aa12/attachment.html>

From shubharamani at yahoo.com  Wed Jan  4 17:30:42 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Wed, 4 Jan 2017 22:30:42 +0000 (UTC)
Subject: [pypy-dev] cannot use "id" in Rpython
In-Reply-To: <7B7E7261-9541-4644-B48B-A5DA9B7F2CF2@yahoo.com>
References: <1704861386.3369487.1483039114642.ref@mail.yahoo.com>
 <1704861386.3369487.1483039114642@mail.yahoo.com>
 <CAO41-mMhp0QQtS+ZdckAQhf0sOH6Q5gqKMGWxippiJA2ttuytw@mail.gmail.com>
 <1A8EE0A6-9316-45BF-BE10-66C80055B178@yahoo.com>
 <b9ee8e713e108b9757c6033eecc91a5d@manueljacob.de>
 <21737E83-B661-4961-9F9E-2A15B9D1F34F@yahoo.com>
 <1128804479.3847063.1483121449862@mail.yahoo.com>
 <b2df1510b2d104e5dac2843adc3cfef8@manueljacob.de>
 <7B7E7261-9541-4644-B48B-A5DA9B7F2CF2@yahoo.com>
Message-ID: <1089495868.16909.1483569042491@mail.yahoo.com>

Thank you Manuel. Your answer here helped me in other ways - the thing about beingable to do things at the module top-level. That concept helped me a lot.
You were right. I don't need id(var). Never did. I mis-understood. The JITTED load addressis all I needed and Armin already provided that in his vtune branch.?
Shubha 

    On Monday, January 2, 2017 6:45 AM, Shubha Ramani via pypy-dev <pypy-dev at python.org> wrote:
 

 Thank you Manuel. 

Sent from Shubha Ramani's iPhone 7

> On Jan 1, 2017, at 8:06 PM, Manuel Jacob <me at manueljacob.de> wrote:
> 
> Hi Shubha,
> 
> I forgot to mention that the code line starting with "fptr_type = " is valid Python code, but not valid RPython code.? More details on this in the documentation [1].? You should be able to fix the error by moving the line to the module toplevel.
> 
> -Manuel
> 
> [1] http://rpython.readthedocs.io/en/latest/getting-started.html
> 
>> On 2016-12-30 19:10, Shubha Ramani wrote:
>> I don't know the signature of the function coming into execute_frame
>> ahead of time. Therefore as you suggested Manuel,I am leaving the
>> argument is an empty list []. This doesn't seem to work though. See
>> the assertion error I'm getting below.All other examples of this
>> technique I see being used are passing in specific argument types to
>> define a succinctfunction signature, i.e. in
>> test_warmstate.py: ENTER_JIT =
>> lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Float,? ? ? ? ? ? ? 
>>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? lltype.Signed],
>> lltype.Bool))? 
>> The following causes an assertion error:
>>? fun_name = _safe(pycode.co_name)? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
>>? ? ? ? ? ? ? ? ? ? fptr_type = lltype.Ptr(lltype.FuncType([],
>> lltype.Void))? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? fptr =
>> llhelper(fptr_type, fun_name)? ? 
>> [translation] start debugger...>
>> /opt/shubha_vtune_pypy/rpython/rtyper/lltypesystem/lltype.py(776)constPtr()->
>> assert T.is_constant()
>>? ? On Friday, December 30, 2016 8:59 AM, Shubha Ramani
>> <shubharamani at yahoo.com> wrote:
>> This is very helpful. Thank you Manuel !
>> I will try your suggestion.
>> Shubha
>>> On Dec 30, 2016, at 8:04 AM, Manuel Jacob <me at manueljacob.de> wrote:
>>> The id of an object is not necessarily related in any way to the address of an object in Python.
>>> In RPython, you can convert a function to a low-level function pointer with llhelper() from rpython.rtyper.annlowlevel.? You can call it like this:
>>> fptr_type = lltype.Ptr(lltype.FuncType([], lltype.Void))
>>> fptr = llhelper(fptr_type, func)
>>> Depending on what you want to do with the function address exactly, you might need to cast the function pointer to a RPython "Address" with cast_ptr_to_adr() from rpython.rtyper.lltypesystem.llmemory.
>>>> On 2016-12-30 14:42, Shubha Ramani via pypy-dev wrote:
>>>> It's not a bad idea for me. I'm hooking up to a tool made by intel
>>>> called "vtune" which requires the function name, function size and
>>>> function address from the original python script code. I got the first
>>>> two from Python reflection (the way VmProf does it) but I need the
>>>> third (original script function? address). Is __repr__ supported in
>>>> Rpython ? The string returned by __repr__ contains the Python address
>>>> at least in Python. If id() doesn't work but __repr__ does then I can
>>>> use it in the overloaded execute_frame and that should be fine.
>>>> Shubha
>>>>> On Dec 29, 2016, at 4:29 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>>> What exactly are you trying to do. Using id to get the address of a function is usually a bad idea...
>>>>> --
>>>>> Ryan (????)
>>>>> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
>>>>> http://kirbyfan64.github.io/
>>>>>> On Dec 29, 2016 1:19 PM, "Shubha Ramani via pypy-dev" <pypy-dev at python.org> wrote:
>>>>>> What can I use then ? I need the address of the function ?
>>>>>> [translation:ERROR] Exception: cannot use id() in RPython; see objectmodel.compute_xxx()
>>>>>> Processing block:
>>>>>> block at 19 is a <class 'rpython.flowspace.flowcontext.SpamBlock'>
>>>>>> in (pypy.module._vtune.interp_vtune:28)_get_full_name
>>>>>> containing the following operations:
>>>>>>? ? ? buf_0 = simple_call((type RStringIO))
>>>>>>? ? ? v943 = getattr(buf_0, ('write'))
>>>>>>? ? ? v944 = getattr(pycode_0, ('co_code'))
>>>>>>? ? ? v945 = simple_call(v943, v944)
>>>>>>? ? ? v946 = getattr(buf_0, ('getsize'))
>>>>>>? ? ? size_code_0 = simple_call(v946)
>>>>>>? ? ? v947 = getattr(pycode_0, ('co_name'))
>>>>>>? ? ? id_func_0 = id(v947)
>>>>>>? ? ? v948 = getattr(pycode_0, ('co_name'))
>>>>>>? ? ? v949 = simple_call((function _safe), v948)
>>>>>>? ? ? v950 = getattr(pycode_0, ('co_firstlineno'))
>>>>>>? ? ? v951 = getattr(pycode_0, ('co_filename'))
>>>>>>? ? ? v952 = simple_call((function _safe), v951)
>>>>>>? ? ? v953 = newtuple(v949, v950, v952, size_code_0, id_func_0)
>>>>>>? ? ? v954 = mod(('vtune:%s:%d:%s:%d:%d'), v953)
>>>>>> --end--
>>>>>> [translation] start debugger...
>>>>>> > /opt/shubha_vtune_pypy/rpython/annotator/unaryop.py(188)id()
>>>>>> -> raise Exception("cannot use id() in RPython; "
>>>>>> (Pdb+)
>>>>>> _______________________________________________
>>>>>> pypy-dev mailing list
>>>>>> pypy-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>> _______________________________________________
>>>> pypy-dev mailing list
>>>> pypy-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pypy-dev

_______________________________________________
pypy-dev mailing list
pypy-dev at python.org
https://mail.python.org/mailman/listinfo/pypy-dev


   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170104/a76fd088/attachment-0001.html>

From shubharamani at yahoo.com  Wed Jan  4 18:01:09 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Wed, 4 Jan 2017 23:01:09 +0000 (UTC)
Subject: [pypy-dev] who calls handle_builtin_call of jtransform.py ?
In-Reply-To: <CAMSv6X2Tv0phOkviTckUohsoTakQ9U5N+Bgb6331U2KC8uG5+w@mail.gmail.com>
References: <846803444.6207930.1483558400697.ref@mail.yahoo.com>
 <846803444.6207930.1483558400697@mail.yahoo.com>
 <CAMSv6X2Tv0phOkviTckUohsoTakQ9U5N+Bgb6331U2KC8uG5+w@mail.gmail.com>
Message-ID: <268737237.22301.1483570869186@mail.yahoo.com>

Indeed.
Shubha 

    On Wednesday, January 4, 2017 2:00 PM, Armin Rigo <armin.rigo at gmail.com> wrote:
 

 Hi Shubha,

Standard Python answer: add this line in the function:

? ? import pdb; pdb.set_trace()

Then run the tests (pytest.py rpython/jit/codewriter/).


A bient?t,

Armin.

   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170104/155ef49a/attachment.html>

From shubharamani at yahoo.com  Wed Jan  4 19:44:27 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Thu, 5 Jan 2017 00:44:27 +0000 (UTC)
Subject: [pypy-dev] how do we jump from pypy/interpreter to rpython ?
References: <46965882.74660.1483577067452.ref@mail.yahoo.com>
Message-ID: <46965882.74660.1483577067452@mail.yahoo.com>

I see space.call_function() but how exactly do we end up in the rpython directory of code ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170105/27b19788/attachment.html>

From shubharamani at yahoo.com  Fri Jan  6 11:42:56 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Fri, 6 Jan 2017 08:42:56 -0800
Subject: [pypy-dev] jitlog.MARK_TRACE_OPT in do_compile_bridge and
 do_compile_loop of compile.py
References: <2052619597.1149947.1483719160276.ref@mail.yahoo.com>
Message-ID: <0160205F-3EA1-455E-8AA4-35730BC0953B@yahoo.com>

I'm thinking that jitlog.MARK_TRACE_OPT is the only marker I need to keep track of
as I'm dumping pycode  (overloading execute_frame as vmprof does). 
When I see a MARK_TRACE_OPT, I just walk backward to find the closest pycode dump.
This will give me the actual source code lines that was JITTED.

Would this approach work ?

Thanks,

SHubha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170106/f6f9ae75/attachment.html>

From shubharamani at yahoo.com  Sun Jan  8 10:41:28 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Sun, 8 Jan 2017 15:41:28 +0000 (UTC)
Subject: [pypy-dev] Somebody has already provided code objects in
 assemble_loop of x86/ assembler.py !
References: <21927459.521295.1483890088191.ref@mail.yahoo.com>
Message-ID: <21927459.521295.1483890088191@mail.yahoo.com>

I just downloaded the pypy code. Please inspect the following log and search for 'code object'. This is the?
exact data which vtune needs ! How did this happen ? Was it there all along ? Did someone justadd this (perhaps Armin himself) ? The signature for 'assemble_loop' has not changed between the?vtune branch (last April) and now. I'm confused.
Also we need the same functionality in 'assemble_bridge'. Would it be hard to add loopname to'assemble_bridge' ? The reason I am assuming assemble_bridge needs the same info is becausein the vtune_branch the vtune API was called from assemble_loop and assemble_bridge.
def assemble_loop(self, jd_id, unique_id, logger, loopname, inputargs, ? ? ? ? ? ??? ? ? ? ? ? ? ? ? ? ? operations, looptoken, log):?
def assemble_bridge(self, faildescr, inputargs, operations, ? ? ? ? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ? original_loop_token, log, logger):?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170108/eda3c1ac/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pypy_log_containing_function_info.txt
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170108/eda3c1ac/attachment.txt>

From armin.rigo at gmail.com  Sun Jan  8 12:01:01 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sun, 8 Jan 2017 18:01:01 +0100
Subject: [pypy-dev] Somebody has already provided code objects in
 assemble_loop of x86/ assembler.py !
In-Reply-To: <21927459.521295.1483890088191@mail.yahoo.com>
References: <21927459.521295.1483890088191.ref@mail.yahoo.com>
 <21927459.521295.1483890088191@mail.yahoo.com>
Message-ID: <CAMSv6X1DKVP+hN6rSzvLQQ_P9ak6+zX=N5pS_RiDx5708BfO+w@mail.gmail.com>

Hi Shubha,

On 8 January 2017 at 16:41, Shubha Ramani via pypy-dev
<pypy-dev at python.org> wrote:
> I just downloaded the pypy code. Please inspect the following log and search
> for 'code object'. This is the exact data which vtune needs !
> How did this happen ? Was it there all along ?

Yes.  But:

> This is the exact data which vtune needs !

Not exactly.  This is only the outermost function.  It misses all
inlined Python functions.


Armin

From shubharamani at yahoo.com  Sun Jan  8 12:16:01 2017
From: shubharamani at yahoo.com (Shubha Ramani)
Date: Sun, 8 Jan 2017 09:16:01 -0800
Subject: [pypy-dev] Somebody has already provided code objects in
 assemble_loop of x86/ assembler.py !
In-Reply-To: <CAMSv6X1DKVP+hN6rSzvLQQ_P9ak6+zX=N5pS_RiDx5708BfO+w@mail.gmail.com>
References: <21927459.521295.1483890088191.ref@mail.yahoo.com>
 <21927459.521295.1483890088191@mail.yahoo.com>
 <CAMSv6X1DKVP+hN6rSzvLQQ_P9ak6+zX=N5pS_RiDx5708BfO+w@mail.gmail.com>
Message-ID: <DA9A2E93-E375-4FE7-9645-FF801615FF65@yahoo.com>

Got it. I now fully understand how vmprof is doing it. By manipulating the TLS (Thread Local Stack and inserting "kind" on the stack, where I care about the JITTED " kind). My struggle though is mapping rawaddress in assemble_loop and assemble_bridge to the addresses that pop up in vmprof profile.py. 

Any suggestions ?

Shubha 

> On Jan 8, 2017, at 9:01 AM, Armin Rigo <armin.rigo at gmail.com> wrote:
> 
> Hi Shubha,
> 
> On 8 January 2017 at 16:41, Shubha Ramani via pypy-dev
> <pypy-dev at python.org> wrote:
>> I just downloaded the pypy code. Please inspect the following log and search
>> for 'code object'. This is the exact data which vtune needs !
>> How did this happen ? Was it there all along ?
> 
> Yes.  But:
> 
>> This is the exact data which vtune needs !
> 
> Not exactly.  This is only the outermost function.  It misses all
> inlined Python functions.
> 
> 
> Armin


From frankw at mit.edu  Wed Jan 11 17:08:43 2017
From: frankw at mit.edu (Frank Wang)
Date: Wed, 11 Jan 2017 14:08:43 -0800
Subject: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
In-Reply-To: <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
References: <93774165-2ebb-492e-bc94-8366beefbd84@email.android.com>
 <03a439ec-71fa-454a-a9e0-4238d7505513@email.android.com>
 <CAD=7XjOzwMMc+6TBL31s4xh71CKG-MtA7LQhqdCxuSw4G_zJXA@mail.gmail.com>
 <CAD=7XjOLqAeuTqok0Pje18xFTVD1X8EjfjCY5Rf7kafxwEpnBw@mail.gmail.com>
 <CAMSv6X110t+1iyWvoOt_UfKXrZu0oprYScikrh4+gPMD8wBhBA@mail.gmail.com>
 <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
Message-ID: <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>

Hi Armin,

I've taken your advice, and my W_Root has these additions (in bold):

class W_Root(object):
    _attrs_ = ['__weakref__', *'rb_flags'*]
    _must_be_light_finalizer_ = True
    user_overridden_class = False
    # Riverbed flags
*    rb_flags = None*

*    def add_rbflags(self, new_flags):*
*        if self.rb_flags is None:*
*            self.rb_flags = {}*
*        for flag in new_flags:*
*            self.rb_flags[flag] = None*

*    def set_rbflags(self, new_flags):*
*        self.rb_flags = new_flags*

*    def get_rbflags(self):*
*        if self.rb_flags is None:*
*            self.rb_flags = {}*
*        return self.rb_flags*

In pyopcode.py, I've made the following modifications. I've shown them in
bold. The high level idea is that for a binary operation, I want to take
the union of the rb_flags in w_1 and w_2 and set the union to be the
rb_flags in w_result.

def binaryoperation(operationname):
    """NOT_RPYTHON"""
    def opimpl(self, *ignored):
        operation = getattr(self.space, operationname)
        w_2 = self.popvalue()
        w_1 = self.popvalue()
        w_result = operation(w_1, w_2)

        # union flags in w_1 and w_2 and propagate to result
*        w_1_rbflags = w_1.get_rbflags()*
*        w_2_rbflags = w_2.get_rbflags()*
*        w_1_rbflags.update(w_2_rbflags)*
*        w_result.set_rbflags(w_1_rbflags)*

        self.pushvalue(w_result)

    opimpl.binop = operationname

    return func_with_new_name(opimpl, "opcode_impl_for_%s" % operationname)

However, when I do this, I get the following error:

[translation:ERROR] UnionError:

Offending annotations:
  SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>)
  SomeInstance(can_be_None=True,
classdef=pypy.interpreter.baseobjspace.W_Root)


    v945 = setattr(self_268, ('rb_flags'), v944)

In <FunctionGraph of (pypy.interpreter.baseobjspace:46)W_Root.get_rbflags
at 0x40f26de8>:
Happened at file
/home/ubuntu/pypy2-v5.3.1-src/pypy/interpreter/baseobjspace.py line 48

==>             self.rb_flags = {}

Known variable annotations:
 self_268 = SomeInstance(can_be_None=False,
classdef=pypy.interpreter.baseobjspace.W_Root)
 v944 = SomeOrderedDict(dictdef=<{SomeImpossibleValue():
SomeImpossibleValue()}>)

Processing block:
 block at 15 is a <class 'rpython.flowspace.flowcontext.SpamBlock'>
 in (pypy.interpreter.baseobjspace:46)W_Root.get_rbflags
 containing the following operations:
       v944 = newdict()
       v945 = setattr(self_268, ('rb_flags'), v944)
 --end--

Any thoughts on this?

Frank

On Mon, Dec 19, 2016 at 10:32 PM, Frank Wang <frankw at mit.edu> wrote:

> Hi Armin,
>
> It seems that with your suggestions, things have started to work!
>
> Thanks again for the help!
>
> On Mon, Dec 19, 2016 at 5:57 PM Armin Rigo <armin.rigo at gmail.com> wrote:
>
>> Hi Frank,
>>
>> On 20 December 2016 at 01:29, Frank Wang <frankw at mit.edu> wrote:
>> > This also seems to happen in a lot of classes because W_Root is the
>> parent
>> > class for many classes. Is there a way to universally turn off this
>> > immutable flag (assuming it's okay) in all classes?
>>
>> You can hack at the place where the error is raised.  Assuming you're
>> translating without the JIT, it should not cause too much problems.
>>
>> As an alternative to adding an attribute to every object, you could
>> also use a rpython.rlib.rweakref.RWeakKeyDictionary(W_Root, RbFlags),
>> for some class RbFlags which has got the rb_flags dictionary and/or
>> any other related data.  The advantage is that it is a less intrusive
>> change, reduces memory usage if many objects don't need the extra
>> information at all, and avoids the immutable problem.  The
>> inconvenient is that going from the object to the rb_flags dictionary
>> is slower.
>>
>> There are also alternatives that are better if you are ok with
>> supporting not *all* objects.  For example, you can edit
>> objspace/std/mapdict.py to add a new dynamic attribute that is
>> internally called ``"rbflags", SPECIAL``; see ``"weakref", SPECIAL``
>> for an existing example.  This would work for all objects that use
>> mapdicts, i.e. all instances of user-defined classes, and additionally
>> instances of many built-in types as well---but not instances of the
>> core types like "int" or "str" or "list" or "tuple" or "dict".
>>
>>
>> A bient?t,
>>
>> Armin.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170111/ac2d260b/attachment.html>

From william.leslie.ttg at gmail.com  Wed Jan 11 18:00:36 2017
From: william.leslie.ttg at gmail.com (William ML Leslie)
Date: Thu, 12 Jan 2017 10:00:36 +1100
Subject: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
In-Reply-To: <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>
References: <93774165-2ebb-492e-bc94-8366beefbd84@email.android.com>
 <03a439ec-71fa-454a-a9e0-4238d7505513@email.android.com>
 <CAD=7XjOzwMMc+6TBL31s4xh71CKG-MtA7LQhqdCxuSw4G_zJXA@mail.gmail.com>
 <CAD=7XjOLqAeuTqok0Pje18xFTVD1X8EjfjCY5Rf7kafxwEpnBw@mail.gmail.com>
 <CAMSv6X110t+1iyWvoOt_UfKXrZu0oprYScikrh4+gPMD8wBhBA@mail.gmail.com>
 <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
 <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>
Message-ID: <CAHgd1hG4qhgphTtZm+9pX-Z+sXQ2rSQ_syQG+2qeAG9M_zNR3w@mail.gmail.com>

On 12 January 2017 at 09:08, Frank Wang <frankw at mit.edu> wrote:
> def binaryoperation(operationname):
>     """NOT_RPYTHON"""
>     def opimpl(self, *ignored):
>         operation = getattr(self.space, operationname)
>         w_2 = self.popvalue()
>         w_1 = self.popvalue()
>         w_result = operation(w_1, w_2)
>
>         # union flags in w_1 and w_2 and propagate to result
>         w_1_rbflags = w_1.get_rbflags()
>         w_2_rbflags = w_2.get_rbflags()
>         w_1_rbflags.update(w_2_rbflags)
>         w_result.set_rbflags(w_1_rbflags)
>
>         self.pushvalue(w_result)

It looks like, if an app-level exception is in progress, w_result may
be None.  Try checking for that before setting the flags.

Also:  did you mean to alter w_1's rbflags?  Seems strange that you'd
alter w_1 but not w_2.

-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely MAY reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
prior contractual agreement.

From frankw at mit.edu  Wed Jan 11 21:35:57 2017
From: frankw at mit.edu (Frank Wang)
Date: Wed, 11 Jan 2017 18:35:57 -0800
Subject: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
In-Reply-To: <CAHgd1hG4qhgphTtZm+9pX-Z+sXQ2rSQ_syQG+2qeAG9M_zNR3w@mail.gmail.com>
References: <93774165-2ebb-492e-bc94-8366beefbd84@email.android.com>
 <03a439ec-71fa-454a-a9e0-4238d7505513@email.android.com>
 <CAD=7XjOzwMMc+6TBL31s4xh71CKG-MtA7LQhqdCxuSw4G_zJXA@mail.gmail.com>
 <CAD=7XjOLqAeuTqok0Pje18xFTVD1X8EjfjCY5Rf7kafxwEpnBw@mail.gmail.com>
 <CAMSv6X110t+1iyWvoOt_UfKXrZu0oprYScikrh4+gPMD8wBhBA@mail.gmail.com>
 <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
 <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>
 <CAHgd1hG4qhgphTtZm+9pX-Z+sXQ2rSQ_syQG+2qeAG9M_zNR3w@mail.gmail.com>
Message-ID: <CAD=7XjMjFhVDcvmYA4ZO_u-=xLwoLVcBZ2ppi2Y55XtGv+zJzw@mail.gmail.com>

Hi William,

Good catch! I was just trying to get something to work, but it seems like
you found another bug. I am now checking if it is None:

        # union flags in w_1 and w_2 and propagate to result
        if isinstance(w_result, W_Root) and w_result is not None:
            rb_results = {}
            if isinstance(w_1, W_Root) and w_1 is not None:
                w_1_rbflags = w_1.get_rbflags()
                rb_results.update(w_1_rbflags)
            if isinstance(w_2, W_Root) and w_2 is not None:
                w_2_rbflags = w_2.get_rbflags()
                rb_results.update(w_2_rbflags)
            w_result.set_rbflags(rb_results)

I now get this error:

[translation:ERROR] UnionError:

Offending annotations:
  SomeInstance(can_be_None=False,
classdef=pypy.interpreter.baseobjspace.W_Root)
  SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>)


Occurred processing the following simple_call:
  <MethodDesc 'set_rbflags' of <ClassDef
'pypy.interpreter.baseobjspace.W_Root'> bound to <ClassDef
'pypy.interpreter.baseobjspace.W_Root'> {}> returning

    v831 = simple_call(v830, rb_results_0)

In <FunctionGraph of
(pypy.interpreter.pyopcode:40)opcode_impl_for_div__star_2 at 0x1c190b10>:
Happened at file /home/ubuntu/pypy2-v5.3.1-src/pypy/interpreter/pyopcode.py
line 55

==>             w_result.set_rbflags(rb_results)

Known variable annotations:
 v830 = SomePBC(can_be_None=False, descriptions={...1...},
knowntype=method, subset_of=None)
 rb_results_0 = SomeOrderedDict(dictdef=<{SomeImpossibleValue():
SomeImpossibleValue()}>)

Processing block:
 block at 203 is a <class 'rpython.flowspace.flowcontext.SpamBlock'>
 in (pypy.interpreter.pyopcode:40)opcode_impl_for_div__star_2
 containing the following operations:
       v830 = getattr(w_result_0, ('set_rbflags'))
       v831 = simple_call(v830, rb_results_0)
 --end--

I assume this happens because w_result can be a "subset" of None. What is
the proper way to check if it's None?

Frank

On Wed, Jan 11, 2017 at 3:00 PM, William ML Leslie <
william.leslie.ttg at gmail.com> wrote:

> On 12 January 2017 at 09:08, Frank Wang <frankw at mit.edu> wrote:
> > def binaryoperation(operationname):
> >     """NOT_RPYTHON"""
> >     def opimpl(self, *ignored):
> >         operation = getattr(self.space, operationname)
> >         w_2 = self.popvalue()
> >         w_1 = self.popvalue()
> >         w_result = operation(w_1, w_2)
> >
> >         # union flags in w_1 and w_2 and propagate to result
> >         w_1_rbflags = w_1.get_rbflags()
> >         w_2_rbflags = w_2.get_rbflags()
> >         w_1_rbflags.update(w_2_rbflags)
> >         w_result.set_rbflags(w_1_rbflags)
> >
> >         self.pushvalue(w_result)
>
> It looks like, if an app-level exception is in progress, w_result may
> be None.  Try checking for that before setting the flags.
>
> Also:  did you mean to alter w_1's rbflags?  Seems strange that you'd
> alter w_1 but not w_2.
>
> --
> William Leslie
>
> Notice:
> Likely much of this email is, by the nature of copyright, covered
> under copyright law.  You absolutely MAY reproduce any part of it in
> accordance with the copyright law of the nation you are reading this
> in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
> prior contractual agreement.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170111/8ea9731c/attachment-0001.html>

From armin.rigo at gmail.com  Fri Jan 13 06:34:30 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Fri, 13 Jan 2017 12:34:30 +0100
Subject: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
In-Reply-To: <CAD=7XjMjFhVDcvmYA4ZO_u-=xLwoLVcBZ2ppi2Y55XtGv+zJzw@mail.gmail.com>
References: <93774165-2ebb-492e-bc94-8366beefbd84@email.android.com>
 <03a439ec-71fa-454a-a9e0-4238d7505513@email.android.com>
 <CAD=7XjOzwMMc+6TBL31s4xh71CKG-MtA7LQhqdCxuSw4G_zJXA@mail.gmail.com>
 <CAD=7XjOLqAeuTqok0Pje18xFTVD1X8EjfjCY5Rf7kafxwEpnBw@mail.gmail.com>
 <CAMSv6X110t+1iyWvoOt_UfKXrZu0oprYScikrh4+gPMD8wBhBA@mail.gmail.com>
 <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
 <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>
 <CAHgd1hG4qhgphTtZm+9pX-Z+sXQ2rSQ_syQG+2qeAG9M_zNR3w@mail.gmail.com>
 <CAD=7XjMjFhVDcvmYA4ZO_u-=xLwoLVcBZ2ppi2Y55XtGv+zJzw@mail.gmail.com>
Message-ID: <CAMSv6X32apcnhSrbzb+N-HEJpX=FFqi+UTPhq8naW4OBgfK4RA@mail.gmail.com>

Hi,

On 12 January 2017 at 03:35, Frank Wang <frankw at mit.edu> wrote:
> Good catch! I was just trying to get something to work, but it seems like
> you found another bug. I am now checking if it is None:

No, I think these latest changes don't do anything.  The error message
says: you are storing in the rb_flags attribute something that is
sometimes a dict, and sometimes an instance of W_Root.  Check if there
is somewhere else a subclass of W_Root with its own rb_flags attribute
which would be set to a W_Root.  Then you'd get a crash using that
subclass, also untranslated, simply because 'rb_flags' is supposed to
be two different things.  To be on the safe side, you should change
the name of the attribute in W_Root from rb_flags to something much
more obscure like "_frank_rb_flags".


A bient?t,

Armin.

From armin.rigo at gmail.com  Fri Jan 13 06:36:00 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Fri, 13 Jan 2017 12:36:00 +0100
Subject: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
In-Reply-To: <CAMSv6X32apcnhSrbzb+N-HEJpX=FFqi+UTPhq8naW4OBgfK4RA@mail.gmail.com>
References: <93774165-2ebb-492e-bc94-8366beefbd84@email.android.com>
 <03a439ec-71fa-454a-a9e0-4238d7505513@email.android.com>
 <CAD=7XjOzwMMc+6TBL31s4xh71CKG-MtA7LQhqdCxuSw4G_zJXA@mail.gmail.com>
 <CAD=7XjOLqAeuTqok0Pje18xFTVD1X8EjfjCY5Rf7kafxwEpnBw@mail.gmail.com>
 <CAMSv6X110t+1iyWvoOt_UfKXrZu0oprYScikrh4+gPMD8wBhBA@mail.gmail.com>
 <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
 <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>
 <CAHgd1hG4qhgphTtZm+9pX-Z+sXQ2rSQ_syQG+2qeAG9M_zNR3w@mail.gmail.com>
 <CAD=7XjMjFhVDcvmYA4ZO_u-=xLwoLVcBZ2ppi2Y55XtGv+zJzw@mail.gmail.com>
 <CAMSv6X32apcnhSrbzb+N-HEJpX=FFqi+UTPhq8naW4OBgfK4RA@mail.gmail.com>
Message-ID: <CAMSv6X2Zg_wRx+AXQ+TFJA0fTzF7A2C1cXZoV9T2KMnvAuA1pA@mail.gmail.com>

Re-hi,

On 13 January 2017 at 12:34, Armin Rigo <armin.rigo at gmail.com> wrote:
> says: you are storing in the rb_flags attribute something that is
> sometimes a dict, and sometimes an instance of W_Root.

Another potential reason would be that there is one buggy call
somewhere that calls .set_rbflags(x) where x is an instance of W_Root
instead of a dictionary.


A bient?t,

Armin.

From armin.rigo at gmail.com  Sat Jan 14 11:37:04 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sat, 14 Jan 2017 17:37:04 +0100
Subject: [pypy-dev] RevDB
Message-ID: <CAMSv6X0oR0d77r2Y1Wq-W0bC9Tb_OX=fG9udX13is+KuEnfJbA@mail.gmail.com>

Hi all,

RevDB (https://bitbucket.org/pypy/revdb), the reverse debugger, was
updated to 5.6.x.  It contains a few RevDB-specific fixes as well.  See the
new tag RevDB-pypy2.7-v5.6.1 in the repository, or on direct download at
https://bitbucket.org/pypy/pypy/downloads?tab=tags .

Please contact me if you'd really like Windows support.


A bient?t,

Armin.

From frankw at mit.edu  Sat Jan 14 11:54:30 2017
From: frankw at mit.edu (Frank Wang)
Date: Sat, 14 Jan 2017 11:54:30 -0500
Subject: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
In-Reply-To: <CAMSv6X2Zg_wRx+AXQ+TFJA0fTzF7A2C1cXZoV9T2KMnvAuA1pA@mail.gmail.com>
References: <93774165-2ebb-492e-bc94-8366beefbd84@email.android.com>
 <03a439ec-71fa-454a-a9e0-4238d7505513@email.android.com>
 <CAD=7XjOzwMMc+6TBL31s4xh71CKG-MtA7LQhqdCxuSw4G_zJXA@mail.gmail.com>
 <CAD=7XjOLqAeuTqok0Pje18xFTVD1X8EjfjCY5Rf7kafxwEpnBw@mail.gmail.com>
 <CAMSv6X110t+1iyWvoOt_UfKXrZu0oprYScikrh4+gPMD8wBhBA@mail.gmail.com>
 <CAD=7XjNuw_2j1vAAKJZGht-sMtoqJd4B_=wJzc485zfBWEierA@mail.gmail.com>
 <CAD=7XjMrfMyBd9aHQcO2APY+Lpc9ec5Owqjfs-u48KASC27R8Q@mail.gmail.com>
 <CAHgd1hG4qhgphTtZm+9pX-Z+sXQ2rSQ_syQG+2qeAG9M_zNR3w@mail.gmail.com>
 <CAD=7XjMjFhVDcvmYA4ZO_u-=xLwoLVcBZ2ppi2Y55XtGv+zJzw@mail.gmail.com>
 <CAMSv6X32apcnhSrbzb+N-HEJpX=FFqi+UTPhq8naW4OBgfK4RA@mail.gmail.com>
 <CAMSv6X2Zg_wRx+AXQ+TFJA0fTzF7A2C1cXZoV9T2KMnvAuA1pA@mail.gmail.com>
Message-ID: <CAD=7XjM_3BK-E5o8PD-CSExo6uNpgcg8cdr6ZygJA+9WhgPHyA@mail.gmail.com>

Hi Armin,

Thanks for the help! It seems like the problem is that somewhere in my
code, I called .set_rbflags(x), and it's not checked that x is a dict.



Frank

On Fri, Jan 13, 2017 at 6:36 AM, Armin Rigo <armin.rigo at gmail.com> wrote:

> Re-hi,
>
> On 13 January 2017 at 12:34, Armin Rigo <armin.rigo at gmail.com> wrote:
> > says: you are storing in the rb_flags attribute something that is
> > sometimes a dict, and sometimes an instance of W_Root.
>
> Another potential reason would be that there is one buggy call
> somewhere that calls .set_rbflags(x) where x is an instance of W_Root
> instead of a dictionary.
>
>
> A bient?t,
>
> Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170114/c2a91e67/attachment.html>

From armin.rigo at gmail.com  Sun Jan 15 03:41:51 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sun, 15 Jan 2017 09:41:51 +0100
Subject: [pypy-dev] Leysin sprint?
Message-ID: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>

Hi all,

I'm starting to organize the Leysin sprint for this winter.

The first note is that the Swiss Python Summit will take place on Feb
17th near Zurich (http://www.python-summit.ch/).  I'll give a talk
about RevDB, the reverse debugger.

One option would be to add a few days of sprint at or near the
conference location.  That may be a way to attract more people.  The
other option would be a regular sprint in Leysin.  For that case, the
dates are still completely open.  (It snowed a lot!)

Anyone that thinks about coming, please tell me your preferences and dates!


A bient?t,

Armin.

From planrichi at gmail.com  Sun Jan 15 06:52:49 2017
From: planrichi at gmail.com (Richard Plangger)
Date: Sun, 15 Jan 2017 12:52:49 +0100
Subject: [pypy-dev] Leysin sprint?
In-Reply-To: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
References: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
Message-ID: <25b13db6-42ec-a240-98cd-fadb183c83d8@gmail.com>

Hi Armin,

I'm happy to join the sprint (either location). I would prefer:
All of February or March 1-10th.

Cheers,
Richard

On 01/15/2017 09:41 AM, Armin Rigo wrote:
> Hi all,
> 
> I'm starting to organize the Leysin sprint for this winter.
> 
> The first note is that the Swiss Python Summit will take place on Feb
> 17th near Zurich (http://www.python-summit.ch/).  I'll give a talk
> about RevDB, the reverse debugger.
> 
> One option would be to add a few days of sprint at or near the
> conference location.  That may be a way to attract more people.  The
> other option would be a regular sprint in Leysin.  For that case, the
> dates are still completely open.  (It snowed a lot!)
> 
> Anyone that thinks about coming, please tell me your preferences and dates!
> 
> 
> A bient?t,
> 
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
> 

From remi.meier at gmail.com  Mon Jan 16 04:15:37 2017
From: remi.meier at gmail.com (Remi Meier)
Date: Mon, 16 Jan 2017 09:15:37 +0000
Subject: [pypy-dev] Leysin sprint?
In-Reply-To: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
References: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
Message-ID: <CAJ8fdYy3Yq8BkRWzWQWaAVj73z4FGHrS6HxcH=_+Y8=pjY7g6Q@mail.gmail.com>

Hi Armin,

Unfortunately, I'll be on holiday the week of the Swiss Python Summit.

I would like to come to Leysin again. The week 6.2-11.2 should work for me,
but it's quite soon. After the 20th of February I'm generally open. But
since it would be during the semester, I may not stay for a whole week.

Cheers,
Remi

On Sun, 15 Jan 2017 at 09:43 Armin Rigo <armin.rigo at gmail.com> wrote:

> Hi all,
>
> I'm starting to organize the Leysin sprint for this winter.
>
> The first note is that the Swiss Python Summit will take place on Feb
> 17th near Zurich (http://www.python-summit.ch/).  I'll give a talk
> about RevDB, the reverse debugger.
>
> One option would be to add a few days of sprint at or near the
> conference location.  That may be a way to attract more people.  The
> other option would be a regular sprint in Leysin.  For that case, the
> dates are still completely open.  (It snowed a lot!)
>
> Anyone that thinks about coming, please tell me your preferences and dates!
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170116/4fd42f46/attachment.html>

From me at manueljacob.de  Mon Jan 16 13:43:30 2017
From: me at manueljacob.de (Manuel Jacob)
Date: Mon, 16 Jan 2017 19:43:30 +0100
Subject: [pypy-dev] Leysin sprint?
In-Reply-To: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
References: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
Message-ID: <80086789a0e56a06b1ccef6cd17e8325@manueljacob.de>

Hi,

Fortunately, this year I can place my exams quite flexibly.  I'm staying 
in Brussels until the February 7th after FOSDEM, so anything after that 
would work perfectly for me.

I'd prefer a Leysin sprint, but mostly for reasons of nostalgia rather 
than more serious considerations.

-Manuel

On 2017-01-15 09:41, Armin Rigo wrote:
> Hi all,
> 
> I'm starting to organize the Leysin sprint for this winter.
> 
> The first note is that the Swiss Python Summit will take place on Feb
> 17th near Zurich (http://www.python-summit.ch/).  I'll give a talk
> about RevDB, the reverse debugger.
> 
> One option would be to add a few days of sprint at or near the
> conference location.  That may be a way to attract more people.  The
> other option would be a regular sprint in Leysin.  For that case, the
> dates are still completely open.  (It snowed a lot!)
> 
> Anyone that thinks about coming, please tell me your preferences and 
> dates!
> 
> 
> A bient?t,
> 
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From fijall at gmail.com  Mon Jan 16 15:59:12 2017
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 16 Jan 2017 22:59:12 +0200
Subject: [pypy-dev] Leysin sprint?
In-Reply-To: <80086789a0e56a06b1ccef6cd17e8325@manueljacob.de>
References: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
 <80086789a0e56a06b1ccef6cd17e8325@manueljacob.de>
Message-ID: <CAK5idxQ4nLNNA0osZ7LSUVVyFnf+i5S_9rhOceB-0pBDr1FGgA@mail.gmail.com>

I have a place to stay in Zurich so it's a bit of a win for me :-)

I would prefer this to be somewhere around early March I think, but I
can't yet commit

Cheers,
fijal

On Mon, Jan 16, 2017 at 8:43 PM, Manuel Jacob <me at manueljacob.de> wrote:
> Hi,
>
> Fortunately, this year I can place my exams quite flexibly.  I'm staying in
> Brussels until the February 7th after FOSDEM, so anything after that would
> work perfectly for me.
>
> I'd prefer a Leysin sprint, but mostly for reasons of nostalgia rather than
> more serious considerations.
>
> -Manuel
>
>
> On 2017-01-15 09:41, Armin Rigo wrote:
>>
>> Hi all,
>>
>> I'm starting to organize the Leysin sprint for this winter.
>>
>> The first note is that the Swiss Python Summit will take place on Feb
>> 17th near Zurich (http://www.python-summit.ch/).  I'll give a talk
>> about RevDB, the reverse debugger.
>>
>> One option would be to add a few days of sprint at or near the
>> conference location.  That may be a way to attract more people.  The
>> other option would be a regular sprint in Leysin.  For that case, the
>> dates are still completely open.  (It snowed a lot!)
>>
>> Anyone that thinks about coming, please tell me your preferences and
>> dates!
>>
>>
>> A bient?t,
>>
>> Armin.
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From anto.cuni at gmail.com  Tue Jan 17 11:23:52 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 17 Jan 2017 16:23:52 +0000
Subject: [pypy-dev] Leysin sprint?
In-Reply-To: <CAK5idxQ4nLNNA0osZ7LSUVVyFnf+i5S_9rhOceB-0pBDr1FGgA@mail.gmail.com>
References: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
 <80086789a0e56a06b1ccef6cd17e8325@manueljacob.de>
 <CAK5idxQ4nLNNA0osZ7LSUVVyFnf+i5S_9rhOceB-0pBDr1FGgA@mail.gmail.com>
Message-ID: <CAHPg78xZNT-r7cR1k-RTU4xa_1BCWy=XSi0vS78tMYE1HTwqJg@mail.gmail.com>

Hi,
I'd prefer the classic Leysin sprint :).
I will probably be unavailable the first two weeks of March; other than
that, any week should be fine for me.

On Mon, Jan 16, 2017 at 8:59 PM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> I have a place to stay in Zurich so it's a bit of a win for me :-)
>
> I would prefer this to be somewhere around early March I think, but I
> can't yet commit
>
> Cheers,
> fijal
>
> On Mon, Jan 16, 2017 at 8:43 PM, Manuel Jacob <me at manueljacob.de> wrote:
> > Hi,
> >
> > Fortunately, this year I can place my exams quite flexibly.  I'm staying
> in
> > Brussels until the February 7th after FOSDEM, so anything after that
> would
> > work perfectly for me.
> >
> > I'd prefer a Leysin sprint, but mostly for reasons of nostalgia rather
> than
> > more serious considerations.
> >
> > -Manuel
> >
> >
> > On 2017-01-15 09:41, Armin Rigo wrote:
> >>
> >> Hi all,
> >>
> >> I'm starting to organize the Leysin sprint for this winter.
> >>
> >> The first note is that the Swiss Python Summit will take place on Feb
> >> 17th near Zurich (http://www.python-summit.ch/).  I'll give a talk
> >> about RevDB, the reverse debugger.
> >>
> >> One option would be to add a few days of sprint at or near the
> >> conference location.  That may be a way to attract more people.  The
> >> other option would be a regular sprint in Leysin.  For that case, the
> >> dates are still completely open.  (It snowed a lot!)
> >>
> >> Anyone that thinks about coming, please tell me your preferences and
> >> dates!
> >>
> >>
> >> A bient?t,
> >>
> >> Armin.
> >> _______________________________________________
> >> pypy-dev mailing list
> >> pypy-dev at python.org
> >> https://mail.python.org/mailman/listinfo/pypy-dev
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/daacdb18/attachment.html>

From tobias.oberstein at gmail.com  Tue Jan 17 12:51:59 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 18:51:59 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
Message-ID: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>

Hi guys,

please see (reposting here to grab attention as hinted by Armin)

https://bitbucket.org/pypy/pypy/issues/2467/cppyy-fails-to-build-on-gcc-5-and-clang

Note: for now I am fine, I managed to build it using gcc 4.9 toolchain. 
Need to see how far I get with captnproto now ..

Cheers,
/Tobias

From rymg19 at gmail.com  Tue Jan 17 13:03:58 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 17 Jan 2017 18:03:58 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
Message-ID: <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>

What happens if you remove the `-stdlib=libc++`? Seems like that's where the
Clang build error is coming from.

  

\--  

Ryan (????)

Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else

<http://refi64.com/>

  

On Jan 17 2017, at 11:53 am, Tobias Oberstein <tobias.oberstein at gmail.com>
wrote:  

> Hi guys,

>

> please see (reposting here to grab attention as hinted by Armin)

>

> https://bitbucket.org/pypy/pypy/issues/2467/cppyy-fails-to-build-on-gcc-5
-and-clang

>

> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.  
Need to see how far I get with captnproto now ..

>

> Cheers,  
/Tobias  
_______________________________________________  
pypy-dev mailing list  
pypy-dev at python.org  
https://mail.python.org/mailman/listinfo/pypy-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/123c0949/attachment.html>

From wlavrijsen at lbl.gov  Tue Jan 17 13:20:17 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 17 Jan 2017 10:20:17 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
Message-ID: <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>

Hi,

On Tuesday 2017-01-17 18:03, Ryan Gonzalez wrote:
> What happens if you remove the `-stdlib=libc++`? Seems like that's where the
> Clang build error is coming from.

that will likely be a solution, b/c that wrapper file is put in place if the
libcxx from clang is not used. Seems a configuration mismatch: to use libcxx,
the cmake machinery needs to be told to do so (-Dlibcxx=ON IIRC), as it's off
by default.

Sorry, is my first ever pip and had to intermarry python and C++ installs w/o
wanting to rewrite the infrastructure. :} I'll have to look how to provide
these extra arguments to cmake, but should be possible. I don't think libcxx
has been tested outside MacOS X, though.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From rymg19 at gmail.com  Tue Jan 17 13:28:18 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 17 Jan 2017 18:28:18 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
Message-ID: <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>

I think you misunderstood me. Your build log shows:

  

oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$ make
cpppypy  

CC=`which clang` \  
CXX=`which clang` \  
CXXFLAGS="-std=c++11 -stdlib=libc++" \  
LDFLAGS="-stdlib=libc++" \  
MAKE_NPROCS=4 \  
~/pypy-5.6-linux_x86_64-portable/bin/pip install --verbose PyPy-cppyy-backend  

  

  

I'm saying to _remove_  the `-stdlib=libc++` that's currently there.

  
\--  

Ryan (????)

Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else

<http://refi64.com/>

  
On Jan 17 2017, at 12:20 pm, wlavrijsen at lbl.gov wrote:  

> Hi,

>

> On Tuesday 2017-01-17 18:03, Ryan Gonzalez wrote:  
> What happens if you remove the `-stdlib=libc++`? Seems like that's where the  
> Clang build error is coming from.

>

> that will likely be a solution, b/c that wrapper file is put in place if the  
libcxx from clang is not used. Seems a configuration mismatch: to use libcxx,  
the cmake machinery needs to be told to do so (-Dlibcxx=ON IIRC), as it's off  
by default.

>

> Sorry, is my first ever pip and had to intermarry python and C++ installs
w/o  
wanting to rewrite the infrastructure. :} I'll have to look how to provide  
these extra arguments to cmake, but should be possible. I don't think libcxx  
has been tested outside MacOS X, though.

>

> Best regards,  
????????????Wim  
\--  
WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/c18ba25e/attachment.html>

From wlavrijsen at lbl.gov  Tue Jan 17 13:37:28 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 17 Jan 2017 10:37:28 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
 <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
Message-ID: <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>

Hi Ryan,

On Tuesday 2017-01-17 18:28, Ryan Gonzalez wrote:
> I think you misunderstood me. Your build log shows:

no, I didn't. :) It's not my build log, it's Tobias's.

> I'm saying to _remove_  the `-stdlib=libc++` that's currently there.

Yes and I expect it to work. But I wrote the pip, and ideally, I get that
fixed to allow what Tobias is doing. B/c if the '-stdlib=libc++' is removed,
the build system will not pick it up either (it's off by default). But the
option exists, so if someone wanted libc++ for whatever purpose, it seems
reasonable enough to support it.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From rymg19 at gmail.com  Tue Jan 17 13:59:33 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 17 Jan 2017 12:59:33 -0600
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
 <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
Message-ID: <CAO41-mNnm7kGPJDQToNB-i3Ca+WC_iEscWdfT5Wqx9QH9fJYSw@mail.gmail.com>

On Jan 17, 2017 12:37 PM, <wlavrijsen at lbl.gov> wrote:

Hi Ryan,


On Tuesday 2017-01-17 18:28, Ryan Gonzalez wrote:

> I think you misunderstood me. Your build log shows:
>

no, I didn't. :) It's not my build log, it's Tobias's.


*facepalm* I need to double-check who exactly I'm replying to... :/



I'm saying to _remove_  the `-stdlib=libc++` that's currently there.
>

Yes and I expect it to work. But I wrote the pip, and ideally, I get that
fixed to allow what Tobias is doing. B/c if the '-stdlib=libc++' is removed,
the build system will not pick it up either (it's off by default). But the
option exists, so if someone wanted libc++ for whatever purpose, it seems
reasonable enough to support it.


Best regards,
           Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net



--
Ryan (????)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/a5f41845/attachment.html>

From wlavrijsen at lbl.gov  Tue Jan 17 14:01:57 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 17 Jan 2017 11:01:57 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <b43f8d1c-86c5-eadf-e970-6a7275b1c543@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
 <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
 <b43f8d1c-86c5-eadf-e970-6a7275b1c543@gmail.com>
Message-ID: <alpine.LSU.2.11.1701171050000.13553@cp3-jyni>

Hi Tobias,

> 1. gcc 5 / libstdc++
> 2. gcc 4.9 / libstdc++
> 3. clang / libc++
> 4. clang / libstdc++
>
> 1/3/4 fails building cppy.

how does 3 fail? I presume differently than 1, 4?

> 2 builds cppyy, but the example bails out.

In which way?

> What toolchain/system are you using?

Gentoo, SuSE and Mac. But the build files are taken from a larger project
that I've stripped down. The larger project has a plethora of systems that
have nightly builds:

   http://cdash.cern.ch/index.php?project=ROOT

So in theory, it should work (famous last words).

> Do you have a Docker image that works?

No, haven't. ROOT itself is part of most distro's. I think it's called
"root-cern" on Ubuntu? That only gets you most of the way, however: there's
also a clingcwrapper.cxx (in the pip and the cxx source) that provides all
the C entry points that the PyPy cppyy module needs and it's not currently
in the ROOT builds (yet).

Sorry for the trouble, but it hasn't even been announced, so you're if not
the first, then one of the first users of this pip. :}

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From rymg19 at gmail.com  Tue Jan 17 14:02:43 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 17 Jan 2017 13:02:43 -0600
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
Message-ID: <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>

In your Makefile, change this:

CXX=`which clang` \

to:

CXX=`which clang++` \

--
Ryan (????)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://kirbyfan64.github.io/

On Jan 17, 2017 12:41 PM, "Tobias Oberstein" <tobias.oberstein at gmail.com>
wrote:

> Am 17.01.2017 um 19:03 schrieb Ryan Gonzalez:
>
>> What happens if you remove the `-stdlib=libc++`? Seems like that's where
>> the
>> Clang build error is coming from.
>>
>
> 1)
> removing explicit stdlib selection leads to the following linker error
>
> https://gist.github.com/oberstet/f75b9618b12b1e61289b147fcfd
> 90bc0#file-gistfile1-txt-L885
>
> makefile target was: https://github.com/oberstet/sc
> ratchbox/blob/master/cpp/capnproto/test1/Makefile#L10
>
>
> 2)
> explicitly selecting GNU c++ stdlib for clang results in
>
> https://gist.github.com/oberstet/df1f59b9ce674d8984ee5f9ce045dccb
>
> makefile target was: https://github.com/oberstet/sc
> ratchbox/blob/master/cpp/capnproto/test1/Makefile#L27
>
>
> 3)
> when using the gcc-4.9 build of cppyy, and trying the basic example (
> http://doc.pypy.org/en/latest/cppyy.html#basic-bindings-example) leads to:
>
> https://gist.github.com/oberstet/d260ee15c81954bea8298b7400d04870
>
> ---
>
> so right now I cannot get the basic example working =(
>
>
>>
>>
>> \--
>>
>> Ryan (????)
>>
>> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
>>
>> <http://refi64.com/>
>>
>>
>>
>> On Jan 17 2017, at 11:53 am, Tobias Oberstein <tobias.oberstein at gmail.com
>> >
>> wrote:
>>
>> Hi guys,
>>>
>>
>>
>>>
>> please see (reposting here to grab attention as hinted by Armin)
>>>
>>
>>
>>>
>> https://bitbucket.org/pypy/pypy/issues/2467/cppyy-fails-to-build-on-gcc-5
>>>
>> -and-clang
>>
>>
>>>
>> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.
>>>
>> Need to see how far I get with captnproto now ..
>>
>>
>>>
>> Cheers,
>>>
>> /Tobias
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/0749e74c/attachment.html>

From anto.cuni at gmail.com  Tue Jan 17 14:04:42 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 17 Jan 2017 19:04:42 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
Message-ID: <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>

?Hi,?

On Tue, Jan 17, 2017 at 5:51 PM, Tobias Oberstein <
tobias.oberstein at gmail.com> wrote:

> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.
> Need to see how far I get with captnproto now ..
>

?if you are interested in capnproto, you might want to try capnpy: it's
written in pure python and it has been designed to be super fast on pypy
(and it's very fast on CPython as well):
https://github.com/antocuni/capnpy
?http://capnpy.readthedocs.io/en/latest/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/a54c9762/attachment-0001.html>

From wlavrijsen at lbl.gov  Tue Jan 17 14:23:16 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 17 Jan 2017 11:23:16 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <7cddf017-433a-22d2-5db9-93de48cba75d@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
 <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
 <b43f8d1c-86c5-eadf-e970-6a7275b1c543@gmail.com>
 <alpine.LSU.2.11.1701171050000.13553@cp3-jyni>
 <7cddf017-433a-22d2-5db9-93de48cba75d@gmail.com>
Message-ID: <alpine.LSU.2.11.1701171118440.16565@cp3-jyni>

Hi Tobias,

> https://gist.github.com/oberstet/d260ee15c81954bea8298b7400d04870

those symbols (ZN4ROOT17TGenericClassInfoD1Ev and friends) live in libCore.so
which is installed under site packages and you can link with that. However,
the backend should add all those symbols and make them available to the
linker (it's loaded with RTLD_GLOBAL), so not sure why it doesn't work as-is.

> FWIW, I do think that would be very useful to get started.

Yes; but it's another thing for me to learn (albeit a good thing).

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From rymg19 at gmail.com  Tue Jan 17 14:56:28 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 17 Jan 2017 19:56:28 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
Message-ID: <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>

This one looks like a bug in cppyy!

  

Try going to clingcwrapper.cxx and changing line 525 from:

  

  

cppresult->std::string::~string();  

  

  

to:

  

  

cppresult->std::string::~basic_string();  

  

  

(e.g. change `~string` to `~basic_string`).  
\--  

Ryan (????)

Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else

<http://refi64.com/>

  
On Jan 17 2017, at 1:51 pm, Tobias Oberstein <tobias.oberstein at gmail.com>
wrote:  

> Hi Ryan,

>

> with CXX=clang++ the build nearly finishes (I am a moron, sorry, clang  
isn't that clever), that is

>

> CC=`which clang` \  
CXX=`which clang++` \  
CXXFLAGS="-std=c++11" \  
MAKE_NPROCS=4 \  
~/pypy-5.6-linux_x86_64-portable/bin/pip install --no-cache-dir  
\--verbose PyPy-cppyy-backend

>

> I get to

>

> ?????[ 95%] Linking CXX shared library ../../lib/libCling.so  
?????make[3]: Verzeichnis  
?/tmp/pip-build-EJ5mSe/PyPy-cppyy-backend/builddir? wird verlassen  
[ 95%] Built target Cling

>

> but then it fails compiling clingcwrapper.cxx.o

>

> https://gist.github.com/oberstet/4f60bd02bab8921e09b94c47188c291c#file-
gistfile1-txt-L8

>

> This is

>

> oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$  
clang++ --version  
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)  
Target: x86_64-pc-linux-gnu  
Thread model: posix  
InstalledDir: /usr/bin

>

> Thanks for your help!  
Tobias

>

>  
Am 17.01.2017 um 20:02 schrieb Ryan Gonzalez:  
> In your Makefile, change this:  
>  
> CXX=`which clang` \  
>  
> to:  
>  
> CXX=`which clang++` \  
>  
> \--  
> Ryan (????)  
> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else  
> <http://kirbyfan64.github.io/>  
>  
> On Jan 17, 2017 12:41 PM, "Tobias Oberstein" <tobias.oberstein at gmail.com>  
> wrote:  
>  
>> Am 17.01.2017 um 19:03 schrieb Ryan Gonzalez:  
>>  
>>> What happens if you remove the `-stdlib=libc++`? Seems like that's where  
>>> the  
>>> Clang build error is coming from.  
>>>  
>>  
>> 1)  
>> removing explicit stdlib selection leads to the following linker error  
>>  
>> https://gist.github.com/oberstet/f75b9618b12b1e61289b147fcfd  
>> 90bc0#file-gistfile1-txt-L885  
>>  
>> makefile target was: https://github.com/oberstet/sc  
>> ratchbox/blob/master/cpp/capnproto/test1/Makefile#L10  
>>  
>>  
>> 2)  
>> explicitly selecting GNU c++ stdlib for clang results in  
>>  
>> https://gist.github.com/oberstet/df1f59b9ce674d8984ee5f9ce045dccb  
>>  
>> makefile target was: https://github.com/oberstet/sc  
>> ratchbox/blob/master/cpp/capnproto/test1/Makefile#L27  
>>  
>>  
>> 3)  
>> when using the gcc-4.9 build of cppyy, and trying the basic example (  
>> <http://doc.pypy.org/en/latest/cppyy.html#basic-bindings-example)> leads
to:  
>>  
>> https://gist.github.com/oberstet/d260ee15c81954bea8298b7400d04870  
>>  
>> \---  
>>  
>> so right now I cannot get the basic example working =(  
>>  
>>  
>>>  
>>>  
>>> \\--  
>>>  
>>> Ryan (????)  
>>>  
>>> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else  
>>>  
>>> <http://refi64.com/>  
>>>  
>>>  
>>>  
>>> On Jan 17 2017, at 11:53 am, Tobias Oberstein <tobias.oberstein at gmail.com  
>>>>  
>>> wrote:  
>>>  
>>> Hi guys,  
>>>>  
>>>  
>>>  
>>>>  
>>> please see (reposting here to grab attention as hinted by Armin)  
>>>>  
>>>  
>>>  
>>>>  
>>> https://bitbucket.org/pypy/pypy/issues/2467/cppyy-fails-to-build-on-gcc-5  
>>>>  
>>> -and-clang  
>>>  
>>>  
>>>>  
>>> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.  
>>>>  
>>> Need to see how far I get with captnproto now ..  
>>>  
>>>  
>>>>  
>>> Cheers,  
>>>>  
>>> /Tobias  
>>> _______________________________________________  
>>> pypy-dev mailing list  
>>> pypy-dev at python.org  
>>> https://mail.python.org/mailman/listinfo/pypy-dev  
>>>  
>>>  
>>  
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/cff3cdc5/attachment.html>

From rymg19 at gmail.com  Tue Jan 17 16:39:27 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 17 Jan 2017 21:39:27 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
Message-ID: <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>

Whoops, missed that one! Change line 715 to:

  

std::cerr << "Warning: " << msg.str() << '\n';

  

  
\--  

Ryan (????)

Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else

<http://refi64.com/>

  
On Jan 17 2017, at 3:30 pm, Tobias Oberstein <tobias.oberstein at gmail.com>
wrote:  

> Hi Ryan,

>

> Am 17.01.2017 um 20:56 schrieb Ryan Gonzalez:  
> This one looks like a bug in cppyy!  
> Try going to clingcwrapper.cxx and changing line 525 from:

>

> Yeah, that helps: the issue at line 525 is gone, but the 2nd issue at  
line 715 persists:

>

>  
/tmp/pip-BVnCrW-build/src/backend/cppyy/src/clingcwrapper.cxx:715:35:  
error: invalid operands to binary expression ('basic_ostream<char,  
std::char_traits<char> >' and 'std::ostringstream' (aka  
'basic_ostringstream<char>'))  
??????????????std::cerr << "Warning: " << msg << '\n';  
??????????????~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~

>

> https://gist.github.com/oberstet/271719f569087f423197b6d8808ea6f1

>

> /Tobias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170117/73065d76/attachment-0001.html>

From wlavrijsen at lbl.gov  Tue Jan 17 19:34:16 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 17 Jan 2017 16:34:16 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
Message-ID: <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>

Hi Ryan, Tobias,

On Tuesday 2017-01-17 21:39, Ryan Gonzalez wrote:
> Whoops, missed that one! Change line 715 to:
> std::cerr << "Warning: " << msg.str() << '\n';

indeed, thanks! Now fixed in the repository. Hopefully up soon in the pip
(takes more testing, hence time) ...

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From John.Zhang at anu.edu.au  Tue Jan 17 19:20:10 2017
From: John.Zhang at anu.edu.au (John Zhang)
Date: Wed, 18 Jan 2017 00:20:10 +0000
Subject: [pypy-dev] os_read & rposix.c_read
Message-ID: <AEA9B4E3-2CF3-4DF2-8A51-1C31BAB16DE0@anu.edu.au>

Hi all,
Anyone knows why pypy.module._file.readinto uses a locally defined ?os_read? rather than rpython.rlib.rposix.c_read? It seems that the functions are defined almost the same way, except one uses rffi.CCHARP the other uses rffi.VOIDP, and the rposix.c_read has a more complete ExternalCompilationInfo.
Whilst this doesn?t pose any problem to C backend, it does cause a small trouble in my Mu VM backend, as the two external functions have deemed different even though they are the same, thus causing a name conflict.
The fix is simple in either case: use c_read in _file.readinto, or I just add a small hack in my backend. But I?d like to know why this is the case in the first place, is there a special reason for it?

Regards,
John Zhang

------------------------------------------------------
John Zhang
Research Assistant
Programming Languages, Design & Implementation Division
Computer Systems Group
ANU College of Engineering & Computer Science
108 North Rd
The Australian National University
Acton ACT 2601
john.zhang at anu.edu.au<mailto:john.zhang at anu.edu.au>





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170118/43df36c6/attachment.html>

From tobias.oberstein at gmail.com  Tue Jan 17 13:41:06 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 19:41:06 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
Message-ID: <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>

Am 17.01.2017 um 19:03 schrieb Ryan Gonzalez:
> What happens if you remove the `-stdlib=libc++`? Seems like that's where the
> Clang build error is coming from.

1)
removing explicit stdlib selection leads to the following linker error

https://gist.github.com/oberstet/f75b9618b12b1e61289b147fcfd90bc0#file-gistfile1-txt-L885

makefile target was: 
https://github.com/oberstet/scratchbox/blob/master/cpp/capnproto/test1/Makefile#L10


2)
explicitly selecting GNU c++ stdlib for clang results in

https://gist.github.com/oberstet/df1f59b9ce674d8984ee5f9ce045dccb

makefile target was: 
https://github.com/oberstet/scratchbox/blob/master/cpp/capnproto/test1/Makefile#L27


3)
when using the gcc-4.9 build of cppyy, and trying the basic example 
(http://doc.pypy.org/en/latest/cppyy.html#basic-bindings-example) leads to:

https://gist.github.com/oberstet/d260ee15c81954bea8298b7400d04870

---

so right now I cannot get the basic example working =(

>
>
>
> \--
>
> Ryan (????)
>
> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
>
> <http://refi64.com/>
>
>
>
> On Jan 17 2017, at 11:53 am, Tobias Oberstein <tobias.oberstein at gmail.com>
> wrote:
>
>> Hi guys,
>
>>
>
>> please see (reposting here to grab attention as hinted by Armin)
>
>>
>
>> https://bitbucket.org/pypy/pypy/issues/2467/cppyy-fails-to-build-on-gcc-5
> -and-clang
>
>>
>
>> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.
> Need to see how far I get with captnproto now ..
>
>>
>
>> Cheers,
> /Tobias
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>


From tobias.oberstein at gmail.com  Tue Jan 17 13:47:41 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 19:47:41 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
 <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
Message-ID: <b43f8d1c-86c5-eadf-e970-6a7275b1c543@gmail.com>

Hi Wim,

>> I'm saying to _remove_  the `-stdlib=libc++` that's currently there.
>
> Yes and I expect it to work. But I wrote the pip, and ideally, I get that
> fixed to allow what Tobias is doing. B/c if the '-stdlib=libc++' is
> removed,
> the build system will not pick it up either (it's off by default). But the
> option exists, so if someone wanted libc++ for whatever purpose, it seems
> reasonable enough to support it.

FreeBSD is fully a clang/libc++ system (both kernel and userland), and 
it would be very nice to support it=)

However, my problem right now is: I cannot get the basic example working 
on my desktop (Ubuntu 16.04) with any toolchain:

1. gcc 5 / libstdc++
2. gcc 4.9 / libstdc++
3. clang / libc++
4. clang / libstdc++

1/3/4 fails building cppy.

2 builds cppyy, but the example bails out.

So I am stuck =(

What toolchain/system are you using? Do you have a Docker image that works?

Cheers,
/Tobias

>
> Best regards,
>            Wim


From tobias.oberstein at gmail.com  Tue Jan 17 14:17:17 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 20:17:17 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701171050000.13553@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171010290.13553@cp3-jyni>
 <1zn6uta5d35m9wbdjvzwuxdrq-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171030500.13553@cp3-jyni>
 <b43f8d1c-86c5-eadf-e970-6a7275b1c543@gmail.com>
 <alpine.LSU.2.11.1701171050000.13553@cp3-jyni>
Message-ID: <7cddf017-433a-22d2-5db9-93de48cba75d@gmail.com>

Hi Wim,

Am 17.01.2017 um 20:01 schrieb wlavrijsen at lbl.gov:
> Hi Tobias,
>
>> 1. gcc 5 / libstdc++
>> 2. gcc 4.9 / libstdc++
>> 3. clang / libc++
>> 4. clang / libstdc++
>>
>> 1/3/4 fails building cppy.
>
> how does 3 fail? I presume differently than 1, 4?

https://gist.github.com/oberstet/1771d9142cf566c5a877de0df5500899

Note: right now, I try what Ryan suggests in his new reply .. 
CXX=clang++ (not clang) - I'd expected that clang is clever enough to 
autoselect/do the right thing, but ..

>
>> 2 builds cppyy, but the example bails out.
>
> In which way?

https://gist.github.com/oberstet/d260ee15c81954bea8298b7400d04870

>
>> What toolchain/system are you using?
>
> Gentoo, SuSE and Mac. But the build files are taken from a larger project
> that I've stripped down. The larger project has a plethora of systems that
> have nightly builds:
>
>   http://cdash.cern.ch/index.php?project=ROOT

Wow.

>
> So in theory, it should work (famous last words).

The build matrix seems to include Ubuntu 16.04 / gcc 5

http://cdash.cern.ch/buildSummary.php?buildid=314808

.. so it "should work";)

>
>> Do you have a Docker image that works?
>
> No, haven't. ROOT it

FWIW, I do think that would be very useful to get started.

All this C++ machinery, combined with PyPy etc etc is tricky. A "known 
good" can be very helpful, if only as a starting point ..

self is part of most distro's. I think it's called
> "root-cern" on Ubuntu? That only gets you most of the way, however: there's
> also a clingcwrapper.cxx (in the pip and the cxx source) that provides all
> the C entry points that the PyPy cppyy module needs and it's not currently
> in the ROOT builds (yet).

There is a lot in your words I don't understand;) I need to dig deeper ..

>
> Sorry for the trouble, but it hasn't even been announced, so you're if not
> the first, then one of the first users of this pip. :}

Nothing to apologize! I am very well aware of / used to the pain that 
C++ stuff usually comes with;) Expected.

Thanks for a cool tool!

Cheers,
/Tobias

>
> Best regards,
>            Wim


From tobias.oberstein at gmail.com  Tue Jan 17 14:27:24 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 20:27:24 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
Message-ID: <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>

Hi Antonio,

>> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.
>> Need to see how far I get with captnproto now ..
>>
>
> ?if you are interested in capnproto, you might want to try capnpy: it's
> written in pure python and it has been designed to be super fast on pypy
> (and it's very fast on CPython as well):
> https://github.com/antocuni/capnpy
> ?http://capnpy.readthedocs.io/en/latest/
>

Fantastic!!

Actually, if this is true (performance), this would be wonderful - I can 
spare my nerves (hello C++, you are a moooonster!), and eat my cake ..

The benchmarks

http://capnpy.readthedocs.io/en/latest/benchmarks.html

for scalar attribute access look awesome (close to mere instance access).

Structured value access (lists/dict): bigger difference. I am also 
concerned about GC pressure - is it still "zero copy"?

Note: I am purely interested in performance on PyPy ..

In general: I thought it would be a good idea to use capnproto C++ 
generator, and then cppyy to get the best performance (on pypy). Given 
there is "antocuni/capnpy", do you think this is a pointless endeavour?

--

Anyways, thanks so much for that hint - I will try it. (I will try the 
other hints given to me by Ryan and Wim too, since I already consumed 
their time, and maybe there is some gain/insight)

Cheers,
/Tobias


From tobias.oberstein at gmail.com  Tue Jan 17 14:51:15 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 20:51:15 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
Message-ID: <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>

Hi Ryan,

with CXX=clang++ the build nearly finishes (I am a moron, sorry, clang 
isn't that clever), that is

	CC=`which clang` \
	CXX=`which clang++` \
	CXXFLAGS="-std=c++11" \
	MAKE_NPROCS=4 \
~/pypy-5.6-linux_x86_64-portable/bin/pip install --no-cache-dir 
--verbose PyPy-cppyy-backend

I get to

     [ 95%] Linking CXX shared library ../../lib/libCling.so
     make[3]: Verzeichnis 
?/tmp/pip-build-EJ5mSe/PyPy-cppyy-backend/builddir? wird verlassen
[ 95%] Built target Cling

but then it fails compiling clingcwrapper.cxx.o

https://gist.github.com/oberstet/4f60bd02bab8921e09b94c47188c291c#file-gistfile1-txt-L8

This is

oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$ 
clang++ --version
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Thanks for your help!
Tobias


Am 17.01.2017 um 20:02 schrieb Ryan Gonzalez:
> In your Makefile, change this:
>
> CXX=`which clang` \
>
> to:
>
> CXX=`which clang++` \
>
> --
> Ryan (????)
> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
> http://kirbyfan64.github.io/
>
> On Jan 17, 2017 12:41 PM, "Tobias Oberstein" <tobias.oberstein at gmail.com>
> wrote:
>
>> Am 17.01.2017 um 19:03 schrieb Ryan Gonzalez:
>>
>>> What happens if you remove the `-stdlib=libc++`? Seems like that's where
>>> the
>>> Clang build error is coming from.
>>>
>>
>> 1)
>> removing explicit stdlib selection leads to the following linker error
>>
>> https://gist.github.com/oberstet/f75b9618b12b1e61289b147fcfd
>> 90bc0#file-gistfile1-txt-L885
>>
>> makefile target was: https://github.com/oberstet/sc
>> ratchbox/blob/master/cpp/capnproto/test1/Makefile#L10
>>
>>
>> 2)
>> explicitly selecting GNU c++ stdlib for clang results in
>>
>> https://gist.github.com/oberstet/df1f59b9ce674d8984ee5f9ce045dccb
>>
>> makefile target was: https://github.com/oberstet/sc
>> ratchbox/blob/master/cpp/capnproto/test1/Makefile#L27
>>
>>
>> 3)
>> when using the gcc-4.9 build of cppyy, and trying the basic example (
>> http://doc.pypy.org/en/latest/cppyy.html#basic-bindings-example) leads to:
>>
>> https://gist.github.com/oberstet/d260ee15c81954bea8298b7400d04870
>>
>> ---
>>
>> so right now I cannot get the basic example working =(
>>
>>
>>>
>>>
>>> \--
>>>
>>> Ryan (????)
>>>
>>> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
>>>
>>> <http://refi64.com/>
>>>
>>>
>>>
>>> On Jan 17 2017, at 11:53 am, Tobias Oberstein <tobias.oberstein at gmail.com
>>>>
>>> wrote:
>>>
>>> Hi guys,
>>>>
>>>
>>>
>>>>
>>> please see (reposting here to grab attention as hinted by Armin)
>>>>
>>>
>>>
>>>>
>>> https://bitbucket.org/pypy/pypy/issues/2467/cppyy-fails-to-build-on-gcc-5
>>>>
>>> -and-clang
>>>
>>>
>>>>
>>> Note: for now I am fine, I managed to build it using gcc 4.9 toolchain.
>>>>
>>> Need to see how far I get with captnproto now ..
>>>
>>>
>>>>
>>> Cheers,
>>>>
>>> /Tobias
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>>
>


From tobias.oberstein at gmail.com  Tue Jan 17 16:30:42 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 22:30:42 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
Message-ID: <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>

Hi Ryan,

Am 17.01.2017 um 20:56 schrieb Ryan Gonzalez:
> This one looks like a bug in cppyy!
> Try going to clingcwrapper.cxx and changing line 525 from:

Yeah, that helps: the issue at line 525 is gone, but the 2nd issue at 
line 715 persists:

 
/tmp/pip-BVnCrW-build/src/backend/cppyy/src/clingcwrapper.cxx:715:35: 
error: invalid operands to binary expression ('basic_ostream<char, 
std::char_traits<char> >' and 'std::ostringstream' (aka 
'basic_ostringstream<char>'))
              std::cerr << "Warning: " << msg << '\n';
              ~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~

https://gist.github.com/oberstet/271719f569087f423197b6d8808ea6f1

/Tobias

From tobias.oberstein at gmail.com  Tue Jan 17 17:42:08 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Tue, 17 Jan 2017 23:42:08 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
Message-ID: <57feea7b-fd09-43b0-a3e3-63879db157b1@gmail.com>

Am 17.01.2017 um 22:39 schrieb Ryan Gonzalez:
> Whoops, missed that one! Change line 715 to:
> std::cerr << "Warning: " << msg.str() << '\n';

Success!

With these 2 fixes, it now builds using clang (with GNU libstdc++):

     -- Set runtime path of 
"/home/oberstet/pypy-5.6-linux_x86_64-portable/site-packages/cppyy_backend/lib/libcppyy_backend.so" 
to ""
     install finished
done
   Removing source in /tmp/pip-YrtrlD-build
Successfully installed PyPy-cppyy-backend-6.6.9.0
Cleaning up...
oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1/PyPy-cppyy-backend-6.6.9.0$

--

However, the basic example spills out weird things:

oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$ 
pypy MyClass.py
unknown type code: P
1
unknown type code: ?
-178968160
77


Here:

https://gist.github.com/oberstet/a3814c77e5400df3456d7ac74a26bdf7

Code is all here:

https://github.com/oberstet/scratchbox/tree/master/cpp/capnproto/test1

Example was built like this:

https://github.com/oberstet/scratchbox/tree/master/cpp/capnproto/test1#test

--

>
>
>
>
> \--
>
> Ryan (????)
>
> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
>
> <http://refi64.com/>
>
>
> On Jan 17 2017, at 3:30 pm, Tobias Oberstein <tobias.oberstein at gmail.com>
> wrote:
>
>> Hi Ryan,
>
>>
>
>> Am 17.01.2017 um 20:56 schrieb Ryan Gonzalez:
>> This one looks like a bug in cppyy!
>> Try going to clingcwrapper.cxx and changing line 525 from:
>
>>
>
>> Yeah, that helps: the issue at line 525 is gone, but the 2nd issue at
> line 715 persists:
>
>>
>
>>
> /tmp/pip-BVnCrW-build/src/backend/cppyy/src/clingcwrapper.cxx:715:35:
> error: invalid operands to binary expression ('basic_ostream<char,
> std::char_traits<char> >' and 'std::ostringstream' (aka
> 'basic_ostringstream<char>'))
>               std::cerr << "Warning: " << msg << '\n';
>               ~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~
>
>>
>
>> https://gist.github.com/oberstet/271719f569087f423197b6d8808ea6f1
>
>>
>
>> /Tobias
>


From armin.rigo at gmail.com  Wed Jan 18 03:11:51 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Wed, 18 Jan 2017 09:11:51 +0100
Subject: [pypy-dev] os_read & rposix.c_read
In-Reply-To: <AEA9B4E3-2CF3-4DF2-8A51-1C31BAB16DE0@anu.edu.au>
References: <AEA9B4E3-2CF3-4DF2-8A51-1C31BAB16DE0@anu.edu.au>
Message-ID: <CAMSv6X3BwgDex9e21-TFx0xT8koKewwrNbAnUAhzwHMHWrgyAw@mail.gmail.com>

Hi John,

On 18 January 2017 at 01:20, John Zhang <John.Zhang at anu.edu.au> wrote:
> Anyone knows why pypy.module._file.readinto uses a locally defined ?os_read?
> rather than rpython.rlib.rposix.c_read?

I think it's only because rposix.c_read is supposed to be private.
But yes, if it makes your life easier we can as well call it anyway
from pypy.module._file.readinto.  Maybe you should open a pull
request.


A bient?t,

Armin.

From tobias.oberstein at gmail.com  Wed Jan 18 04:44:08 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Wed, 18 Jan 2017 10:44:08 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>
Message-ID: <c709e7ac-036e-1a11-3f18-f380dfdb5fa2@gmail.com>

Hi Wim,

Am 18.01.2017 um 01:34 schrieb wlavrijsen at lbl.gov:
> Hi Ryan, Tobias,
>
> On Tuesday 2017-01-17 21:39, Ryan Gonzalez wrote:
>> Whoops, missed that one! Change line 715 to:
>> std::cerr << "Warning: " << msg.str() << '\n';
>
> indeed, thanks! Now fixed in the repository. Hopefully up soon in the pip
> (takes more testing, hence time) ...

Awesome, thanks!

However, I still can't get the basic example working =(

It spills out weird things:

oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$ 
pypy MyClass.py
unknown type code: P
1
unknown type code: ?
-178968160
77

This doesn't seem right.

Here:

https://gist.github.com/oberstet/a3814c77e5400df3456d7ac74a26bdf7

Code is all here:

https://github.com/oberstet/scratchbox/tree/master/cpp/capnproto/test1

Example was built like this:

https://github.com/oberstet/scratchbox/tree/master/cpp/capnproto/test1#test


Cheers,
/Tobias

>
> Best regards,
>            Wim


From anto.cuni at gmail.com  Wed Jan 18 05:02:29 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 18 Jan 2017 10:02:29 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
Message-ID: <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>

Hi Tobias,

On Tue, Jan 17, 2017 at 7:27 PM, Tobias Oberstein <
tobias.oberstein at gmail.com> wrote:

> Hi Antonio,
>
> Fantastic!!
>
> Actually, if this is true (performance), this would be wonderful - I can
> spare my nerves (hello C++, you are a moooonster!), and eat my cake ..
>
> The benchmarks
>
> http://capnpy.readthedocs.io/en/latest/benchmarks.html
>
> for scalar attribute access look awesome (close to mere instance access).
>
> Structured value access (lists/dict): bigger difference. I am also
> concerned about GC pressure - is it still "zero copy"?
>

?yes: under the hood, capnpy objects are represented as immutable strings;
then the generated classes contain accessors which look like this (overly
simplified):

class Point(_Struct):
    @property
    def x?(self):
        offset = statically_known_x_offset + self._offset
        return struct.unpack_from('q', self._buf.s, offset)

where self._buf.s is a string.
The nice thing is that the pypy JIT does a very good job at optimizing
struct.unpack_from: if you look at the generated code, you see that it
loads the 8 bytes directly from the in-memory buffer. So, it's very close
to optimal performance.

For lists is the same: when you look up a list field, you get a wrapper
around the very same buffer, then the custom __getitem__ does the actual
lookup in memory. Note that currently lists on pypy are slowish, but I
should be able to solve the problem in the near future.

The only exception are strings: getting a text attribute means to take a
slice of the buffer. However, if the returned value is short-lived, the JIT
might be able to optimize the slicing away.
(note: there is no automatic conversion to unicode, although I might add an
option for that in the future).

I don't understand what you mean by dicts are there are no dictionaries in
capnproto.

Note also that capnpy is serialization only: there is no support for RPC
stuff.



> Note: I am purely interested in performance on PyPy ..
>
> In general: I thought it would be a good idea to use capnproto C++
> generator, and then cppyy to get the best performance (on pypy). Given
> there is "antocuni/capnpy", do you think this is a pointless endeavour?
>

The original goal of capnpy was to be as fast as possible on PyPy. However,
if you find that C++ + cppyy is faster, I'd be very interested to know :).

ciao,
Anto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170118/a8e882e3/attachment.html>

From william.leslie.ttg at gmail.com  Wed Jan 18 08:44:04 2017
From: william.leslie.ttg at gmail.com (William ML Leslie)
Date: Thu, 19 Jan 2017 00:44:04 +1100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
Message-ID: <CAHgd1hHGnUxx0JHtEcKxXkO0ywKzvXR789xHnGMZbZ=FyR2Wqg@mail.gmail.com>

On 18 January 2017 at 21:02, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Hi Tobias,
>
> Note also that capnpy is serialization only: there is no support for RPC
> stuff.
>

Although if you want to work on it, I've got some preliminary support
for it in my fork that I mostly ripped from an internal CapTP
implementation I wrote at work.  I wasn't going to look at it in the
short term, but if there's interest I can get back to it, fix
interface generation and rewrite it to be SansIO but otherwise feel
free to use whatever code you like from it.

https://bitbucket.org/william_ml_leslie/capnpy

I'd love too see CapnProto hooked up to Autobahn/WS on Twisted too!  I
don't think there's a good client-side implementation of CapnProto
though, so there's still a missing part of that pipeline.

-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely MAY reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
prior contractual agreement.

From wlavrijsen at lbl.gov  Wed Jan 18 12:09:44 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 18 Jan 2017 09:09:44 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1701180859300.2757@cp3-jyni>

Hi Anto,

On Wednesday 2017-01-18 10:02, Antonio Cuni wrote:
> The original goal of capnpy was to be as fast as possible on PyPy. However,
> if you find that C++ + cppyy is faster, I'd be very interested to know :).

since you wrote the initial data access part yourself way back when, I'd
expect you to fix cppyy if it were slower! :)

(Only changes I made is to support multiple virtual inheritance and to
specialize to make sure the common case is not slowed down by that.)

But the only time that I have found where C++/cppyy is faster on PyPy than
pure Python is for data member access precisely b/c the C++ structs have
fixed offsets, and you seem to have covered that case by specializing.

See this report, figure 4:

   http://conferences.computer.org/pyhpc/2016/papers/5220a027.pdf

But, in cppyy, if there is string handling going on, with C-strings being
a stand-in for "byte*" (a type that was again voted down, so is not coming
soon), there may be spurious copies if not otherwise specialized.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From tobias.oberstein at gmail.com  Wed Jan 18 12:27:06 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Wed, 18 Jan 2017 18:27:06 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
Message-ID: <74ce16a7-1619-1f65-73cd-81e56e447f08@gmail.com>

Hi Antonio,

> The only exception are strings: getting a text attribute means to take a
> slice of the buffer. However, if the returned value is short-lived, the JIT
> might be able to optimize the slicing away.
> (note: there is no automatic conversion to unicode, although I might add an
> option for that in the future).

Are you aware of

https://github.com/alex/zero_buffer

?

This emulates Python strings using zero-copy read-only buffer views.

I think zero_buffer only supports byte strings not unicode.

But for that, maybe it would be cool to have capnpy return zero_buffers 
upon byte string attribute access, because that should finally provide 
zero copy ..

Caveat: this is a hypothesis I haven't really explored / thought through ..

--

Holy grail for me (use case being IPC) would be:

Python process 1 mmap's a file shared with Python process 2.

Python process 1 puts a string into mmap'ed file, pointer to that is 
"somehow transferred" to process 2 (eg pushing the index into the 
mmap'ed file over Unix domain socket .. a single uint64), and Python 
code in process 2 can do stuff with this string _without_ copying - 
probably via zero_buffer.

> I don't understand what you mean by dicts are there are no dictionaries in
> capnproto.

Right. Structs.

>
> Note also that capnpy is serialization only: there is no support for RPC
> stuff.

That's fine .. I don't need RPC (we have WAMP for that in 
Autobahn/Crossbar.io).

>> Note: I am purely interested in performance on PyPy ..
>>
>> In general: I thought it would be a good idea to use capnproto C++
>> generator, and then cppyy to get the best performance (on pypy). Given
>> there is "antocuni/capnpy", do you think this is a pointless endeavour?
>>
>
> The original goal of capnpy was to be as fast as possible on PyPy. However,
> if you find that C++ + cppyy is faster, I'd be very interested to know :).

Sure .. right now I cannot progress on cppyy as I can't get the basic 
example running..

Cheers,
/Tobias

> ciao,
> Anto
>


From tobias.oberstein at gmail.com  Wed Jan 18 12:32:20 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Wed, 18 Jan 2017 18:32:20 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHgd1hHGnUxx0JHtEcKxXkO0ywKzvXR789xHnGMZbZ=FyR2Wqg@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
 <CAHgd1hHGnUxx0JHtEcKxXkO0ywKzvXR789xHnGMZbZ=FyR2Wqg@mail.gmail.com>
Message-ID: <092868fe-8a57-8bd5-f279-86acc99fcc08@gmail.com>

Hi William,

Am 18.01.2017 um 14:44 schrieb William ML Leslie:
> On 18 January 2017 at 21:02, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> Hi Tobias,
>>
>> Note also that capnpy is serialization only: there is no support for RPC
>> stuff.
>>
>
> Although if you want to work on it, I've got some preliminary support
> for it in my fork that I mostly ripped from an internal CapTP
> implementation I wrote at work.  I wasn't going to look at it in the
> short term, but if there's interest I can get back to it, fix
> interface generation and rewrite it to be SansIO but otherwise feel
> free to use whatever code you like from it.
>
> https://bitbucket.org/william_ml_leslie/capnpy
>
> I'd love too see CapnProto hooked up to Autobahn/WS on Twisted too!  I
> don't think there's a good client-side implementation of CapnProto
> though, so there's still a missing part of that pipeline.
>

Thanks for the pointer!

I don't have a use case for the RPC part of Captnproto though.

Autobahn/Crossbar.io has WAMP, which already has RPC, more so _routed 
RPC_. And there are more than a dozen client side WAMP implementations.

I just need a ultra-fast way for IPC between Python processes .. doing 
away with serialization/deserialization overhead and copying.

Cheers,
/Tobias

From wlavrijsen at lbl.gov  Wed Jan 18 12:34:50 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 18 Jan 2017 09:34:50 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <c709e7ac-036e-1a11-3f18-f380dfdb5fa2@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>
 <c709e7ac-036e-1a11-3f18-f380dfdb5fa2@gmail.com>
Message-ID: <alpine.LSU.2.11.1701180920560.2757@cp3-jyni>

Hi Tobias,

> unknown type code: P
> 1
> unknown type code: ???
> -178968160
> 77

that falls in the category "should never happen." Arguments are passed as
a patch of memory, with a type code indicating how the individual arguments
should be treated. The only way this can be wrong, is if the offset is
wrong. I don't understand how that can happen: the offset is provided at
run-time by the backend, where it is compiled in. The pypy-side module
then uses that offset, so there should be only one value throughout.

Still, is pypy-c build with a different compiler than the pip? Not that I
think that's the problem, but maybe it provides some clues.

Also, can you try the following (this is not a public interface):

% pypy-c
>>>> import cppyy
>>>> cppyy.gbl.gInterpreter.Declare("""class MyClass {
.... public:
....     MyClass(int i = -99) : m_myint(i) {}
....
....     int GetMyInt() { return m_myint; }
....         void SetMyInt(int i) { m_myint = i; }
....
.... public:
....     int m_myint;
....     };""")
True
>>>> from cppyy.gbl import MyClass
>>>> m = MyClass(42)
>>>> print m.m_myint
42
>>>>

(The reason this is not a public interface, is b/c I want to hide all this
behind an interface that mimics cffi, call it cppffi. Lacking time, though,
so if anyone is interested? :) )

Anyway, I expect it to give the same problem, but if it does, it excludes
one other place where the bug could hide.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From tobias.oberstein at gmail.com  Wed Jan 18 12:58:02 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Wed, 18 Jan 2017 18:58:02 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701180920560.2757@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>
 <c709e7ac-036e-1a11-3f18-f380dfdb5fa2@gmail.com>
 <alpine.LSU.2.11.1701180920560.2757@cp3-jyni>
Message-ID: <f29947af-e43c-ecf2-1f47-ceca399914b8@gmail.com>

Hi Wim,

Am 18.01.2017 um 18:34 schrieb wlavrijsen at lbl.gov:
> Hi Tobias,
>
>> unknown type code: P
>> 1
>> unknown type code: ???
>> -178968160
>> 77
>
> that falls in the category "should never happen." Arguments are passed as
> a patch of memory, with a type code indicating how the individual arguments
> should be treated. The only way this can be wrong, is if the offset is
> wrong. I don't understand how that can happen: the offset is provided at
> run-time by the backend, where it is compiled in. The pypy-side module
> then uses that offset, so there should be only one value throughout.
>
> Still, is pypy-c build with a different compiler than the pip? Not that I

Yes. I am using

https://github.com/squeaky-pl/portable-pypy

and this is built with GCC 6.2.0.

As there isn't a standard C++ ABI, this would explain all sorts of weird 
things happening.

Or does clang deliberately try to follow gcc ABI?

I am aware that the term "C++ ABI" is an oxymoron in a way .. what's the 
ABI of a templated template? =)

>
> Also, can you try the following (this is not a public interface):

Doesn't work for me:

oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$ 
pypy MyClass.py
unknown type code: ?
1
unknown type code:
1294245728
77

oberstet at thinkpad-t430s:~/scm/oberstet/scratchbox/cpp/capnproto/test1$ pypy
Python 2.7.12 (aff251e543859ce4508159dd9f1a82a2f553de00, Nov 12 2016, 
08:50:18)
[PyPy 5.6.0 with GCC 6.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>>> import cppyy
 >>>> cppyy.gbl.gInterpreter.Declare("""
.... class MyClass {
.... public:
....     MyClass(int i = -99) : m_myint(i) {}
....         int GetMyInt() { return m_myint; }
....             void SetMyInt(int i) { m_myint = i; }
....             public:
....                 int m_myint;
....                 };""")
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
AttributeError: <class '__main__.::'> object has no attribute 
'gInterpreter' (details: "function 'cppyy_get_template' not found in 
library 'libcppyy_backend.so'")
 >>>>

--

Given all these problems even running a basic example (I haven't even 
got to capnproto generated C++ .. and this is heavily templated code, so 
even more will go wrong), and given that I have a specific use case 
(captnproto) rather than a general need to wrap C++, I think I will 
evaluate capnpy ..

--

But I think we didn't waste time:

- 2 bugs identified and fixed in cppyy
- using clang _might_ work, but pypy binaries are gcc most of the time, 
and that leads to more issues

So I think gcc 5 support is more important:

"Found GCC compiler with new ABI. Cling does not support the GCC 5 ABI yet"

The bundled Cling is too old? I am totally guessing here .. in fact I 
don't really know what I am doing;)

Cheers,
/Tobias


From anto.cuni at gmail.com  Wed Jan 18 13:38:01 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 18 Jan 2017 18:38:01 +0000
Subject: [pypy-dev] numpypy include files
Message-ID: <CAHPg78x9TmcX9KGi2iVpLW5NZ2vE1JC-yrbf-VOwXoi4y33u1g@mail.gmail.com>

Hello Matti,
I am having some troubles with the latest pypy and numpypy: basically,
numpy.get_include() returns the wrong directory.

I investigated a bit and I think the culprit are these two commits:
https://bitbucket.org/pypy/pypy/commits/ad36a29d0fcc
https://bitbucket.org/pypy/numpy/commits/26e09b343f

Probably it's just a typo in the numpy commit, which specify '_numpy'
instead of '_numpypy'.

However, before committing the fix I wanted to ask you, to make sure I am
not missing anything important :).

ciao,
Anto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170118/94c011be/attachment.html>

From wlavrijsen at lbl.gov  Wed Jan 18 13:40:52 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 18 Jan 2017 10:40:52 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <f29947af-e43c-ecf2-1f47-ceca399914b8@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>
 <c709e7ac-036e-1a11-3f18-f380dfdb5fa2@gmail.com>
 <alpine.LSU.2.11.1701180920560.2757@cp3-jyni>
 <f29947af-e43c-ecf2-1f47-ceca399914b8@gmail.com>
Message-ID: <alpine.LSU.2.11.1701181024390.2757@cp3-jyni>

Hi Tobias,

> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AttributeError: <class '__main__.::'> object has no attribute 'gInterpreter' 
> (details: "function 'cppyy_get_template' not found in library 
> 'libcppyy_backend.so'")

okay, problem understood: the version of pypy-c that you have does not
support the Cling backend (there is no release of that yet: next pypy).

See, 'cppyy_get_template' is from the Reflex era: it no longer exists. So
in your case the cppyy module (the RPython portion inside pypy-c) is doing
Reflex-thingies while being fed Cling-thingies from the (C++) backend.

The real bug here is that I need to have a version check in cppyy. I did
plan to, but didn't get around implementing it. Sorry about that!

Btw., if it interests you: the reason that 'cppyy_get_template' disappeared
is that Cling supports proper overloading of templated and non-templated
methods (including instantiation if necessary). So a returned method may
well be templated, and thus the relevant question is "is_template". That is,
if a method exists, it may be either templated or not, but an additional
call to look for a templated method would be pointless.

As for ABI-compatibility, it's not that bad: contemporary compilers on the
same OS are designed to be compatible. Their standard libraries are a
different matter. That said, cppyy per se should not be hampered by that,
as all the offsets are provided to the module by the backend at runtime.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From matti.picus at gmail.com  Wed Jan 18 16:05:05 2017
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 18 Jan 2017 23:05:05 +0200
Subject: [pypy-dev] numpypy include files
In-Reply-To: <CAHPg78x9TmcX9KGi2iVpLW5NZ2vE1JC-yrbf-VOwXoi4y33u1g@mail.gmail.com>
References: <CAHPg78x9TmcX9KGi2iVpLW5NZ2vE1JC-yrbf-VOwXoi4y33u1g@mail.gmail.com>
Message-ID: <7563fde6-74f3-3da6-4cff-3e720e39e752@gmail.com>

On 18/01/17 20:38, Antonio Cuni wrote:
> Hello Matti,
> I am having some troubles with the latest pypy and numpypy: basically, 
> numpy.get_include() returns the wrong directory.
>
> I investigated a bit and I think the culprit are these two commits:
> https://bitbucket.org/pypy/pypy/commits/ad36a29d0fcc
> https://bitbucket.org/pypy/numpy/commits/26e09b343f
>
> Probably it's just a typo in the numpy commit, which specify '_numpy' 
> instead of '_numpypy'.
>
> However, before committing the fix I wanted to ask you, to make sure I 
> am not missing anything important :).
>
> ciao,
> Anto
Indeed, the changeset https://bitbucket.org/pypy/numpy/commits/26e09b343f
seems to be an error, should be _numpypy. Interesting that you are the 
first to report it, I guess not many people try to build c-extensions 
with numpypy
<https://bitbucket.org/pypy/numpy/commits/26e09b343f>

From tobias.oberstein at gmail.com  Wed Jan 18 17:55:01 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Wed, 18 Jan 2017 23:55:01 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701181024390.2757@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <988dgpbansm663v5f80p3ovtk-0@mailer.nylas.com>
 <55899ca5-21ae-9bb6-48b7-9c1cb3a2a466@gmail.com>
 <CAO41-mNFjfsFp1XywNT3ey_ofMoArZn-5-TXot0Ef0qJYGB_Jw@mail.gmail.com>
 <35db07f3-3367-b5d8-1c9a-3a110e997abd@gmail.com>
 <axh48fwcsoc4v8exfsz5do459-0@mailer.nylas.com>
 <77f91500-7677-876f-6f42-06864cceca6a@gmail.com>
 <62ja7yqgrswoyrpxa0hd18t6o-0@mailer.nylas.com>
 <alpine.LSU.2.11.1701171622160.18301@cp3-jyni>
 <c709e7ac-036e-1a11-3f18-f380dfdb5fa2@gmail.com>
 <alpine.LSU.2.11.1701180920560.2757@cp3-jyni>
 <f29947af-e43c-ecf2-1f47-ceca399914b8@gmail.com>
 <alpine.LSU.2.11.1701181024390.2757@cp3-jyni>
Message-ID: <673a2538-3238-bed0-bcfa-0cae8e912969@gmail.com>

Hi Wim,

> okay, problem understood: the version of pypy-c that you have does not
> support the Cling backend (there is no release of that yet: next pypy).

Ah ..

> See, 'cppyy_get_template' is from the Reflex era: it no longer exists. So
> in your case the cppyy module (the RPython portion inside pypy-c) is doing
> Reflex-thingies while being fed Cling-thingies from the (C++) backend.
>
> The real bug here is that I need to have a version check in cppyy. I did
> plan to, but didn't get around implementing it. Sorry about that!

No problem .. great that it will be in the next release!

Also, fixing the gcc5 issue I have on Ubuntu 16.04 would be really worth 
it, not least because this is such a common (developer) platform nowerdays.

If that would have worked for me, I wouldn't have run down the clang 
rabbit hole;) Notwithstanding that clang/libc++ support would _also_ be 
cool (because of FreeBSD ..)

> As for ABI-compatibility, it's not that bad: contemporary compilers on the
> same OS are designed to be compatible. Their standard libraries are a
> different matter. That said, cppyy per se should not be hampered by that,
> as all the offsets are provided to the module by the backend at runtime.

Ok, sounds good .. I believe. I have skimmed into your paper .. heavy 
stuff .. you obviously know more than me here;) Sadly, I don't have time 
to really dig into it.

Thanks for all your assistence and time!

Cheers,
/Tobias

From anto.cuni at gmail.com  Thu Jan 19 05:04:44 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 19 Jan 2017 10:04:44 +0000
Subject: [pypy-dev] numpypy include files
In-Reply-To: <7563fde6-74f3-3da6-4cff-3e720e39e752@gmail.com>
References: <CAHPg78x9TmcX9KGi2iVpLW5NZ2vE1JC-yrbf-VOwXoi4y33u1g@mail.gmail.com>
 <7563fde6-74f3-3da6-4cff-3e720e39e752@gmail.com>
Message-ID: <CAHPg78wM76K0-ioNmCoNfmtNQVQ7O76fM0i-K=WpuOrG0N3PFg@mail.gmail.com>

Ok, thank you, I fixed&committed.
About me being the first to report: actually, I think it's because there is
a lot of confusion around about numpy+pypy.

I started to investigate this because I saw by chance a colleague switching
from installing numpypy to installing numpy. When I asked, he just said
that his C extension did no longer work with numpypy on newer versions of
PyPy. And as a result his code ran slower (because for his particular
workload numpypy is better).

So, lesson learnt:
  - people are confused about this numpy vs numpypy thing, Also, the fact
that the numpypy repo is called "numpy" does not help, as they tend to
think it's just an outdated copy of the offical numpy.

  - it's hard to find docs about this: the only docs I found are
http://pypy.org/download.html which is not the first place I'd look at :)

So: what is the current status of numpy vs numpypy? Is the latter still
maintained or is it slowly dying?

If numpypy is still relevant, I propose to hack things so that:

1) we have a dedicated page about numpy either on pypy.org or readthedocs

2) when you execute pip install numpy, it installs the upstream numpy but
also displays a link to that page

3) pip install numpypy works as well (and displays the same link).

What do you think?


On Wed, Jan 18, 2017 at 9:05 PM, Matti Picus <matti.picus at gmail.com> wrote:

> On 18/01/17 20:38, Antonio Cuni wrote:
>
>> Hello Matti,
>> I am having some troubles with the latest pypy and numpypy: basically,
>> numpy.get_include() returns the wrong directory.
>>
>> I investigated a bit and I think the culprit are these two commits:
>> https://bitbucket.org/pypy/pypy/commits/ad36a29d0fcc
>> https://bitbucket.org/pypy/numpy/commits/26e09b343f
>>
>> Probably it's just a typo in the numpy commit, which specify '_numpy'
>> instead of '_numpypy'.
>>
>> However, before committing the fix I wanted to ask you, to make sure I am
>> not missing anything important :).
>>
>> ciao,
>> Anto
>>
> Indeed, the changeset https://bitbucket.org/pypy/numpy/commits/26e09b343f
> seems to be an error, should be _numpypy. Interesting that you are the
> first to report it, I guess not many people try to build c-extensions with
> numpypy
> <https://bitbucket.org/pypy/numpy/commits/26e09b343f>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170119/05f93a12/attachment-0001.html>

From anto.cuni at gmail.com  Thu Jan 19 05:52:06 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 19 Jan 2017 10:52:06 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <alpine.LSU.2.11.1701180859300.2757@cp3-jyni>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
 <alpine.LSU.2.11.1701180859300.2757@cp3-jyni>
Message-ID: <CAHPg78wB6K9LKgcZneJU+RhUFHtTioPdj2KYntvTRCC9otbiuw@mail.gmail.com>

Hi Wim,

On Wed, Jan 18, 2017 at 5:09 PM, <wlavrijsen at lbl.gov> wrote:


> since you wrote the initial data access part yourself way back when, I'd
> expect you to fix cppyy if it were slower! :)
>
>
?that's a fair point, indeed :).
But on top of that you need to put a layer which exposes a pythonic
interface (for example, offering list-like classes with an __iter__ and a
__getitem__). So I have no idea of how the final speed of the thing will
be, until someone tries and measure :).

?ciao,
Anto?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170119/d7c5d747/attachment.html>

From anto.cuni at gmail.com  Thu Jan 19 06:00:15 2017
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 19 Jan 2017 11:00:15 +0000
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <74ce16a7-1619-1f65-73cd-81e56e447f08@gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
 <74ce16a7-1619-1f65-73cd-81e56e447f08@gmail.com>
Message-ID: <CAHPg78xXjrqGtespLysiak71xf0-TaADpfv7zyzzQummXKh6Lg@mail.gmail.com>

Hi Tobias,

On Wed, Jan 18, 2017 at 5:27 PM, Tobias Oberstein <
tobias.oberstein at gmail.com> wrote:

> Are you aware of
>
> https://github.com/alex/zero_buffer
>
> ?
>
> This emulates Python strings using zero-copy read-only buffer views.
>

?yes, I saw it in the past and I considered to use it for capnpy. IIRC, I
measured that at the end of the days, the overhead of using it was larger
than simply doing string slicing, especially for short strings.
It might be useful for very large strings, however.

Depending of what you need to do, for zero copy you could also consider
returning a memoryview slice of the original underlying buffer.



> Holy grail for me (use case being IPC) would be:
>
> Python process 1 mmap's a file shared with Python process 2.
>
> Python process 1 puts a string into mmap'ed file, pointer to that is
> "somehow transferred" to process 2 (eg pushing the index into the mmap'ed
> file over Unix domain socket .. a single uint64), and Python code in
> process 2 can do stuff with this string _without_ copying - probably via
> zero_buffer.
>
>
?Have you actually measured that copying the data between processes is the
bottleneck? Using shared memory is something I tried also for a client of
mine but at the end we switched back to pass messages using network because
the extra complexity was not worth the gain. But again, I suppose it
depends on the size of the message.

?ciao,
Anto?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170119/dd553671/attachment.html>

From tobias.oberstein at gmail.com  Thu Jan 19 06:35:32 2017
From: tobias.oberstein at gmail.com (Tobias Oberstein)
Date: Thu, 19 Jan 2017 12:35:32 +0100
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHPg78xXjrqGtespLysiak71xf0-TaADpfv7zyzzQummXKh6Lg@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
 <74ce16a7-1619-1f65-73cd-81e56e447f08@gmail.com>
 <CAHPg78xXjrqGtespLysiak71xf0-TaADpfv7zyzzQummXKh6Lg@mail.gmail.com>
Message-ID: <09683d50-0a2f-1746-e147-ddf6fccecc7a@gmail.com>

>> Holy grail for me (use case being IPC) would be:
>>
>> Python process 1 mmap's a file shared with Python process 2.
>>
>> Python process 1 puts a string into mmap'ed file, pointer to that is
>> "somehow transferred" to process 2 (eg pushing the index into the mmap'ed
>> file over Unix domain socket .. a single uint64), and Python code in
>> process 2 can do stuff with this string _without_ copying - probably via
>> zero_buffer.
>>
>>
> ?Have you actually measured that copying the data between processes is the
> bottleneck? Using shared memory is something I tried also for a client of
> mine but at the end we switched back to pass messages using network because
> the extra complexity was not worth the gain. But again, I suppose it
> depends on the size of the message.

I have measured (and profiled using vmprof) that we have bottlenecks in 
serialization/deserialization .. the overhead is lowest (on PyPy) when 
using a pure CBOR implementation, but still.

Hence my interest in captnproto/flatbuffers.

I also benchmarked Unix domain socket pure bandwidth (in C) .. which is 
very high indeed (UDS only involves a single copy when done in C, not 2 
copies like loopback TCP).

I probably should stop worrying/guessing and try capnpy over UDS and see 
how far this will get us;)

Cheers,
/Tobias


>
> ?ciao,
> Anto?
>


From armin.rigo at gmail.com  Thu Jan 19 09:55:06 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Thu, 19 Jan 2017 15:55:06 +0100
Subject: [pypy-dev] numpypy include files
In-Reply-To: <CAHPg78wM76K0-ioNmCoNfmtNQVQ7O76fM0i-K=WpuOrG0N3PFg@mail.gmail.com>
References: <CAHPg78x9TmcX9KGi2iVpLW5NZ2vE1JC-yrbf-VOwXoi4y33u1g@mail.gmail.com>
 <7563fde6-74f3-3da6-4cff-3e720e39e752@gmail.com>
 <CAHPg78wM76K0-ioNmCoNfmtNQVQ7O76fM0i-K=WpuOrG0N3PFg@mail.gmail.com>
Message-ID: <CAMSv6X3VsT+KBzBP1xE=Z1qjJGm5ofp96SNG=_hmRfkNn0WB2Q@mail.gmail.com>

Hi Anto,

On 19 January 2017 at 11:04, Antonio Cuni <anto.cuni at gmail.com> wrote:
>   - it's hard to find docs about this: the only docs I found are
> http://pypy.org/download.html which is not the first place I'd look at :)

Yes, I added this piece of doc recently.  It was only a minimal
attempt to fix the previously entirely undocumented state of the
universe.  And yes, definitely please document it more widely at
places that make sense to someone that comes to PyPy with the
intention to install numpy and no prior knowledge about the existence
of such a double implementation.

It would be cool to have some feedback from someone who actually works
on pypy+numpy this time. :-)


A bient?t,

Armin.

From wlavrijsen at lbl.gov  Thu Jan 19 18:12:55 2017
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Thu, 19 Jan 2017 15:12:55 -0800 (PST)
Subject: [pypy-dev] cppyy fails to build on gcc 5 and clang
In-Reply-To: <CAHPg78wB6K9LKgcZneJU+RhUFHtTioPdj2KYntvTRCC9otbiuw@mail.gmail.com>
References: <df047149-8840-c89e-9353-2132b4f6c7d4@gmail.com>
 <CAHPg78xWk8+Ckv-UKL9gJ838rosTQzgzvrJxgYqNR8RKf3wqRw@mail.gmail.com>
 <f3351315-8a52-418e-8005-f4c1589b93a4@gmail.com>
 <CAHPg78w8uxu7XkFx=JDK9AHqFq13u-SOVHz9z4Xf_0poawWbBA@mail.gmail.com>
 <alpine.LSU.2.11.1701180859300.2757@cp3-jyni>
 <CAHPg78wB6K9LKgcZneJU+RhUFHtTioPdj2KYntvTRCC9otbiuw@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1701191510480.2757@cp3-jyni>

Anto,

On Thursday 2017-01-19 10:52, Antonio Cuni wrote:
> But on top of that you need to put a layer which exposes a pythonic
> interface (for example, offering list-like classes with an __iter__ and a
> __getitem__). So I have no idea of how the final speed of the thing will
> be, until someone tries and measure :).

yes, I did something like that for std::vector (C++11 guarantees that the
memory it contains is contiguous). Still, I'm going to compare what I did
with your code as std::vector in C++ is as efficient as it gets, so there
is a lot of ground to cover.

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From frankw at mit.edu  Thu Jan 19 23:05:42 2017
From: frankw at mit.edu (Frank Wang)
Date: Thu, 19 Jan 2017 23:05:42 -0500
Subject: [pypy-dev] Compiling PyPy without JIT
Message-ID: <CAD=7XjNtDZqbgY59HnPQ-PzOV6xWfQYs1wg9FAdyPD4kc+kZ=Q@mail.gmail.com>

Hi,

When I compile PyPy in the "goal" folder, I compile it with the following
options:

pypy ../../rpython/bin/rpython --opt=2

Is this the proper way to compile pypy without JIT, or am I doing something
wrong?

Thanks,
Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170119/135e16d2/attachment.html>

From armin.rigo at gmail.com  Fri Jan 20 03:21:32 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Fri, 20 Jan 2017 09:21:32 +0100
Subject: [pypy-dev] Compiling PyPy without JIT
In-Reply-To: <CAD=7XjNtDZqbgY59HnPQ-PzOV6xWfQYs1wg9FAdyPD4kc+kZ=Q@mail.gmail.com>
References: <CAD=7XjNtDZqbgY59HnPQ-PzOV6xWfQYs1wg9FAdyPD4kc+kZ=Q@mail.gmail.com>
Message-ID: <CAMSv6X3GfUKNCEGBT0xh0fkJh_Kp-ZwUeMRAZfRw=d-RrWjcpw@mail.gmail.com>

Hi Frank,

On 20 January 2017 at 05:05, Frank Wang <frankw at mit.edu> wrote:
> When I compile PyPy in the "goal" folder, I compile it with the following
> options:
>
> pypy ../../rpython/bin/rpython --opt=2
>
> Is this the proper way to compile pypy without JIT, or am I doing something
> wrong?

That's correct.  http://pypy.org/download.html#building-from-source
specifies "-O2" instead of "--opt=2", but they are fully equivalent.
And the 'targetpypystandalone.py' part is optional unless you want to
give more PyPy-specific options afterwards.


A bient?t,

Armin.

From armin.rigo at gmail.com  Fri Jan 20 07:43:08 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Fri, 20 Jan 2017 13:43:08 +0100
Subject: [pypy-dev] Leysin sprint?
In-Reply-To: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
References: <CAMSv6X0s0bL6BkJ33k_yMwKwAhDONYnEqHy82-A0dmO_+M3N+Q@mail.gmail.com>
Message-ID: <CAMSv6X0U52OJYNxVx3aOE2TnczV8zWhYp7E8W+eTJz=i-Ub8hA@mail.gmail.com>

Hi all,

I've pre-booked the Leysin sprint from the 25/26th of February to the
4th of March.  Please tell me if there is some issue with these dates.
Otherwise I'll make an official announcement next week.  See you soon!

Armin

From armin.rigo at gmail.com  Tue Jan 24 06:06:54 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Tue, 24 Jan 2017 12:06:54 +0100
Subject: [pypy-dev] PyPy Leysin Winter Sprint: 25/26th Feb. - 4th March 2017
Message-ID: <CAMSv6X1b_4N64JQYH_q27UG1ZoNKjKpgAdBtO5dX3TXjjw2YGg@mail.gmail.com>

Hi all,

The next PyPy sprint will be in Leysin, Switzerland, for the twelveth time.
This is a fully public sprint: newcomers and topics other than those
proposed below are welcome.

Work days: starting 26th Feb (~noon), ending March 4th (~noon).

----------------------------------------
Goals and topics of the sprint
----------------------------------------

The list of topics is very open.

* The main topic is Python 3.5 support in PyPy, as most py3.5
  contributors should be present.  It is also a good topic if you have
  no or limited experience with PyPy contribution: we can easily find
  something semi-independent that is not done in py3.5 so far, and
  do pair-programming with you.

* Any other topic is fine too: JIT compiler optimizations, CFFI,
  the RevDB reverse debugger, improving to speed of your program on
  PyPy, etc.

* And as usual, the main side goal is to have fun in winter sports :-)
  We can take a day off (for ski or anything else).

-------------
More info
-------------

For more information, see the complete announcement at:

   http://bitbucket.org/pypy/extradoc/raw/extradoc/sprintinfo/leysin-winter-2017/announcement.txt


A bient?t,

Armin.

From christoph.lassner at tuebingen.mpg.de  Wed Jan 25 06:32:24 2017
From: christoph.lassner at tuebingen.mpg.de (Christoph Lassner)
Date: Wed, 25 Jan 2017 12:32:24 +0100
Subject: [pypy-dev] pypy tensorflow compatibility
Message-ID: <87wpdjqss7.fsf@aragorn-g.i-did-not-set--mail-host-address--so-tickle-me>

Dear pypy devs,

I'm a big fan of pypy's ideas and was very happy to see the great
improvements introduced with the cpyext library in the latest release.
I could out-of-the box compile numpy, scipy and scikit learn, which
allows to use pypy easily for machine learning.

Tensorflow would be another great package to use for deep learning and
other tensor computations. It builds out-of-the box, however it is not
possible to import the SWIG generated python module. Looking at it with
`ldd -r`, several undefined symbols become apparent, hinting at a
missing linked file.

The missing symbols are, for example:

PyPyInt_FromLong
PyPyList_SET_ITEM
PyPyErr_Occurred
_PyPyInstance_Lookup
PyPyDict_SetItem
...

so it looks like the pypy standard library it `missing'. Which file
would have to be linked against to resolve them?

For reference, I also commented on this github issue:
https://github.com/tensorflow/tensorflow/issues/252 .

Thank you for the infos!

Best,
Christoph

From armin.rigo at gmail.com  Wed Jan 25 07:22:56 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Wed, 25 Jan 2017 13:22:56 +0100
Subject: [pypy-dev] pypy tensorflow compatibility
In-Reply-To: <87wpdjqss7.fsf@aragorn-g.i-did-not-set--mail-host-address--so-tickle-me>
References: <87wpdjqss7.fsf@aragorn-g.i-did-not-set--mail-host-address--so-tickle-me>
Message-ID: <CAMSv6X0fCaHX5y8PA15uEndD4PPaX44Q-=Pq4stv88tEcE13-A@mail.gmail.com>

Hi,

On 25 January 2017 at 12:32, Christoph Lassner
<christoph.lassner at tuebingen.mpg.de> wrote:
> The missing symbols are, for example:
>
> PyPyInt_FromLong
> PyPyList_SET_ITEM
> PyPyErr_Occurred
> _PyPyInstance_Lookup
> PyPyDict_SetItem
> ...

These are all symbols defined in the standard libpypy-c.so.  I don't
know why in your case the linking is not working.


Armin

From christoph.lassner at tuebingen.mpg.de  Wed Jan 25 12:08:56 2017
From: christoph.lassner at tuebingen.mpg.de (Christoph Lassner)
Date: Wed, 25 Jan 2017 18:08:56 +0100
Subject: [pypy-dev] pypy tensorflow compatibility
In-Reply-To: <CAMSv6X0fCaHX5y8PA15uEndD4PPaX44Q-=Pq4stv88tEcE13-A@mail.gmail.com>
References: <87wpdjqss7.fsf@aragorn-g.i-did-not-set--mail-host-address--so-tickle-me>
 <CAMSv6X0fCaHX5y8PA15uEndD4PPaX44Q-=Pq4stv88tEcE13-A@mail.gmail.com>
Message-ID: <874m0n83tj.fsf@gepard.i-did-not-set--mail-host-address--so-tickle-me>

Hi Armin,

the Tensorflow build is quite a process driven by the bazel build
engine and not standard python/pypy tools. My guess is, that it just
misses out on that file.

Thank you for the very quick response, will keep you posted!

Best,
Christoph

Armin Rigo <armin.rigo at gmail.com> writes:

> Hi,
>
> On 25 January 2017 at 12:32, Christoph Lassner
> <christoph.lassner at tuebingen.mpg.de> wrote:
>> The missing symbols are, for example:
>>
>> PyPyInt_FromLong
>> PyPyList_SET_ITEM
>> PyPyErr_Occurred
>> _PyPyInstance_Lookup
>> PyPyDict_SetItem
>> ...
>
> These are all symbols defined in the standard libpypy-c.so.  I don't
> know why in your case the linking is not working.
>
>
> Armin

From william.leslie.ttg at gmail.com  Fri Jan 27 08:50:30 2017
From: william.leslie.ttg at gmail.com (William ML Leslie)
Date: Sat, 28 Jan 2017 00:50:30 +1100
Subject: [pypy-dev] Fwd: [MoreVMs'17] Call for Contributions: 1st Workshop
 on Modern Language Runtimes, Ecosystems, and VMs at <Programming> 2017
In-Reply-To: <0309BE8B-DBE1-42AF-B194-B920E730DAD3@stefan-marr.de>
References: <0309BE8B-DBE1-42AF-B194-B920E730DAD3@stefan-marr.de>
Message-ID: <CAHgd1hEe8dAX-bUfTUpcjOrWXpYNL8vfsxtsq35Tzi7LghmhCA@mail.gmail.com>

Hello pypy contributors and rpython users,

In case any of you missed this CFP this looks like an exciting workshop!


---------- Forwarded message ----------
From: Stefan Marr <java at stefan-marr.de>
Date: 6 December 2016 at 00:56
Subject: [MoreVMs'17] Call for Contributions: 1st Workshop on Modern
Language Runtimes, Ecosystems, and VMs at <Programming> 2017
To: mlvm-dev at openjdk.java.net


============================================================================
                        Call for Papers: MoreVMs?17

                              1st Workshop on
               Modern Language Runtimes, Ecosystems, and VMs

                     Co-located with <Programming> 2017
                       April, 2017, Brussels, Belgium

      http://2017.programming-conference.org/track/MoreVMs-2017-papers
============================================================================

The MoreVMs'17 workshop aims bring together programmers from industry and
academy to discuss the design, implementation, and usage of modern languages
and runtimes. This includes aspects such as reuse of language runtimes, modular
implementation, or design and compilation strategies to target existing
runtimes.

The main goals of the workshop is to bring together both researchers and
practitioners and facilitate effective sharing of their respective experiences
and ideas on how languages and runtimes are utilized and where they need to
improve further. We welcome presentation proposals in the form of extended
abstracts discussing experiences, work-in-progress, as well as future visions
from the academic as well as industrial perspective. Relevant topics include,
but are definitely not limited to, the following:

- extensible VM design (compiler- or interpreter-based VMs)
- reusable runtime components (e.g. interpreters, garbage collectors,
  intermediate representations)
- static and dynamic compiler techniques
- techniques for compilation to high-level languages such as JavaScript
- runtimes and mechanisms for interoperability between languages
- tooling support (e.g. debugging, profiling, etc.)
- programming language development environments and virtual machines
- case studies of existing language implementations, virtual machines, and
  runtime components (e.g. design choices, tradeoffs, etc.)
- language implementation challenges and trade-offs (e.g. performance,
completeness, etc.)
- surveys and applications usage reports to understand runtime usage in the wild
- surveys on frameworks and their impact on runtime usage
- new research ideas on how we want to build languages in the future

### Workshop Format and Submissions

This workshop welcomes the presentation and discussion of new ideas and
emerging problems to facilitate interaction among workshop participants and
exchange of ideas. We accept presentation proposals in the form of extended
abstracts (1-2 pages). Accepted abstracts will be published on the workshop's
website before the workshop date.

For preparing your abstract, please use the provided author kit:
https://github.com/smarr/morevms17-author-kit. It is based on the ACM SIGPLAN
Conference Format with 10 point font, and includes a Creative Commons License,
which will allow us to publish the abstract on the workshop web site.

Please submit abstracts through http://ssw.jku.at/morevms17/


### Important Dates

  Abstract submission: 15 February 2017
  Author notification:   01 March 2017

  Workshop: 3 April 2017

All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC?12:00 hour

### Program Committee

Matthis Grimmer, Oracle Labs
Christine H. Flood, Red Hat
Tony Hosking, Australian National University
Hannes Payer, Purdue University
Jeremy Singer, University of Glasgow
Mark Stoodley, IBM Canada
Sam Tobin-Hochstadt, Indiana University

### Workshop Organizers

Laurence Tratt, King's College London, United Kingdom
Adam Welc, Oracle Labs, United States
Stefan Marr, Johannes Kepler University Linz, Austria





--
Stefan Marr
Johannes Kepler Universit?t Linz
http://stefan-marr.de/research/



_______________________________________________
mlvm-dev mailing list
mlvm-dev at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely MAY reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
prior contractual agreement.

From frankw at mit.edu  Fri Jan 27 10:45:14 2017
From: frankw at mit.edu (Frank Wang)
Date: Fri, 27 Jan 2017 10:45:14 -0500
Subject: [pypy-dev] Modifying Interpreter-level Object from Application
Message-ID: <CAD=7XjNXHuSJ8cMzqQDe6vCmtPx6w7mFhRP2MwSucTtqw8LpHw@mail.gmail.com>

Hi,

At the application level, I want to modify some interpreter-level
attributes of an object. Right now, I have the interpreter level functions
that allow me to modify the interpreter object. Is the easiest way to have
an application access interpreter level attributes to use a Mixed Module
with interpreter level definitions?

I'm having a bit of trouble getting it to work, so I was wondering if there
was a much easier way for an application to modify interpreter objects or
call interpreter level functions that modify an object.

Thanks,
Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170127/77099372/attachment.html>

From cfbolz at gmx.de  Fri Jan 27 11:35:20 2017
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 27 Jan 2017 17:35:20 +0100
Subject: [pypy-dev] Modifying Interpreter-level Object from Application
In-Reply-To: <CAD=7XjNXHuSJ8cMzqQDe6vCmtPx6w7mFhRP2MwSucTtqw8LpHw@mail.gmail.com>
References: <CAD=7XjNXHuSJ8cMzqQDe6vCmtPx6w7mFhRP2MwSucTtqw8LpHw@mail.gmail.com>
Message-ID: <f26d37bd-160b-d993-e937-e3ff4d6269e7@gmx.de>

Hi Frank,

no, unfortunately there's not really a shortcut to exposing the methods,
functions and attributes via a mixed module, because you need to think
about the semantics of parameter type and result conversion for every
such function anyway.

Do you have trouble to get it to work at all? Or is it just tedious?

If the former, there are some hints about mixed modules here:

http://doc.pypy.org/en/latest/coding-guide.html#implementing-a-mixed-interpreter-application-level-module

For the latter, if you need to do this for absolutely *every*
interpreter level attribute, there may be a way to achieve this effect
with some magic meta-programming, though I'd have to think a bit how.

Cheers,

Carl Friedrich

On 27/01/17 16:45, Frank Wang wrote:
> Hi,
> 
> At the application level, I want to modify some interpreter-level
> attributes of an object. Right now, I have the interpreter level
> functions that allow me to modify the interpreter object. Is the easiest
> way to have an application access interpreter level attributes to use a
> Mixed Module with interpreter level definitions?
> 
> I'm having a bit of trouble getting it to work, so I was wondering if
> there was a much easier way for an application to modify interpreter
> objects or call interpreter level functions that modify an object.
> 
> Thanks,
> Frank
> 
> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
> 


From frankw at mit.edu  Fri Jan 27 12:40:21 2017
From: frankw at mit.edu (Frank Wang)
Date: Fri, 27 Jan 2017 12:40:21 -0500
Subject: [pypy-dev] Modifying Interpreter-level Object from Application
In-Reply-To: <f26d37bd-160b-d993-e937-e3ff4d6269e7@gmx.de>
References: <CAD=7XjNXHuSJ8cMzqQDe6vCmtPx6w7mFhRP2MwSucTtqw8LpHw@mail.gmail.com>
 <f26d37bd-160b-d993-e937-e3ff4d6269e7@gmx.de>
Message-ID: <CAD=7XjMwL_+Gv3_6X1pwey8xv=8QhSWY878W6Lbc3q+naQU6YQ@mail.gmail.com>

Hi Carl,

Thanks for the information! I just have to do it for a specific attribute.
It is just a bit tedious as you said, making sure the semantics of the
parameter type and result conversion work properly is a bit tricky.



Frank

On Fri, Jan 27, 2017 at 11:35 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:

> Hi Frank,
>
> no, unfortunately there's not really a shortcut to exposing the methods,
> functions and attributes via a mixed module, because you need to think
> about the semantics of parameter type and result conversion for every
> such function anyway.
>
> Do you have trouble to get it to work at all? Or is it just tedious?
>
> If the former, there are some hints about mixed modules here:
>
> http://doc.pypy.org/en/latest/coding-guide.html#implementing-a-mixed-
> interpreter-application-level-module
>
> For the latter, if you need to do this for absolutely *every*
> interpreter level attribute, there may be a way to achieve this effect
> with some magic meta-programming, though I'd have to think a bit how.
>
> Cheers,
>
> Carl Friedrich
>
> On 27/01/17 16:45, Frank Wang wrote:
> > Hi,
> >
> > At the application level, I want to modify some interpreter-level
> > attributes of an object. Right now, I have the interpreter level
> > functions that allow me to modify the interpreter object. Is the easiest
> > way to have an application access interpreter level attributes to use a
> > Mixed Module with interpreter level definitions?
> >
> > I'm having a bit of trouble getting it to work, so I was wondering if
> > there was a much easier way for an application to modify interpreter
> > objects or call interpreter level functions that modify an object.
> >
> > Thanks,
> > Frank
> >
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170127/31371021/attachment.html>

From armin.rigo at gmail.com  Sat Jan 28 07:28:18 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sat, 28 Jan 2017 13:28:18 +0100
Subject: [pypy-dev] RPython hashes no longer preserved across translation
Message-ID: <CAMSv6X2B9Myg6WkHT=Z4Jp0nTXPWVqU0tfDr2fVonXouU-jJkA@mail.gmail.com>

Hi all,

I would like to merge the branch "rpython-hash".  That branch drops
the RPython requirement that hashes (notably of prebuilt instances)
don't change across translation.  By dropping it, I could remove some
cruft at various places, and it makes the life a bit easier for other
systems like STM, too.

The motivation is that in py3.5 we're considering the hash
randomization feature again, for strings.  This feature is not
compatible at all with the idea that hashes should not change.  So
with "rpython-hash" it should be possible, at translation-time, to
choose the algorithm used by CPython 3.5 (siphash-2-4), and moreover
with an optional seed at runtime.

The main place that relied on the hashes not changing is prebuilt
dictionaries.  That branch fixes them by not including the "indexes"
part at all during translation, and rebuilding the index on the first
usage.

There were only three places I found in PyPy that relied on this
feature: weakrefs caching the hash of their referent; frozensets
caching their own hash; and a detail inside _cffi_backend.  All places
needed fixing, but that was easy.

I think it's a good move and a small simplification of RPython
internals.  Of course it might break other RPython-based interpreters
out there.  Any feedback about that would be appreciated.


A bient?t,

Armin.

From cfbolz at gmx.de  Sat Jan 28 07:42:35 2017
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 28 Jan 2017 13:42:35 +0100
Subject: [pypy-dev] RPython hashes no longer preserved across translation
In-Reply-To: <CAMSv6X2B9Myg6WkHT=Z4Jp0nTXPWVqU0tfDr2fVonXouU-jJkA@mail.gmail.com>
References: <CAMSv6X2B9Myg6WkHT=Z4Jp0nTXPWVqU0tfDr2fVonXouU-jJkA@mail.gmail.com>
Message-ID: <d0aaccd0-3041-42b3-9382-cbf6afa03eb9@email.android.com>

Hi Armin, 

That sounds like a good simplification to me. 

Cheers, 
Carl Friedrich

On January 28, 2017 1:28:18 PM GMT+01:00, Armin Rigo <armin.rigo at gmail.com> wrote:
>Hi all,
>
>I would like to merge the branch "rpython-hash".  That branch drops
>the RPython requirement that hashes (notably of prebuilt instances)
>don't change across translation.  By dropping it, I could remove some
>cruft at various places, and it makes the life a bit easier for other
>systems like STM, too.
>
>The motivation is that in py3.5 we're considering the hash
>randomization feature again, for strings.  This feature is not
>compatible at all with the idea that hashes should not change.  So
>with "rpython-hash" it should be possible, at translation-time, to
>choose the algorithm used by CPython 3.5 (siphash-2-4), and moreover
>with an optional seed at runtime.
>
>The main place that relied on the hashes not changing is prebuilt
>dictionaries.  That branch fixes them by not including the "indexes"
>part at all during translation, and rebuilding the index on the first
>usage.
>
>There were only three places I found in PyPy that relied on this
>feature: weakrefs caching the hash of their referent; frozensets
>caching their own hash; and a detail inside _cffi_backend.  All places
>needed fixing, but that was easy.
>
>I think it's a good move and a small simplification of RPython
>internals.  Of course it might break other RPython-based interpreters
>out there.  Any feedback about that would be appreciated.
>
>
>A bient?t,
>
>Armin.
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170128/35096e51/attachment.html>

From armin.rigo at gmail.com  Sat Jan 28 08:20:11 2017
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sat, 28 Jan 2017 14:20:11 +0100
Subject: [pypy-dev] RPython hashes no longer preserved across translation
In-Reply-To: <d0aaccd0-3041-42b3-9382-cbf6afa03eb9@email.android.com>
References: <CAMSv6X2B9Myg6WkHT=Z4Jp0nTXPWVqU0tfDr2fVonXouU-jJkA@mail.gmail.com>
 <d0aaccd0-3041-42b3-9382-cbf6afa03eb9@email.android.com>
Message-ID: <CAMSv6X3iybptS_31+D77WB-_zWqmbEP92fHCO1kQu2BFGQ0C1g@mail.gmail.com>

Hi,

On 28 January 2017 at 13:42, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> That sounds like a good simplification to me.

Might come with a few bugs for now :-(  I just realized I missed
prebuilt non-empty RWeakValueDictionary().


Armin