From jdoran at lexmachina.com  Tue Nov  3 17:57:09 2015
From: jdoran at lexmachina.com (Jeff Doran)
Date: Tue, 3 Nov 2015 14:57:09 -0800
Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy
 2.6.1
In-Reply-To: <CAMSv6X3LjwgFasc9G-C=uhcJQm6SAe9PsU8kS5NQ7QaNq29_Vw@mail.gmail.com>
References: <CAFsQrzAj0++9AuJ-2YMmMY_vMh1JHbGD5cD-t_bvm1X2O0KdDQ@mail.gmail.com>
 <CAMSv6X0RUmnpBm0bNNyExrFpbySz9Z-ZaSY3dDSJu1O-ai0yfQ@mail.gmail.com>
 <CAFsQrzARTEiHNDotbqRF_8-Wvy6JLM7o0eJGXgCVGuZCbVB=8w@mail.gmail.com>
 <CAMSv6X0Q_cNTruxH+Yd67Y4SRDAEG-ygc+d3feOVDDSUqOBzMg@mail.gmail.com>
 <CAGmFidYp_Pz37671OuMDpuhufYMXy6-k5H2Xvz49=WmovjyrxQ@mail.gmail.com>
 <CAMSv6X3LjwgFasc9G-C=uhcJQm6SAe9PsU8kS5NQ7QaNq29_Vw@mail.gmail.com>
Message-ID: <CAFsQrzDEjN3Gs1apzC9g9vFfFpj0Ck_8f5Hx5Uak=dXQYE8Jkw@mail.gmail.com>

Hi,

Just an update on this problem.  I retested using the latest lxml source
from here <https://github.com/lxml/lxml> ( 3.5.0b2?) and a version of
PyPy-4.0 from here
<https://bitbucket.org/pypy/pypy/downloads/pypy-4.0.0-src.tar.bz2> that I
built for Debian 7.9.    The good news is I got much farther in the lxml
test suite then before, but the bad news is I still encountered my old
friend  SIGSEGV.  This is consistently occuring during
 test_delslice_negative2 in lxml.tests.test_elementtree.ETreeTestCase which
as I said is much further thru the suite of tests then previously.    This
is encouraging.

For now I'm still unable to use this combination so if anyone has a
 suggestion on how I might proceed to utilize both PyPy and lxml I would be
most appreciative.

Thanks.
- Jeff

---------------------------------------------------------

 nosetests  -vv  --nocapture
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
nose.selector: INFO: /home/jeff/lxml/src/lxml/etree.pypy-26.so is
executable; skipped
nose.selector: INFO: /home/jeff/lxml/src/lxml/objectify.pypy-26.so is
executable; skipped
Comparing with ElementTree 1.3.0
Comparing with cElementTree 1.3.0

TESTED VERSION: 3.5.0.beta1
    Python:           (major=2, minor=7, micro=10, releaselevel='final',
serial=42)
    lxml.etree:       (3, 5, 0, -99)
    libxml used:      (2, 8, 0)
    libxml compiled:  (2, 8, 0)
    libxslt used:     (1, 1, 26)
    libxslt compiled: (1, 1, 26)

lxml.html.tests.test_autolink.test_suite ... ok
lxml.html.tests.test_basic.test_suite ... ok
test_allow_tags (lxml.html.tests.test_clean.CleanerTest) ... ok
test_clean_invalid_root_tag (lxml.html.tests.test_clean.CleanerTest) ... ok
test_safe_attrs_excluded (lxml.html.tests.test_clean.CleanerTest) ... ok
test_safe_attrs_included (lxml.html.tests.test_clean.CleanerTest) ... ok
lxml.html.tests.test_clean.test_suite ... ok
lxml.html.tests.test_diff.test_suite ... ok
lxml.html.tests.test_elementsoup.test_suite ... ok

...  lots of successful test output removed

test_delitem (lxml.tests.test_elementtree.ETreeTestCase) ... ok
test_delitem_tail (lxml.tests.test_elementtree.ETreeTestCase) ... ok
test_delslice (lxml.tests.test_elementtree.ETreeTestCase) ... ok
test_delslice_child_tail (lxml.tests.test_elementtree.ETreeTestCase) ... ok
test_delslice_memory (lxml.tests.test_elementtree.ETreeTestCase) ... ok
test_delslice_negative1 (lxml.tests.test_elementtree.ETreeTestCase) ... ok
test_delslice_negative2 (lxml.tests.test_elementtree.ETreeTestCase) ...
fish: Job 2, ?nosetests  -vv  --nocapture ? terminated by signal SIGSEGV
(Address boundary error)

---------------------------------------------------------


On Mon, Oct 12, 2015 at 2:30 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi,
>
> On Mon, Oct 12, 2015 at 11:24 AM, Amaury Forgeot d'Arc
> <amauryfa at gmail.com> wrote:
> > We define it as a string (in pypy/module/cpyext/include/patchlevel.h)
> > #define PYPY_VERSION "2.6.0-alpha0"
>
> Yes, but i don't know how to do this check in Cython.  Well, it's probably
> easy.
>
> Armin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151103/c6a4adac/attachment.html>

From arigo at tunes.org  Wed Nov  4 08:09:32 2015
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 4 Nov 2015 14:09:32 +0100
Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy
 2.6.1
In-Reply-To: <CAFsQrzDEjN3Gs1apzC9g9vFfFpj0Ck_8f5Hx5Uak=dXQYE8Jkw@mail.gmail.com>
References: <CAFsQrzAj0++9AuJ-2YMmMY_vMh1JHbGD5cD-t_bvm1X2O0KdDQ@mail.gmail.com>
 <CAMSv6X0RUmnpBm0bNNyExrFpbySz9Z-ZaSY3dDSJu1O-ai0yfQ@mail.gmail.com>
 <CAFsQrzARTEiHNDotbqRF_8-Wvy6JLM7o0eJGXgCVGuZCbVB=8w@mail.gmail.com>
 <CAMSv6X0Q_cNTruxH+Yd67Y4SRDAEG-ygc+d3feOVDDSUqOBzMg@mail.gmail.com>
 <CAGmFidYp_Pz37671OuMDpuhufYMXy6-k5H2Xvz49=WmovjyrxQ@mail.gmail.com>
 <CAMSv6X3LjwgFasc9G-C=uhcJQm6SAe9PsU8kS5NQ7QaNq29_Vw@mail.gmail.com>
 <CAFsQrzDEjN3Gs1apzC9g9vFfFpj0Ck_8f5Hx5Uak=dXQYE8Jkw@mail.gmail.com>
Message-ID: <CAMSv6X0COcCpp_NT0B6g6UHnXwWDiyAenbxANA+PTbnwwX05wA@mail.gmail.com>

Hi Jeff,

On Tue, Nov 3, 2015 at 11:57 PM, Jeff Doran <jdoran at lexmachina.com> wrote:
> Just an update on this problem.  I retested using the latest lxml source
> from here ( 3.5.0b2?)

There were a few fixes to PyPy's cpyext layer.  But there was no
change to lxml itself.  That means it is still using a broken approach
to work around a CPython C API difference.  Namely, this is the
weakref stuff which is done incorrectly---although I don't know how to
do that correctly.

For now, lxml (the non-cffi version) is not compatible with PyPy at all.


A bient?t,

Armin.

From tismer at stackless.com  Wed Nov  4 17:54:44 2015
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 4 Nov 2015 23:54:44 +0100
Subject: [pypy-dev] sandbox small bug patch
Message-ID: <563A8CB4.6060903@stackless.com>

Hi guys,

I was playing with sandbox and found that the interactive mode
does not work because of 'import time'.

Wanted to create a pull request, but for some reason I can't push
to bitbucket right now.

A simple patch is attached.

Cheers - Chris

-- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     https://github.com/PySide
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023

-------------- next part --------------
# HG changeset patch
# User Christian Tismer  <tismer at stackless.com>
# Date 1446672957 -3600
# Branch fix-sandbox
# Node ID f4296318c6bf2077c7535cdeebb001a3653aef4a
# Parent  f48af5699c242d420ade3a0389115c5697af3a03
fix the sandbox interactive shell

When the sandbox runs interactivelz, it complains and crashes because of the import
on the time module, which is unsupported in sandbox.

The patch circumvents this case by intercepting MemoryError.

diff --git a/lib_pypy/_pypy_interact.py b/lib_pypy/_pypy_interact.py
--- a/lib_pypy/_pypy_interact.py
+++ b/lib_pypy/_pypy_interact.py
@@ -25,6 +25,12 @@
             print(text)
         except ImportError:
             pass
+        except MemoryError:
+            if sys.pypy_translation_info['translation.sandbox']:
+                print("sandbox: _pypy_irc_topic has a forbidden import")
+                pass
+            else:
+                raise
     #
     try:
         if not os.isatty(sys.stdin.fileno()):

From dmitry at getgrist.com  Wed Nov  4 18:40:50 2015
From: dmitry at getgrist.com (Dmitry Sagalovskiy)
Date: Wed, 4 Nov 2015 18:40:50 -0500
Subject: [pypy-dev] sandbox small bug patch
In-Reply-To: <563A8CB4.6060903@stackless.com>
References: <563A8CB4.6060903@stackless.com>
Message-ID: <CAF2Jo6rUDNGFEti5Mm2XtBj0SNAVdFBT2RUwG8AEZfczmxyd0w@mail.gmail.com>

I've been wondering why it is that 'time' module is so broken in the
sandbox? Is there any fundamental reason why it can't be supported?

On Wed, Nov 4, 2015 at 5:54 PM, Christian Tismer <tismer at stackless.com>
wrote:

> Hi guys,
>
> I was playing with sandbox and found that the interactive mode
> does not work because of 'import time'.
>
> Wanted to create a pull request, but for some reason I can't push
> to bitbucket right now.
>
> A simple patch is attached.
>
> Cheers - Chris
>
> --
> Christian Tismer             :^)   tismer at stackless.com
> Software Consulting          :     http://www.stackless.com/
> Karl-Liebknecht-Str. 121     :     https://github.com/PySide
> 14482 Potsdam                :     GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151104/a08480a7/attachment.html>

From arigo at tunes.org  Thu Nov  5 03:14:36 2015
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 5 Nov 2015 09:14:36 +0100
Subject: [pypy-dev] sandbox small bug patch
In-Reply-To: <CAF2Jo6rUDNGFEti5Mm2XtBj0SNAVdFBT2RUwG8AEZfczmxyd0w@mail.gmail.com>
References: <563A8CB4.6060903@stackless.com>
 <CAF2Jo6rUDNGFEti5Mm2XtBj0SNAVdFBT2RUwG8AEZfczmxyd0w@mail.gmail.com>
Message-ID: <CAMSv6X0LO9UeWJNOKtTao_Wf03DvF=2M5U39DFt5nLtPQWVS-w@mail.gmail.com>

Hi Dmitry,

On Thu, Nov 5, 2015 at 12:40 AM, Dmitry Sagalovskiy <dmitry at getgrist.com> wrote:
> I've been wondering why it is that 'time' module is so broken in the
> sandbox? Is there any fundamental reason why it can't be supported?

No, it's just that we improved the 'time' module at some point by
getting rid of very old code, and that broke sandboxed translations.
Someone needs to look at it.  It's probably easy.


A bient?t,

Armin.

From sebastian at hackinglabs.cl  Wed Nov  4 16:53:42 2015
From: sebastian at hackinglabs.cl (Luis Sebastian Urrutia Fuentes)
Date: Wed, 4 Nov 2015 18:53:42 -0300
Subject: [pypy-dev] PyPy compatibility
Message-ID: <43418135-DD8C-4FA3-A23F-6FFAFAEF1012@hackinglabs.cl>

Hi, pypy is compatible with python 3.5 or python 3.3?

From tismer at stackless.com  Thu Nov  5 03:45:23 2015
From: tismer at stackless.com (Christian Tismer)
Date: Thu, 5 Nov 2015 09:45:23 +0100
Subject: [pypy-dev] sandbox small bug patch
In-Reply-To: <CAMSv6X0LO9UeWJNOKtTao_Wf03DvF=2M5U39DFt5nLtPQWVS-w@mail.gmail.com>
References: <563A8CB4.6060903@stackless.com>
 <CAF2Jo6rUDNGFEti5Mm2XtBj0SNAVdFBT2RUwG8AEZfczmxyd0w@mail.gmail.com>
 <CAMSv6X0LO9UeWJNOKtTao_Wf03DvF=2M5U39DFt5nLtPQWVS-w@mail.gmail.com>
Message-ID: <563B1723.8060308@stackless.com>

On 05/11/15 09:14, Armin Rigo wrote:
> Hi Dmitry,
> 
> On Thu, Nov 5, 2015 at 12:40 AM, Dmitry Sagalovskiy <dmitry at getgrist.com> wrote:
>> I've been wondering why it is that 'time' module is so broken in the
>> sandbox? Is there any fundamental reason why it can't be supported?
> 
> No, it's just that we improved the 'time' module at some point by
> getting rid of very old code, and that broke sandboxed translations.
> Someone needs to look at it.  It's probably easy.

Ok, makes sense.
I did not expect that this is the final solution, but just
shows the problem and an immediate fix against frustration.

If my company (yes, I'm working again) permits, then I can
work on fixing the true reason.

cheers - Chris

-- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     https://github.com/PySide
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 522 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151105/0d945ae7/attachment.sig>

From fijall at gmail.com  Thu Nov  5 04:20:13 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 5 Nov 2015 09:20:13 +0000
Subject: [pypy-dev] PyPy compatibility
In-Reply-To: <43418135-DD8C-4FA3-A23F-6FFAFAEF1012@hackinglabs.cl>
References: <43418135-DD8C-4FA3-A23F-6FFAFAEF1012@hackinglabs.cl>
Message-ID: <CAK5idxT3hLsrwkZbxZ-e_hYSdyaGdF5iJuwpwp3XsV67JBkDYA@mail.gmail.com>

PyPy is compatible with Python 2.7

PyPy3 is compatible with Python 3.2 with the upcoming 3.3 release

On Wed, Nov 4, 2015 at 9:53 PM, Luis Sebastian Urrutia Fuentes
<sebastian at hackinglabs.cl> wrote:
> Hi, pypy is compatible with python 3.5 or python 3.3?
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From andrew at divvycloud.com  Fri Nov  6 23:23:02 2015
From: andrew at divvycloud.com (Andrew Mann)
Date: Fri, 6 Nov 2015 22:23:02 -0600
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
Message-ID: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>

Hi everyone,

I'm testing our software on PyPy 4.0.0, and things generally work well, but
occasionally I encounter the following RPython traceback and subsequent
core dump:

RPython traceback:
  File "rpython_jit_metainterp_optimizeopt_unroll.c", line 9591, in
UnrollOptimizer_optimize_bridge
  File "rpython_jit_metainterp_optimizeopt_unroll.c", line 17372, in
UnrollOptimizer_jump_to_existing_trace
  File "rpython_jit_metainterp_optimizeopt_unroll.c", line 23101, in
UnrollOptimizer_inline_short_preamble
  File "rpython_jit_metainterp_optimizeopt_intbounds.c", line 162, in
OptIntBounds__optimize_guard_true_false_value
  File "rpython_jit_metainterp_optimizeopt_optimizer.c", line 18252, in
Optimizer_make_constant
  File "rpython_jit_metainterp_optimizeopt_info.c", line 21500, in
AbstractStructPtrInfo_copy_fields_to_const
  File "rpython_jit_metainterp_optimizeopt_info.c", line 12522, in
ConstPtrInfo__get_info
  File "rpython_rtyper_lltypesystem_rordereddict_4.c", line 36646, in
ll_dict_get__dicttablePtr_GCREFPtr_rpython_jit_m
Fatal RPython error: AssertionError

I don't want to waste anyone's time diagnosing the issue if it's a) known
or b) probably not anything the pypy devs care about.  However, if it's
something that seems worth investigating, I'm happy to provide whatever
information is useful or debug as needed. I'm not really familiar with the
pypy architecture, but I know my way around debuggers and the various
languages in use here.

Details:

This is a flask based web application wrapper around sql alchemy access to
a mysql database. The operation that triggers this is a retrieval of about
20,000 database rows with a total of maybe 5MB of data - so nothing too
huge.

I also have cProfiler active and recording data. At the moment I'm not sure
if the problem happens with profiling enabled; I haven't tested.

The same section of python code with the same data set (perhaps with
differences of timestamps) runs successfully most of the time.  It seems to
fail about 1 in 4 passes through this section of code.

This is run on an Ubuntu 14.04 VM with the x64 binaries of pypy 4.0.0
retrieved from the public pypy web site downloads.

Does this seem worth investigating further?

-- 
Andrew Mann
DivvyCloud Inc.
www.divvycloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151106/1a3667a3/attachment.html>

From fijall at gmail.com  Sat Nov  7 03:43:10 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 7 Nov 2015 08:43:10 +0000
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
Message-ID: <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>

Hi Andrew

This looks fairly serious, we would love to have a way to reproduce
it. For what is worth, you don't have to minimize the example very
much - a program we can run, even if it has serious dependecies, is
ok, but we NEED to be able to reproduce it.

On Sat, Nov 7, 2015 at 4:23 AM, Andrew Mann <andrew at divvycloud.com> wrote:
> Hi everyone,
>
> I'm testing our software on PyPy 4.0.0, and things generally work well, but
> occasionally I encounter the following RPython traceback and subsequent core
> dump:
>
> RPython traceback:
>   File "rpython_jit_metainterp_optimizeopt_unroll.c", line 9591, in
> UnrollOptimizer_optimize_bridge
>   File "rpython_jit_metainterp_optimizeopt_unroll.c", line 17372, in
> UnrollOptimizer_jump_to_existing_trace
>   File "rpython_jit_metainterp_optimizeopt_unroll.c", line 23101, in
> UnrollOptimizer_inline_short_preamble
>   File "rpython_jit_metainterp_optimizeopt_intbounds.c", line 162, in
> OptIntBounds__optimize_guard_true_false_value
>   File "rpython_jit_metainterp_optimizeopt_optimizer.c", line 18252, in
> Optimizer_make_constant
>   File "rpython_jit_metainterp_optimizeopt_info.c", line 21500, in
> AbstractStructPtrInfo_copy_fields_to_const
>   File "rpython_jit_metainterp_optimizeopt_info.c", line 12522, in
> ConstPtrInfo__get_info
>   File "rpython_rtyper_lltypesystem_rordereddict_4.c", line 36646, in
> ll_dict_get__dicttablePtr_GCREFPtr_rpython_jit_m
> Fatal RPython error: AssertionError
>
> I don't want to waste anyone's time diagnosing the issue if it's a) known or
> b) probably not anything the pypy devs care about.  However, if it's
> something that seems worth investigating, I'm happy to provide whatever
> information is useful or debug as needed. I'm not really familiar with the
> pypy architecture, but I know my way around debuggers and the various
> languages in use here.
>
> Details:
>
> This is a flask based web application wrapper around sql alchemy access to a
> mysql database. The operation that triggers this is a retrieval of about
> 20,000 database rows with a total of maybe 5MB of data - so nothing too
> huge.
>
> I also have cProfiler active and recording data. At the moment I'm not sure
> if the problem happens with profiling enabled; I haven't tested.
>
> The same section of python code with the same data set (perhaps with
> differences of timestamps) runs successfully most of the time.  It seems to
> fail about 1 in 4 passes through this section of code.
>
> This is run on an Ubuntu 14.04 VM with the x64 binaries of pypy 4.0.0
> retrieved from the public pypy web site downloads.
>
> Does this seem worth investigating further?
>
> --
> Andrew Mann
> DivvyCloud Inc.
> www.divvycloud.com
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From arigo at tunes.org  Sat Nov  7 05:27:19 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 7 Nov 2015 11:27:19 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
Message-ID: <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>

Hi Andrew,

On Sat, Nov 7, 2015 at 9:43 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> This looks fairly serious, we would love to have a way to reproduce
> it. For what is worth, you don't have to minimize the example very
> much - a program we can run, even if it has serious dependecies, is
> ok, but we NEED to be able to reproduce it.

We managed to figure it out just by discussion, for once.  It seems
that this time we won't need to reproduce it.

The problem is that in some cases, loop unrolling can create a second
iteration of a loop in which what used to be a variable argument is
now a constant (like NULL).  But the loop can contain operations to
read/write fields on that variable.  In this case, the second
iteration would never be executed in practice (as it would segfault),
but the optimizer doesn't realize that.

This crash occurs when trying to optimize read/write fields on NULLs.
The same problem can occur too (even more rarely I suppose) if the
constant is not NULL, but is an instance of the wrong class.


A bient?t,

Armin.

From hakan at debian.org  Sat Nov  7 06:11:52 2015
From: hakan at debian.org (Hakan Ardo)
Date: Sat, 7 Nov 2015 12:11:52 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
Message-ID: <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>

On Sat, Nov 7, 2015 at 11:27 AM, Armin Rigo <arigo at tunes.org> wrote:

>
> This crash occurs when trying to optimize read/write fields on NULLs.
> The same problem can occur too (even more rarely I suppose) if the
> constant is not NULL, but is an instance of the wrong class.
>

Also maybe for an opaque pointer to an array that was pointing some
instance in the first iteration?


-- 
H?kan Ard?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151107/05815b56/attachment.html>

From arigo at tunes.org  Sat Nov  7 08:25:29 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 7 Nov 2015 14:25:29 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
Message-ID: <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>

Hi Hakan,

On Sat, Nov 7, 2015 at 12:11 PM, Hakan Ardo <hakan at debian.org> wrote:
> Also maybe for an opaque pointer to an array that was pointing some instance
> in the first iteration?

Yes, anything dereferencing a pointer basically---during optimization.
It should only matter for Consts.

I realized it is annoying: in theory, we can try to unroll this loop:

        [i0, i1]
        i2 = int_gt(i1, 0)
        guard_true(i2) []
        i4 = getarrayitem_raw_pure_i(i0, 0, descr=rawarraydescr)
        i3 = int_sub(i1, 1)
        jump(12345, i3)

The "12345" is a nonsensical constant raw pointer.  Again, this only
makes sense if it is called with i1 = 0 or 1, but unrolling will
really try to perform the getarrayitem_raw_pure(12345, 0) during
optimization.

More thoughts needed.


A bient?t,

Armin.

From arigo at tunes.org  Sat Nov  7 08:40:25 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 7 Nov 2015 14:40:25 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
Message-ID: <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>

ReHi,

On Sat, Nov 7, 2015 at 2:25 PM, Armin Rigo <arigo at tunes.org> wrote:
> More thoughts needed.

One solution would be to use two modes when optimizing the two
unrolled versions of the loop.  For the 1st version we do like now.
But the 2nd version is run "speculatively" and so more care is needed:
we should not do any actual memory read (from either GC or raw
pointers).  Instead, we record the reads done during the 1st version's
optimization, and we replay them during the 2nd version.  The common
case should be that there are reads from the same locations anyway.
But if an attempted read occurs *only* during the 2nd version, like in
these strange examples, then we don't do any constant-folding.

Would that miss an important case?


A bient?t,

Armin.

From stefan_ml at behnel.de  Sat Nov  7 23:51:07 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 8 Nov 2015 05:51:07 +0100
Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy
 2.6.1
In-Reply-To: <CAMSv6X0COcCpp_NT0B6g6UHnXwWDiyAenbxANA+PTbnwwX05wA@mail.gmail.com>
References: <CAFsQrzAj0++9AuJ-2YMmMY_vMh1JHbGD5cD-t_bvm1X2O0KdDQ@mail.gmail.com>
 <CAMSv6X0RUmnpBm0bNNyExrFpbySz9Z-ZaSY3dDSJu1O-ai0yfQ@mail.gmail.com>
 <CAFsQrzARTEiHNDotbqRF_8-Wvy6JLM7o0eJGXgCVGuZCbVB=8w@mail.gmail.com>
 <CAMSv6X0Q_cNTruxH+Yd67Y4SRDAEG-ygc+d3feOVDDSUqOBzMg@mail.gmail.com>
 <CAGmFidYp_Pz37671OuMDpuhufYMXy6-k5H2Xvz49=WmovjyrxQ@mail.gmail.com>
 <CAMSv6X3LjwgFasc9G-C=uhcJQm6SAe9PsU8kS5NQ7QaNq29_Vw@mail.gmail.com>
 <CAFsQrzDEjN3Gs1apzC9g9vFfFpj0Ck_8f5Hx5Uak=dXQYE8Jkw@mail.gmail.com>
 <CAMSv6X0COcCpp_NT0B6g6UHnXwWDiyAenbxANA+PTbnwwX05wA@mail.gmail.com>
Message-ID: <n1mkbr$jh2$1@ger.gmane.org>

Armin Rigo schrieb am 04.11.2015 um 14:09:
> On Tue, Nov 3, 2015 at 11:57 PM, Jeff Doran wrote:
>> Just an update on this problem.  I retested using the latest lxml source
>> from here ( 3.5.0b2?)
> 
> There were a few fixes to PyPy's cpyext layer.  But there was no
> change to lxml itself.  That means it is still using a broken approach
> to work around a CPython C API difference.  Namely, this is the
> weakref stuff which is done incorrectly---although I don't know how to
> do that correctly.

https://github.com/lxml/lxml/tree/pypy4

I tried to simply disable the special casing in proxy.pxi, but it only
leads to more crashes:

"""
$ pypy-4.0.0-linux64/bin/pypy test.py -vv -p
TESTED VERSION: 3.5.0.beta1
    Python:           (major=2, minor=7, micro=10, releaselevel='final',
serial=42)
    lxml.etree:       (3, 5, 0, -99)
    libxml used:      (2, 9, 1)
    libxml compiled:  (2, 9, 1)
    libxslt used:     (1, 1, 28)
    libxslt compiled: (1, 1, 28)

RPython traceback:
  File "rpython_memory_gctransform_support.c", line 8320, in
ll_call_destructor__funcPtr_pypy_module_cpyext_p_1
  File "pypy_module_cpyext_pyobject.c", line 2012, in PyOLifeline___del__
Fatal RPython error: AssertionError
"""

Any hints?

Stefan



From arigo at tunes.org  Sun Nov  8 02:46:22 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 8 Nov 2015 08:46:22 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
 <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
Message-ID: <CAMSv6X32CJ+MO4LV49zY-EMhLmVwvrOSEXLq3U0iOXtS=Dux1A@mail.gmail.com>

Hi,

On Sat, Nov 7, 2015 at 2:40 PM, Armin Rigo <arigo at tunes.org> wrote:
> Would that miss an important case?

Fijal shed doubt on that, so I'm going with this solution:

For GC objects, check the gctype or correct subclass when doing an
access from the optimizer.  (We can't do that with Boehm, but then we
never unroll with Boehm anyway, because the same checks are needed for
short preambles.)

For RAW pointers, kill the resoperations
GET{FIELD,ARRAYITEM}_RAW_PURE, but *not* the jitcodes
get{field,arrayitem}_raw_*_pure.  The latter are used for example in
all RPython classes' vtable accesses, and so are essential.  But in
all cases I could find, they'd be constant-folded during tracing
already (e.g. a previous guard_class always promotes the class to a
tracing-time constant).  For the rare case where they are not, we can
simply emit a regular GET{FIELD,ARRAYITEM}_RAW.


A bient?t,

Armin.

From hakan at debian.org  Sun Nov  8 03:39:23 2015
From: hakan at debian.org (Hakan Ardo)
Date: Sun, 8 Nov 2015 09:39:23 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
 <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
Message-ID: <CAEkL+RiMdRRtJaXWO2eZ4rCXFjSX81gYDYBnqKYumx3XArHntQ@mail.gmail.com>

On Sat, Nov 7, 2015 at 2:40 PM, Armin Rigo <arigo at tunes.org> wrote:

>
> Would that miss an important case?
>

I would guess that the interesting cases are when we have the same constant
in both iterations. In that case what we would miss is reads performed on
the constant pointer before it is proven to be a constant. If these are
numerous enough to care about I don't know, but I would not be surprised.

A small adjustment to your approach that would cover those cases is to
compare the values of the constants with the values observed during
tracing. If they are the same it should be safe to perform the read, right?

-- 
H?kan Ard?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151108/145f132b/attachment.html>

From arigo at tunes.org  Sun Nov  8 03:40:20 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 8 Nov 2015 09:40:20 +0100
Subject: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy
 2.6.1
In-Reply-To: <n1mkbr$jh2$1@ger.gmane.org>
References: <CAFsQrzAj0++9AuJ-2YMmMY_vMh1JHbGD5cD-t_bvm1X2O0KdDQ@mail.gmail.com>
 <CAMSv6X0RUmnpBm0bNNyExrFpbySz9Z-ZaSY3dDSJu1O-ai0yfQ@mail.gmail.com>
 <CAFsQrzARTEiHNDotbqRF_8-Wvy6JLM7o0eJGXgCVGuZCbVB=8w@mail.gmail.com>
 <CAMSv6X0Q_cNTruxH+Yd67Y4SRDAEG-ygc+d3feOVDDSUqOBzMg@mail.gmail.com>
 <CAGmFidYp_Pz37671OuMDpuhufYMXy6-k5H2Xvz49=WmovjyrxQ@mail.gmail.com>
 <CAMSv6X3LjwgFasc9G-C=uhcJQm6SAe9PsU8kS5NQ7QaNq29_Vw@mail.gmail.com>
 <CAFsQrzDEjN3Gs1apzC9g9vFfFpj0Ck_8f5Hx5Uak=dXQYE8Jkw@mail.gmail.com>
 <CAMSv6X0COcCpp_NT0B6g6UHnXwWDiyAenbxANA+PTbnwwX05wA@mail.gmail.com>
 <n1mkbr$jh2$1@ger.gmane.org>
Message-ID: <CAMSv6X2rUK7_xPmQZpQwaARpmgjWjJkynx7MXietwCyDaPSNsg@mail.gmail.com>

Hi Stefan,

On Sun, Nov 8, 2015 at 5:51 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> I tried to simply disable the special casing in proxy.pxi, but it only
> leads to more crashes:

Yes, it is expected.  The special-casing was introduced because PyPy
doesn't guarantee the identity of "PyObject *" when you don't own a
refcount.  It's what I'm trying to fix in the branch cpyext-gc-support
but it's not done yet.  It's why I said I don't know how to tweak lxml
to make it work on current versions of PyPy.


A bient?t,

Armin.

From arigo at tunes.org  Sun Nov  8 03:52:22 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 8 Nov 2015 09:52:22 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAEkL+RiMdRRtJaXWO2eZ4rCXFjSX81gYDYBnqKYumx3XArHntQ@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
 <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
 <CAEkL+RiMdRRtJaXWO2eZ4rCXFjSX81gYDYBnqKYumx3XArHntQ@mail.gmail.com>
Message-ID: <CAMSv6X0BbX5PRbgps6SZZr6VJ0dijXqnWdzaGN0O6taQu9N5GQ@mail.gmail.com>

Hi Hakan,

On Sun, Nov 8, 2015 at 9:39 AM, Hakan Ardo <hakan at debian.org> wrote:
> A small adjustment to your approach that would cover those cases is to
> compare the values of the constants with the values observed during tracing.
> If they are the same it should be safe to perform the read, right?

Ah, that's a good plan too.  It needs to have a way to record what we
saw during tracing (preferably not in some huge dict), and we decided
long ago that the optimizer should not essentially look at the content
of boxes (apart to guide some optimizations).  For example, CALL_PURE
uses a big dict.  The problem is that such a dict would be much bigger
for all GETFIELD_PURE.

I'm still going with checking the gctype of the object.  It should
work in the same cases (and theoretically more but well).


A bient?t,

Armin.

From hakan at debian.org  Sun Nov  8 06:09:40 2015
From: hakan at debian.org (Hakan Ardo)
Date: Sun, 8 Nov 2015 12:09:40 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X0BbX5PRbgps6SZZr6VJ0dijXqnWdzaGN0O6taQu9N5GQ@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
 <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
 <CAEkL+RiMdRRtJaXWO2eZ4rCXFjSX81gYDYBnqKYumx3XArHntQ@mail.gmail.com>
 <CAMSv6X0BbX5PRbgps6SZZr6VJ0dijXqnWdzaGN0O6taQu9N5GQ@mail.gmail.com>
Message-ID: <CAEkL+RhwNiKpUTjK-_5FFS-J_zGgX+mvwS2Fk_N_b6gKcW+raA@mail.gmail.com>

On Sun, Nov 8, 2015 at 9:52 AM, Armin Rigo <arigo at tunes.org> wrote:

>
> Ah, that's a good plan too.  It needs to have a way to record what we
> saw during tracing (preferably not in some huge dict), and we decided
> long ago that the optimizer should not essentially look at the content
> of boxes (apart to guide some optimizations).  For example, CALL_PURE
> uses a big dict.  The problem is that such a dict would be much bigger
> for all GETFIELD_PURE.
>

Why not store the observed results on the resops (maybe using some new
OpWithObservedResult mixin to not get an extra field on all of them)?


>
> I'm still going with checking the gctype of the object.  It should
> work in the same cases (and theoretically more but well).
>

Yes, makes sens.

Another thing that strikes me is that those crazy cases does probably not
benefit much from unrolling, and the only result of unrolling them might be
to duplicate a lot of guards, which leads to more tracing of bridges. Maybe
it would be beneficial to compare the constant inputargs of the peeled loop
with the observed inputargs of the preamble and if they don't match (differ
enough?) skip unrolling for that loop. Alternatively demote the
non-matching constants non-constants (as they are obviously not loop
invariant constants) to make the peeled loop a more general target for
bridges to jump directly to.


-- 
H?kan Ard?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151108/ee017a0a/attachment-0001.html>

From arigo at tunes.org  Sun Nov  8 08:23:42 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 8 Nov 2015 14:23:42 +0100
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X0BbX5PRbgps6SZZr6VJ0dijXqnWdzaGN0O6taQu9N5GQ@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
 <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
 <CAEkL+RiMdRRtJaXWO2eZ4rCXFjSX81gYDYBnqKYumx3XArHntQ@mail.gmail.com>
 <CAMSv6X0BbX5PRbgps6SZZr6VJ0dijXqnWdzaGN0O6taQu9N5GQ@mail.gmail.com>
Message-ID: <CAMSv6X1cEuoV8OqbbmG2MALyMybAnQ=yvSx_YiAApb=QAtVYRw@mail.gmail.com>

Hi Andrew,

On Sun, Nov 8, 2015 at 9:52 AM, Armin Rigo <arigo at tunes.org> wrote:
> I'm still going with checking the gctype of the object.  It should
> work in the same cases (and theoretically more but well).

Not done yet, but I think I fixed Andrew's original problem.  Andrew,
could you try again with the latest build?
http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-latest-linux64.tar.bz2


A bient?t,

Armin.

From andrew at divvycloud.com  Mon Nov  9 15:47:01 2015
From: andrew at divvycloud.com (Andrew Mann)
Date: Mon, 9 Nov 2015 14:47:01 -0600
Subject: [pypy-dev] pypy 4.0.0 "Fatal RPython error: AssertionError"
In-Reply-To: <CAMSv6X1cEuoV8OqbbmG2MALyMybAnQ=yvSx_YiAApb=QAtVYRw@mail.gmail.com>
References: <CAOnzOCJ3=jQyi+dgtSqyxiQa80mLoAozYbZtp60tnk5WmSuiqw@mail.gmail.com>
 <CAK5idxQHqKw9D=oRsFk7tYptEn1ssur4HOeJTxEAYrsZ94wC3Q@mail.gmail.com>
 <CAMSv6X0yxuA8YFNct+WzQHVNZ+1ObR21igL7zb1K_Qpi5ZyA6g@mail.gmail.com>
 <CAEkL+RiQUhLBKWoDGPB4oYH7uVwmtdx0vbn_ANWE5Yo9gSQbGg@mail.gmail.com>
 <CAMSv6X2uoyePL4QqU3VHfxeRWnSJ-cezsH-g=_0M7szp=cj3PQ@mail.gmail.com>
 <CAMSv6X0qBOgA1FCfWuAjhsNQjPQqZn3pST7OONvn4yX8NcfoMQ@mail.gmail.com>
 <CAEkL+RiMdRRtJaXWO2eZ4rCXFjSX81gYDYBnqKYumx3XArHntQ@mail.gmail.com>
 <CAMSv6X0BbX5PRbgps6SZZr6VJ0dijXqnWdzaGN0O6taQu9N5GQ@mail.gmail.com>
 <CAMSv6X1cEuoV8OqbbmG2MALyMybAnQ=yvSx_YiAApb=QAtVYRw@mail.gmail.com>
Message-ID: <CAOnzOCKPPhOTKuRTwZo8Lz8iyJScEAAaJdVHeqv0z3t4DwF+HQ@mail.gmail.com>

Nice work on deducing the issue from the limited information provided!
I've downloaded the referenced nightly build and tested today. It looks
like this does indeed fix the issue. I've run through the section of python
code about 30 times so far without issue where it was failing about 1 in 4
times previously.

-Andrew

On Sun, Nov 8, 2015 at 7:23 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Andrew,
>
> On Sun, Nov 8, 2015 at 9:52 AM, Armin Rigo <arigo at tunes.org> wrote:
> > I'm still going with checking the gctype of the object.  It should
> > work in the same cases (and theoretically more but well).
>
> Not done yet, but I think I fixed Andrew's original problem.  Andrew,
> could you try again with the latest build?
> http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-latest-linux64.tar.bz2
>
>
> A bient?t,
>
> Armin.
>



-- 
Andrew Mann
DivvyCloud Inc.
www.divvycloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151109/0bc086b8/attachment.html>

From matti.picus at gmail.com  Sat Nov 14 13:53:14 2015
From: matti.picus at gmail.com (Matti Picus)
Date: Sat, 14 Nov 2015 20:53:14 +0200
Subject: [pypy-dev] PyPy 4.0.1 bug-fix release
Message-ID: <5647831A.9080401@gmail.com>

After fixing the critical bugs 2180 (unrolling double loops) and 2183  
(ssl memory leak), should we release a bug-fix PyPy 4.0.1?
There were also some nice performance enhancements since 4.0.0, esp. 
list and array slicing

Any thoughts if there are critical outstanding issues blocking a bug-fix 
PyPy 4.0.1 very soon?
Matti

From alex.gaynor at gmail.com  Sat Nov 14 13:54:49 2015
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Sat, 14 Nov 2015 13:54:49 -0500
Subject: [pypy-dev] PyPy 4.0.1 bug-fix release
In-Reply-To: <5647831A.9080401@gmail.com>
References: <5647831A.9080401@gmail.com>
Message-ID: <CAFRnB2WAjoZFp5iEQgFLPukQTP+A1u43SYMo=W0pVj4n8xEYxA@mail.gmail.com>

Sounds like a good ideas to me. Thanks!

Alex
On Nov 14, 2015 1:53 PM, "Matti Picus" <matti.picus at gmail.com> wrote:

> After fixing the critical bugs 2180 (unrolling double loops) and 2183
> (ssl memory leak), should we release a bug-fix PyPy 4.0.1?
> There were also some nice performance enhancements since 4.0.0, esp. list
> and array slicing
>
> Any thoughts if there are critical outstanding issues blocking a bug-fix
> PyPy 4.0.1 very soon?
> Matti
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151114/c4cc331d/attachment.html>

From ronan.lamy at gmail.com  Sat Nov 14 18:57:04 2015
From: ronan.lamy at gmail.com (Ronan Lamy)
Date: Sat, 14 Nov 2015 23:57:04 +0000
Subject: [pypy-dev] PyPy 4.0.1 bug-fix release
In-Reply-To: <5647831A.9080401@gmail.com>
References: <5647831A.9080401@gmail.com>
Message-ID: <CALqCCOUD9Oj1p3+3XcmA_A9xFjdx+EpqidqZuQbNu3i5ghKOGw@mail.gmail.com>

Le 14 nov. 2015 6:53 PM, "Matti Picus" <matti.picus at gmail.com> a ?crit :
>
> After fixing the critical bugs 2180 (unrolling double loops) and 2183
(ssl memory leak), should we release a bug-fix PyPy 4.0.1?
> There were also some nice performance enhancements since 4.0.0, esp. list
and array slicing

Yes, releasing the fixes ASAP would be great.
If there are enhancements in addition to the bug fixes, 4.1.0 would be a
more fitting name, though.

> Any thoughts if there are critical outstanding issues blocking a bug-fix
PyPy 4.0.1 very soon?
> Matti
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151114/3c0fc2f3/attachment.html>

From tritium-list at sdamon.com  Sun Nov 15 00:17:54 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Sun, 15 Nov 2015 00:17:54 -0500
Subject: [pypy-dev] PyPy 4.0.1 bug-fix release
In-Reply-To: <CALqCCOUD9Oj1p3+3XcmA_A9xFjdx+EpqidqZuQbNu3i5ghKOGw@mail.gmail.com>
References: <5647831A.9080401@gmail.com>
 <CALqCCOUD9Oj1p3+3XcmA_A9xFjdx+EpqidqZuQbNu3i5ghKOGw@mail.gmail.com>
Message-ID: <56481582.4020905@sdamon.com>

Are there really breaking changes?

On 11/14/2015 18:57, Ronan Lamy wrote:
>
>
> Le 14 nov. 2015 6:53 PM, "Matti Picus" <matti.picus at gmail.com 
> <mailto:matti.picus at gmail.com>> a ?crit :
> >
> > After fixing the critical bugs 2180 (unrolling double loops) and 
> 2183  (ssl memory leak), should we release a bug-fix PyPy 4.0.1?
> > There were also some nice performance enhancements since 4.0.0, esp. 
> list and array slicing
>
> Yes, releasing the fixes ASAP would be great.
> If there are enhancements in addition to the bug fixes, 4.1.0 would be 
> a more fitting name, though.
>
> > Any thoughts if there are critical outstanding issues blocking a 
> bug-fix PyPy 4.0.1 very soon?
> > Matti
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org <mailto:pypy-dev at python.org>
> > https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151115/2d36c590/attachment.html>

From fijall at gmail.com  Sun Nov 15 03:12:15 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 15 Nov 2015 09:12:15 +0100
Subject: [pypy-dev] PyPy 4.0.1 bug-fix release
In-Reply-To: <56481582.4020905@sdamon.com>
References: <5647831A.9080401@gmail.com>
 <CALqCCOUD9Oj1p3+3XcmA_A9xFjdx+EpqidqZuQbNu3i5ghKOGw@mail.gmail.com>
 <56481582.4020905@sdamon.com>
Message-ID: <CAK5idxQWGsGQPBexXSitgHpWAuBxVQ-g=1nE3vJcb_DxUhE9Fg@mail.gmail.com>

IMO we should fix that one:
https://bitbucket.org/pypy/pypy/issues/2185/crash-in-jit-loading-the-django-admin

On Sun, Nov 15, 2015 at 6:17 AM, Alexander Walters
<tritium-list at sdamon.com> wrote:
> Are there really breaking changes?
>
>
> On 11/14/2015 18:57, Ronan Lamy wrote:
>
>
> Le 14 nov. 2015 6:53 PM, "Matti Picus" <matti.picus at gmail.com> a ?crit :
>>
>> After fixing the critical bugs 2180 (unrolling double loops) and 2183
>> (ssl memory leak), should we release a bug-fix PyPy 4.0.1?
>> There were also some nice performance enhancements since 4.0.0, esp. list
>> and array slicing
>
> Yes, releasing the fixes ASAP would be great.
> If there are enhancements in addition to the bug fixes, 4.1.0 would be a
> more fitting name, though.
>
>> Any thoughts if there are critical outstanding issues blocking a bug-fix
>> PyPy 4.0.1 very soon?
>> Matti
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From tobias.oberstein at tavendo.de  Mon Nov 16 17:02:13 2015
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Mon, 16 Nov 2015 23:02:13 +0100
Subject: [pypy-dev] Performance regression from 2.6.1 to 4.0.0 ?
Message-ID: <564A5265.7030407@tavendo.de>

Hi,

I've done some measurements comparing JSON and MsgPack 
serialization/deserialization performance

https://github.com/oberstet/scratchbox/tree/master/python/serperf

MsgPack deserialization seems significantly slower (>30%).

Note: the MsgPack library used runs pure Python code when on PyPy.

Is this expected/known?

Cheers,
/Tobias

From phyo.arkarlwin at gmail.com  Tue Nov 17 03:13:51 2015
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Tue, 17 Nov 2015 14:43:51 +0630
Subject: [pypy-dev] Performance regression from 2.6.1 to 4.0.0 ?
In-Reply-To: <564A5265.7030407@tavendo.de>
References: <564A5265.7030407@tavendo.de>
Message-ID: <CA+HjJrjsGx5BmN5nEnxqrc6eXjN0V7CBeqPLwhijmTONwXHeZA@mail.gmail.com>

can you try with latest master?

On Tue, Nov 17, 2015 at 4:32 AM, Tobias Oberstein <
tobias.oberstein at tavendo.de> wrote:

> Hi,
>
> I've done some measurements comparing JSON and MsgPack
> serialization/deserialization performance
>
> https://github.com/oberstet/scratchbox/tree/master/python/serperf
>
> MsgPack deserialization seems significantly slower (>30%).
>
> Note: the MsgPack library used runs pure Python code when on PyPy.
>
> Is this expected/known?
>
> Cheers,
> /Tobias
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151117/65289228/attachment.html>

From fijall at gmail.com  Tue Nov 17 03:46:04 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 17 Nov 2015 09:46:04 +0100
Subject: [pypy-dev] Performance regression from 2.6.1 to 4.0.0 ?
In-Reply-To: <564A5265.7030407@tavendo.de>
References: <564A5265.7030407@tavendo.de>
Message-ID: <CAK5idxSgagc8kZxxxpJcmG2gRmw2rj6bDfQdBgGsGKx=efanQA@mail.gmail.com>

Hi Tobias.

I'm right now travelling, can you put this into a bug report so it
does not get forgotten?

Cheers,
fijal


On Mon, Nov 16, 2015 at 11:02 PM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
> Hi,
>
> I've done some measurements comparing JSON and MsgPack
> serialization/deserialization performance
>
> https://github.com/oberstet/scratchbox/tree/master/python/serperf
>
> MsgPack deserialization seems significantly slower (>30%).
>
> Note: the MsgPack library used runs pure Python code when on PyPy.
>
> Is this expected/known?
>
> Cheers,
> /Tobias
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From tobias.oberstein at tavendo.de  Tue Nov 17 04:08:07 2015
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Tue, 17 Nov 2015 10:08:07 +0100
Subject: [pypy-dev] Performance regression from 2.6.1 to 4.0.0 ?
In-Reply-To: <CAK5idxSgagc8kZxxxpJcmG2gRmw2rj6bDfQdBgGsGKx=efanQA@mail.gmail.com>
References: <564A5265.7030407@tavendo.de>
 <CAK5idxSgagc8kZxxxpJcmG2gRmw2rj6bDfQdBgGsGKx=efanQA@mail.gmail.com>
Message-ID: <564AEE77.6070907@tavendo.de>

Hi Fijal,

> I'm right now travelling, can you put this into a bug report so it

Ah, have a good trip!

> does not get forgotten?

https://bitbucket.org/pypy/pypy/issues/2190/possible-performance-regression-from-261

Cheers,
/Tobias


From matti.picus at gmail.com  Wed Nov 18 17:04:05 2015
From: matti.picus at gmail.com (Matti Picus)
Date: Thu, 19 Nov 2015 00:04:05 +0200
Subject: [pypy-dev] PyPy 4.0.1 tarballs avaiable
Message-ID: <564CF5D5.6070005@gmail.com>

Please try them out, I want to release in the next few days.
https://bitbucket.org/pypy/pypy/downloads

Also any comments (or edits) to the release notice would be nice
http://pypy.readthedocs.org/en/latest/release-4.0.1.html

Matti

P.S. here are the hashes (without PPC), I will modify pypy.org/downloads 
closer to the release itself

f6721e62f4ba1cdc4cc5ad719369e359  pypy-4.0.1-linux64.tar.bz2
fe8106ac3919c7b4be2766944326a624  pypy-4.0.1-linux-armel.tar.bz2
823b8a457f4c48ebdb8e1ee607b0a893 pypy-4.0.1-linux-armhf-raring.tar.bz2
e45728d413aa88963d4462ebcfaff6ea pypy-4.0.1-linux-armhf-raspbian.tar.bz2
d1d03aa44df354a3f589473a51406795  pypy-4.0.1-linux.tar.bz2
67ac82d88aaaef8c3074e68d700f3968  pypy-4.0.1-osx64.tar.bz2
f5b35ebedee2fa4fdfee82733be59996  pypy-4.0.1-src.tar.bz2
6b81d79886b215a877fee7624883ee94  pypy-4.0.1-src.zip
dc6573828ee5c82df18f9035f3b19edb  pypy-4.0.1-win32.zip

f723ed8aa51c98dcad703ff87c32ade3a54f0101  pypy-4.0.1-linux64.tar.bz2
947a24eff8f50705b403bd67f3357e135f071036 pypy-4.0.1-linux-armel.tar.bz2
bbcc04d524a5e7634173a7272b233f675a719bbf 
pypy-4.0.1-linux-armhf-raring.tar.bz2
adf73fb7bb8001f549472eba5fee970c1800bc56 
pypy-4.0.1-linux-armhf-raspbian.tar.bz2
b306273b86a86ef3c624ecea7e48e45f660e1ef5  pypy-4.0.1-linux.tar.bz2
75286d96b22040723fe1de061d3df43cea712903  pypy-4.0.1-osx64.tar.bz2
805e9a81528a905afa571aa33cc66c650ae4e9f4  pypy-4.0.1-src.tar.bz2
5f29a916eba5f2b31affa56753b7ca108718d9be  pypy-4.0.1-src.zip
992b110210e556cd6803c2ca3837bec82a09aa15  pypy-4.0.1-win32.zip


From hengha.mao at gmail.com  Wed Nov 18 21:24:01 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Thu, 19 Nov 2015 10:24:01 +0800
Subject: [pypy-dev] How to convert C++ char*(contain \0) to python string in
 PyPy?
Message-ID: <CAOEzsGBRjeBh596k4y7U61XaRyN9CA2if+6v3Hkc16tgY1XFtg@mail.gmail.com>

We used ffi.string() to convert C++ char* to pypy string.
But recently, we found out ffi.string() used '\0' as the indicator of
string termination. And it bring some problems of converting a binary
string.

Our case is we read char*(buf) from a binary file, and buf might contain
'\0'. Using ffi.string() we were not able to convert completed buf to PyPy
string. We've tried ffi.string(buf, length), but it also did not work.
Are there any solutions?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151119/d8e93146/attachment.html>

From fijall at gmail.com  Thu Nov 19 01:04:33 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 19 Nov 2015 08:04:33 +0200
Subject: [pypy-dev] How to convert C++ char*(contain \0) to python
 string in PyPy?
In-Reply-To: <CAOEzsGBRjeBh596k4y7U61XaRyN9CA2if+6v3Hkc16tgY1XFtg@mail.gmail.com>
References: <CAOEzsGBRjeBh596k4y7U61XaRyN9CA2if+6v3Hkc16tgY1XFtg@mail.gmail.com>
Message-ID: <CAK5idxQAqKDkc3wHPSyYjVOPeajmYd2Q=4XfaiL3uuEynK6qow@mail.gmail.com>

Use ffi.buffer and not ffi.string

On Thu, Nov 19, 2015 at 4:24 AM, Yicong Huang <hengha.mao at gmail.com> wrote:
> We used ffi.string() to convert C++ char* to pypy string.
> But recently, we found out ffi.string() used '\0' as the indicator of string
> termination. And it bring some problems of converting a binary string.
>
> Our case is we read char*(buf) from a binary file, and buf might contain
> '\0'. Using ffi.string() we were not able to convert completed buf to PyPy
> string. We've tried ffi.string(buf, length), but it also did not work.
> Are there any solutions?
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From hengha.mao at gmail.com  Thu Nov 19 02:33:31 2015
From: hengha.mao at gmail.com (Yicong Huang)
Date: Thu, 19 Nov 2015 15:33:31 +0800
Subject: [pypy-dev] How to convert C++ char*(contain \0) to python
 string in PyPy?
In-Reply-To: <CAK5idxQAqKDkc3wHPSyYjVOPeajmYd2Q=4XfaiL3uuEynK6qow@mail.gmail.com>
References: <CAOEzsGBRjeBh596k4y7U61XaRyN9CA2if+6v3Hkc16tgY1XFtg@mail.gmail.com>
 <CAK5idxQAqKDkc3wHPSyYjVOPeajmYd2Q=4XfaiL3uuEynK6qow@mail.gmail.com>
Message-ID: <CAOEzsGDws3R4hM1C_ridBKQ7+sGNmKh=90UwYLEcYTppKOh3_Q@mail.gmail.com>

Great thanks Alex and Maciej.
ffi.buffer works as expected.

On Thu, Nov 19, 2015 at 2:04 PM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> Use ffi.buffer and not ffi.string
>
> On Thu, Nov 19, 2015 at 4:24 AM, Yicong Huang <hengha.mao at gmail.com>
> wrote:
> > We used ffi.string() to convert C++ char* to pypy string.
> > But recently, we found out ffi.string() used '\0' as the indicator of
> string
> > termination. And it bring some problems of converting a binary string.
> >
> > Our case is we read char*(buf) from a binary file, and buf might contain
> > '\0'. Using ffi.string() we were not able to convert completed buf to
> PyPy
> > string. We've tried ffi.string(buf, length), but it also did not work.
> > Are there any solutions?
> >
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151119/68bd2936/attachment.html>

From anto.cuni at gmail.com  Sun Nov 22 06:10:59 2015
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sun, 22 Nov 2015 12:10:59 +0100
Subject: [pypy-dev] ARM failing test
Message-ID: <CAHPg78w2RM8Y5TVgPp1SJG2aiDZHqRZkcPD7poNhnB4rB=xR2g@mail.gmail.com>

Hi David,

I am developing the "faster-rstruct" branch, which aims to speedup
struct.unpack by reading the values at once from memory, instead of
byte-by-byte as it's doing right now.

The branch works fine on x86 but fails on armhf (see e.g.
http://buildbot.pypy.org/summary/longrepr?testname=AppTestStruct.%28%29.test_unpack_standard_little&builder=pypy-c-jit-linux-armhf-v7&build=843&mod=module.struct.test.test_struct).
I suspect a big/little endian issue.

Two questions:
1) I know that ARM CPUs can be either little or big endiam. What is the
case for our armhf machine?

2) is it possible to have ssh access to one or more of our ARM machines so
I can test easily?

thank you :)

ciao,
Anto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151122/f85a9acc/attachment.html>

From arigo at tunes.org  Sun Nov 22 09:38:19 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 22 Nov 2015 15:38:19 +0100
Subject: [pypy-dev] ARM failing test
In-Reply-To: <CAHPg78w2RM8Y5TVgPp1SJG2aiDZHqRZkcPD7poNhnB4rB=xR2g@mail.gmail.com>
References: <CAHPg78w2RM8Y5TVgPp1SJG2aiDZHqRZkcPD7poNhnB4rB=xR2g@mail.gmail.com>
Message-ID: <CAMSv6X3_1CECfOhB2W+M8OB4JPEL6NHXaAT6QKv=zuW2JWL5+w@mail.gmail.com>

Hi Anto,

On Sun, Nov 22, 2015 at 12:10 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> I suspect a big/little endian issue.

Fwiw, the error doesn't seem to be an endianness issue, but rather
reading garbage (and I'm pretty sure that all our ARMs are little
endian).


A bient?t,

Armin.

From arigo at tunes.org  Sun Nov 22 09:44:41 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 22 Nov 2015 15:44:41 +0100
Subject: [pypy-dev] ARM failing test
In-Reply-To: <CAMSv6X3_1CECfOhB2W+M8OB4JPEL6NHXaAT6QKv=zuW2JWL5+w@mail.gmail.com>
References: <CAHPg78w2RM8Y5TVgPp1SJG2aiDZHqRZkcPD7poNhnB4rB=xR2g@mail.gmail.com>
 <CAMSv6X3_1CECfOhB2W+M8OB4JPEL6NHXaAT6QKv=zuW2JWL5+w@mail.gmail.com>
Message-ID: <CAMSv6X01YJRZxgX8QDnwoYbv0mm-iYnrwmW=sGfNhPUocgpFqQ@mail.gmail.com>

Hi again,

Here's a guess.  The value expected is 0x4142434445464748.  The value
we read is 0xb3f4bce041424344.  That hints at an off-by-4-bytes issue.
My guess is an alignment issue.  When you force-cast the RPython
string to a type with an 8-bytes-per-item array, on some 32-bit
platforms it might be aligned to 8 bytes anyway.  It is not the case
on Linux x86-32, but it is the case on Win32 for example.  So you
force-cast to a type that looks like this:

   gc header (4 bytes)
   hash (4 bytes)
   length (4 bytes)
   PADDING! (4 bytes)
   array items (8 bytes each)

Maybe you should not force-cast to any type whose size is greater than
a Signed, to avoid the issue.


A bient?t,

Armin.

From david.schneider at bivab.de  Mon Nov 23 03:10:34 2015
From: david.schneider at bivab.de (David Schneider)
Date: Mon, 23 Nov 2015 09:10:34 +0100
Subject: [pypy-dev] ARM failing test
In-Reply-To: <CAHPg78w2RM8Y5TVgPp1SJG2aiDZHqRZkcPD7poNhnB4rB=xR2g@mail.gmail.com>
References: <CAHPg78w2RM8Y5TVgPp1SJG2aiDZHqRZkcPD7poNhnB4rB=xR2g@mail.gmail.com>
Message-ID: <1BECD949-C0A0-4E60-A421-0526F4705191@bivab.de>

Hi Anto,

> On 22.11.2015, at 12:10, Antonio Cuni <anto.cuni at gmail.com> wrote:
> 
> 1) I know that ARM CPUs can be either little or big endiam. What is the case for our armhf machine?

Our builders are all little endian. To my knowledge most (maybe all) Linux distributions for ARM are little endian nowadays.

> 
> 2) is it possible to have ssh access to one or more of our ARM machines so I can test easily?

at least for the raspberry pi builders it is not that easy to provide external access. Maybe on the ?cubieboard-bob? builder, which is managed by Matti, it is possible.

Cheers

David

From robin at reportlab.com  Mon Nov 23 07:47:51 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 23 Nov 2015 12:47:51 +0000
Subject: [pypy-dev] debugging extension segfault
Message-ID: <56530AF7.9090401@chamonix.reportlab.co.uk>

I hope this is the right place to ask, but if not just tell me to go away.


At the request of a user I'm trying to get reportlab up to scratch for pypy. I 
have some fairly insignificant bugs in the tests related to Cpython-pypy 
differences and one C extension that crashes.

Firstly what is the best way to approach debugging an extension segfault. Do I 
need to build pypy with debugging turned on or can I just build the extension 
like that?

Are there any guidelines for making c extensions pypy friendly?

Secondly how to cope with some fairly simple testing problems. One of my 
extensions appears to work, but clearly attempting to verify reference counts is 
failing

ie from sys import getrefcount, _getframe

doesn't work. I suppose there's no real point in trying to figure out refcounts 
for pypy.

I have some errors related to co_filename eg

>    if os.path.splitext(obj.__code__.co_filename)[0]==modBn:
> AttributeError: 'builtin-code' object has no attribute 'co_filename'

but it's not clear to me if it's even possible to get the filename.


Lastly what is the right way to get the platform as pypy not cpython?
-- 
Robin Becker

From fijall at gmail.com  Mon Nov 23 09:45:21 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 23 Nov 2015 16:45:21 +0200
Subject: [pypy-dev] debugging extension segfault
In-Reply-To: <56530AF7.9090401@chamonix.reportlab.co.uk>
References: <56530AF7.9090401@chamonix.reportlab.co.uk>
Message-ID: <CAK5idxTGCosgS4vQ=O4o=m_Bjq78jHVVgFpVqnDDmWJt0eDOtw@mail.gmail.com>

as far as I remember, C extension in reportlab is optional. Just
disable it. If it's there for speedups, it's very likely it slows
things down on pypy.

Reference count is a fake thing on pypy, don't use it if you can help it.

builtin codes have no filename, you have to somehow deal with it. I
think the problem comes from a fact that on cpython builtin functions
don't have code.

On Mon, Nov 23, 2015 at 2:47 PM, Robin Becker <robin at reportlab.com> wrote:
> I hope this is the right place to ask, but if not just tell me to go away.
>
>
> At the request of a user I'm trying to get reportlab up to scratch for pypy.
> I have some fairly insignificant bugs in the tests related to Cpython-pypy
> differences and one C extension that crashes.
>
> Firstly what is the best way to approach debugging an extension segfault. Do
> I need to build pypy with debugging turned on or can I just build the
> extension like that?
>
> Are there any guidelines for making c extensions pypy friendly?
>
> Secondly how to cope with some fairly simple testing problems. One of my
> extensions appears to work, but clearly attempting to verify reference
> counts is failing
>
> ie from sys import getrefcount, _getframe
>
> doesn't work. I suppose there's no real point in trying to figure out
> refcounts for pypy.
>
> I have some errors related to co_filename eg
>
>>    if os.path.splitext(obj.__code__.co_filename)[0]==modBn:
>> AttributeError: 'builtin-code' object has no attribute 'co_filename'
>
>
> but it's not clear to me if it's even possible to get the filename.
>
>
> Lastly what is the right way to get the platform as pypy not cpython?
> --
> Robin Becker
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From lac at openend.se  Mon Nov 23 10:30:59 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 23 Nov 2015 16:30:59 +0100
Subject: [pypy-dev] debugging extension segfault
In-Reply-To: <CAK5idxTGCosgS4vQ=O4o=m_Bjq78jHVVgFpVqnDDmWJt0eDOtw@mail.gmail.com>
References: <56530AF7.9090401@chamonix.reportlab.co.uk>
 <CAK5idxTGCosgS4vQ=O4o=m_Bjq78jHVVgFpVqnDDmWJt0eDOtw@mail.gmail.com>
Message-ID: <201511231530.tANFUxwi000422@fido.openend.se>

>> Lastly what is the right way to get the platform as pypy not cpython?
>> --
>> Robin Becker

Maciej already answered the other stuff.  I had pypy working with 
reportlab a long time ago just by not using the C extensions at all.

Detecting pypy is a bit harder.
https://www.python.org/dev/peps/pep-0421/#version-format
was supposed to fix things


lac at smartwheels:~$ python3.5
Python 3.5.0rc1 (default, Aug 12 2015, 14:57:46) 
[GCC 5.2.1 20150808] on linux
Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> sys.implementation
namespace(_multiarch='x86_64-linux-gnu', cache_tag='cpython-35', hexversion=50659521, name='cpython', version=sys.version_info(major=3, minor=5, micro=0, releaselevel='candidate', serial=1))
 >>> 

Okay, you could fish around in that and get what you are looking for.

But

Python 2.7.10 (default, Jul  1 2015, 10:54:53) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> sys.implementation
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'implementation'

This won't.

So I just try to import __pypy__

If it succeeds, I have pypy, and if it doesn't I assume CPython.

Laura


From robin at reportlab.com  Tue Nov 24 05:33:44 2015
From: robin at reportlab.com (Robin Becker)
Date: Tue, 24 Nov 2015 10:33:44 +0000
Subject: [pypy-dev] debugging extension segfault
In-Reply-To: <CAK5idxTGCosgS4vQ=O4o=m_Bjq78jHVVgFpVqnDDmWJt0eDOtw@mail.gmail.com>
References: <56530AF7.9090401@chamonix.reportlab.co.uk>
 <CAK5idxTGCosgS4vQ=O4o=m_Bjq78jHVVgFpVqnDDmWJt0eDOtw@mail.gmail.com>
Message-ID: <56543D08.1030402@chamonix.reportlab.co.uk>

Thanks Maciej,


On 23/11/2015 14:45, Maciej Fijalkowski wrote:
> as far as I remember, C extension in reportlab is optional. Just
> disable it. If it's there for speedups, it's very likely it slows
> things down on pypy.

I know that reportlab can run without the _rl_accel extension, but the one 
that's seg faulting is _renderPM which we use to create pix maps (ie raw rgb 
images). I don't have a python alternative for that so some RL features are 
unsupported. For PDF only pypy seems to work fine.

>
> Reference count is a fake thing on pypy, don't use it if you can help it.
>

I think the reference counts are just used to try and see that the _rl_accel.pyd 
is doing the right thing wrt refcounts.

> builtin codes have no filename, you have to somehow deal with it. I
> think the problem comes from a fact that on cpython builtin functions
> don't have code.
>
the test that fails is just trying to check doctests.

.........

A bit of googling tells me I should be using platform.python_implementation() to 
check for pypy.
-- 
Robin Becker

From arigo at tunes.org  Tue Nov 24 11:10:45 2015
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 24 Nov 2015 17:10:45 +0100
Subject: [pypy-dev] debugging extension segfault
In-Reply-To: <201511231530.tANFUxwi000422@fido.openend.se>
References: <56530AF7.9090401@chamonix.reportlab.co.uk>
 <CAK5idxTGCosgS4vQ=O4o=m_Bjq78jHVVgFpVqnDDmWJt0eDOtw@mail.gmail.com>
 <201511231530.tANFUxwi000422@fido.openend.se>
Message-ID: <CAMSv6X0bDOyo0=ycwp7YBHdDWcG=7LuyME3_g-f5B_ZDZKy00Q@mail.gmail.com>

Hi Laura, hi Robin,

On Mon, Nov 23, 2015 at 4:30 PM, Laura Creighton <lac at openend.se> wrote:
>>> Lastly what is the right way to get the platform as pypy not cpython?
>
> Detecting pypy is a bit harder.

No, no: the official way since many years is

    if "__pypy__" in sys.builtin_module_names:


A bient?t,

Armin.

From luisjosenovoa at gmail.com  Tue Nov 24 12:30:33 2015
From: luisjosenovoa at gmail.com (=?UTF-8?Q?Luis_Jos=C3=A9_Novoa?=)
Date: Tue, 24 Nov 2015 12:30:33 -0500
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
Message-ID: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>

Hi everyone.

Im trying to solve some problems in parallel using the multiprocessing
module in Python 2.7. A general model is built using CPython and then the
subproblems are solved in parallel returning results in a queue. This is
currently working fine. I would like to solve the subproblems using PyPy to
increase speed.
I found
http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy ,
but there it says that the procedure only works with CPython 3.4.

I wonder is there is any clean direct way to do this.

Appreciate any help.

-- 
Luis J. Novoa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151124/2e931344/attachment.html>

From fijall at gmail.com  Tue Nov 24 15:23:43 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 24 Nov 2015 22:23:43 +0200
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
Message-ID: <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>

Hi Luis.

Multiprocessing works under pypy, so just run your program under pypy
with no changes and see what happens

On Tue, Nov 24, 2015 at 7:30 PM, Luis Jos? Novoa
<luisjosenovoa at gmail.com> wrote:
> Hi everyone.
>
> Im trying to solve some problems in parallel using the multiprocessing
> module in Python 2.7. A general model is built using CPython and then the
> subproblems are solved in parallel returning results in a queue. This is
> currently working fine. I would like to solve the subproblems using PyPy to
> increase speed.
> I found
> http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy ,
> but there it says that the procedure only works with CPython 3.4.
>
> I wonder is there is any clean direct way to do this.
>
> Appreciate any help.
>
> --
> Luis J. Novoa
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>

From luisjosenovoa at gmail.com  Tue Nov 24 15:35:26 2015
From: luisjosenovoa at gmail.com (=?UTF-8?Q?Luis_Jos=C3=A9_Novoa?=)
Date: Tue, 24 Nov 2015 15:35:26 -0500
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
Message-ID: <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>

Hi Maciej,

Thanks for your reply. The problem is that Im using gurobipy (for the
interaction with the mathematical programming solver Gurobi) to solve a
master problem and then using the output to solve the subproblems. Now,
gurobipy is not compatible wiith PyPy.


On Tue, Nov 24, 2015 at 3:23 PM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> Hi Luis.
>
> Multiprocessing works under pypy, so just run your program under pypy
> with no changes and see what happens
>
> On Tue, Nov 24, 2015 at 7:30 PM, Luis Jos? Novoa
> <luisjosenovoa at gmail.com> wrote:
> > Hi everyone.
> >
> > Im trying to solve some problems in parallel using the multiprocessing
> > module in Python 2.7. A general model is built using CPython and then the
> > subproblems are solved in parallel returning results in a queue. This is
> > currently working fine. I would like to solve the subproblems using PyPy
> to
> > increase speed.
> > I found
> > http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy
> ,
> > but there it says that the procedure only works with CPython 3.4.
> >
> > I wonder is there is any clean direct way to do this.
> >
> > Appreciate any help.
> >
> > --
> > Luis J. Novoa
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
>



-- 
Luis J. Novoa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151124/95aba394/attachment.html>

From fijall at gmail.com  Tue Nov 24 15:38:54 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 24 Nov 2015 22:38:54 +0200
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
Message-ID: <CAK5idxRG8V5TzSB=6AdEL4K8hAeo-KiGWkb_m+XNuLF0EiPXOQ@mail.gmail.com>

you can try using execnet?

On Tue, Nov 24, 2015 at 10:35 PM, Luis Jos? Novoa
<luisjosenovoa at gmail.com> wrote:
> Hi Maciej,
>
> Thanks for your reply. The problem is that Im using gurobipy (for the
> interaction with the mathematical programming solver Gurobi) to solve a
> master problem and then using the output to solve the subproblems. Now,
> gurobipy is not compatible wiith PyPy.
>
>
> On Tue, Nov 24, 2015 at 3:23 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>>
>> Hi Luis.
>>
>> Multiprocessing works under pypy, so just run your program under pypy
>> with no changes and see what happens
>>
>> On Tue, Nov 24, 2015 at 7:30 PM, Luis Jos? Novoa
>> <luisjosenovoa at gmail.com> wrote:
>> > Hi everyone.
>> >
>> > Im trying to solve some problems in parallel using the multiprocessing
>> > module in Python 2.7. A general model is built using CPython and then
>> > the
>> > subproblems are solved in parallel returning results in a queue. This is
>> > currently working fine. I would like to solve the subproblems using PyPy
>> > to
>> > increase speed.
>> > I found
>> > http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy
>> > ,
>> > but there it says that the procedure only works with CPython 3.4.
>> >
>> > I wonder is there is any clean direct way to do this.
>> >
>> > Appreciate any help.
>> >
>> > --
>> > Luis J. Novoa
>> >
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pypy-dev
>> >
>
>
>
>
> --
> Luis J. Novoa

From lac at openend.se  Tue Nov 24 15:45:43 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 24 Nov 2015 21:45:43 +0100
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
Message-ID: <201511242045.tAOKjhvq002197@fido.openend.se>

In a message of Tue, 24 Nov 2015 15:35:26 -0500, Luis Jos? Novoa writes:
>Hi Maciej,
>
>Thanks for your reply. The problem is that Im using gurobipy (for the
>interaction with the mathematical programming solver Gurobi) to solve a
>master problem and then using the output to solve the subproblems. Now,
>gurobipy is not compatible wiith PyPy.

You've tried and it doesn't work?  Gorobipy wants numpy in order
to work, but the question is _how much numpy_?  If you are lucky,
the parts it needs are already in pypy's numpy support.

Laura


From luisjosenovoa at gmail.com  Tue Nov 24 16:31:14 2015
From: luisjosenovoa at gmail.com (=?UTF-8?Q?Luis_Jos=C3=A9_Novoa?=)
Date: Tue, 24 Nov 2015 16:31:14 -0500
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <201511242045.tAOKjhvq002197@fido.openend.se>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
 <201511242045.tAOKjhvq002197@fido.openend.se>
Message-ID: <CAFKi5F-pvAkAzODeGLPLEwC=hcHcURdjaFWbSCjRi7U=5OTRhQ@mail.gmail.com>

I get the following:

\Downloads\pypy-4.0.0-win32\pypy-4.0.0-win32\site-packages\gurobipy\__init__.py",
line 1, in <module>
    from .gurobipy import *
ImportError: No module named gurobipy.gurobipy


After installing it as pypy setup,py install

On Tue, Nov 24, 2015 at 3:45 PM, Laura Creighton <lac at openend.se> wrote:

> In a message of Tue, 24 Nov 2015 15:35:26 -0500, Luis Jos? Novoa writes:
> >Hi Maciej,
> >
> >Thanks for your reply. The problem is that Im using gurobipy (for the
> >interaction with the mathematical programming solver Gurobi) to solve a
> >master problem and then using the output to solve the subproblems. Now,
> >gurobipy is not compatible wiith PyPy.
>
> You've tried and it doesn't work?  Gorobipy wants numpy in order
> to work, but the question is _how much numpy_?  If you are lucky,
> the parts it needs are already in pypy's numpy support.
>
> Laura
>
>


-- 
Luis J. Novoa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151124/0b87b4c6/attachment-0001.html>

From luisjosenovoa at gmail.com  Tue Nov 24 17:38:02 2015
From: luisjosenovoa at gmail.com (=?UTF-8?Q?Luis_Jos=C3=A9_Novoa?=)
Date: Tue, 24 Nov 2015 17:38:02 -0500
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <201511242045.tAOKjhvq002197@fido.openend.se>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
 <201511242045.tAOKjhvq002197@fido.openend.se>
Message-ID: <CAFKi5F-r_2v3qd0NUXaLP-WPCVjk1HobtdaRAA+4cHWSAYOnjw@mail.gmail.com>

Maybe Im doing something wrong? Appreciate all the help

[image: Inline image 1]

On Tue, Nov 24, 2015 at 3:45 PM, Laura Creighton <lac at openend.se> wrote:

> In a message of Tue, 24 Nov 2015 15:35:26 -0500, Luis Jos? Novoa writes:
> >Hi Maciej,
> >
> >Thanks for your reply. The problem is that Im using gurobipy (for the
> >interaction with the mathematical programming solver Gurobi) to solve a
> >master problem and then using the output to solve the subproblems. Now,
> >gurobipy is not compatible wiith PyPy.
>
> You've tried and it doesn't work?  Gorobipy wants numpy in order
> to work, but the question is _how much numpy_?  If you are lucky,
> the parts it needs are already in pypy's numpy support.
>
> Laura
>
>


-- 
Luis J. Novoa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151124/a8a823a6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 39700 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151124/a8a823a6/attachment-0001.png>

From fijall at gmail.com  Wed Nov 25 01:32:15 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 25 Nov 2015 08:32:15 +0200
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAFKi5F-r_2v3qd0NUXaLP-WPCVjk1HobtdaRAA+4cHWSAYOnjw@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
 <201511242045.tAOKjhvq002197@fido.openend.se>
 <CAFKi5F-r_2v3qd0NUXaLP-WPCVjk1HobtdaRAA+4cHWSAYOnjw@mail.gmail.com>
Message-ID: <CAK5idxTF9T5Z00TkB1oMA6Lrdu9LopYXuBGu1J+nCWxihPEpXQ@mail.gmail.com>

your install looks somehow wrong. I would suggest you use virtualenv

On Wed, Nov 25, 2015 at 12:38 AM, Luis Jos? Novoa <luisjosenovoa at gmail.com>
wrote:

> Maybe Im doing something wrong? Appreciate all the help
>
> [image: Inline image 1]
>
> On Tue, Nov 24, 2015 at 3:45 PM, Laura Creighton <lac at openend.se> wrote:
>
>> In a message of Tue, 24 Nov 2015 15:35:26 -0500, Luis Jos? Novoa writes:
>> >Hi Maciej,
>> >
>> >Thanks for your reply. The problem is that Im using gurobipy (for the
>> >interaction with the mathematical programming solver Gurobi) to solve a
>> >master problem and then using the output to solve the subproblems. Now,
>> >gurobipy is not compatible wiith PyPy.
>>
>> You've tried and it doesn't work?  Gorobipy wants numpy in order
>> to work, but the question is _how much numpy_?  If you are lucky,
>> the parts it needs are already in pypy's numpy support.
>>
>> Laura
>>
>>
>
>
> --
> Luis J. Novoa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151125/06d8311e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 39700 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151125/06d8311e/attachment-0001.png>

From anto.cuni at gmail.com  Wed Nov 25 08:04:43 2015
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 25 Nov 2015 14:04:43 +0100
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
Message-ID: <CAHPg78wGm+YTR0c0w=0E0xHxdp24BY6NwrmmzU3NHp1n7sZ--g@mail.gmail.com>

fwiw, you might want to look at this:
https://github.com/felipecruz/gurobi_cffi

from what I read in the readme, it probably exposes a different interface
than gurobipy, but it might enough for your use case.


On Tue, Nov 24, 2015 at 9:35 PM, Luis Jos? Novoa <luisjosenovoa at gmail.com>
wrote:

> Hi Maciej,
>
> Thanks for your reply. The problem is that Im using gurobipy (for the
> interaction with the mathematical programming solver Gurobi) to solve a
> master problem and then using the output to solve the subproblems. Now,
> gurobipy is not compatible wiith PyPy.
>
>
> On Tue, Nov 24, 2015 at 3:23 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>
>> Hi Luis.
>>
>> Multiprocessing works under pypy, so just run your program under pypy
>> with no changes and see what happens
>>
>> On Tue, Nov 24, 2015 at 7:30 PM, Luis Jos? Novoa
>> <luisjosenovoa at gmail.com> wrote:
>> > Hi everyone.
>> >
>> > Im trying to solve some problems in parallel using the multiprocessing
>> > module in Python 2.7. A general model is built using CPython and then
>> the
>> > subproblems are solved in parallel returning results in a queue. This is
>> > currently working fine. I would like to solve the subproblems using
>> PyPy to
>> > increase speed.
>> > I found
>> > http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy
>> ,
>> > but there it says that the procedure only works with CPython 3.4.
>> >
>> > I wonder is there is any clean direct way to do this.
>> >
>> > Appreciate any help.
>> >
>> > --
>> > Luis J. Novoa
>> >
>> > _______________________________________________
>> > pypy-dev mailing list
>> > pypy-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pypy-dev
>> >
>>
>
>
>
> --
> Luis J. Novoa
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151125/88059dbd/attachment.html>

From luisjosenovoa at gmail.com  Wed Nov 25 08:24:04 2015
From: luisjosenovoa at gmail.com (=?UTF-8?Q?Luis_Jos=C3=A9_Novoa?=)
Date: Wed, 25 Nov 2015 08:24:04 -0500
Subject: [pypy-dev] Multiprocessing - CPython and PyPy
In-Reply-To: <CAHPg78wGm+YTR0c0w=0E0xHxdp24BY6NwrmmzU3NHp1n7sZ--g@mail.gmail.com>
References: <CAFKi5F-GFH3ZbB3rbBwYRgJBWjb+JLKxBNwKV-1LouTAtrO1jw@mail.gmail.com>
 <CAK5idxQa1=bUcFgsnUZW8xsXnFX2UXomLQZ7xwntoXoED+S-tA@mail.gmail.com>
 <CAFKi5F8M73mx9b=Bh6C==1oKf0v0if4VMiytS=fNDrcE7zGT1A@mail.gmail.com>
 <CAHPg78wGm+YTR0c0w=0E0xHxdp24BY6NwrmmzU3NHp1n7sZ--g@mail.gmail.com>
Message-ID: <CAFKi5F9DiyO_ndkGE+6KAFQ21M3LoOdBKZ-FbMqwiRn0Dv6X6Q@mail.gmail.com>

Thank you for your reply. I've looked into
https://github.com/felipecruz/gurobi_cffi before, and, unfortunately, it
does not have the full capabilities of gurobipy

LJN
On Nov 25, 2015 8:05 AM, "Antonio Cuni" <anto.cuni at gmail.com> wrote:

> fwiw, you might want to look at this:
> https://github.com/felipecruz/gurobi_cffi
>
> from what I read in the readme, it probably exposes a different interface
> than gurobipy, but it might enough for your use case.
>
>
> On Tue, Nov 24, 2015 at 9:35 PM, Luis Jos? Novoa <luisjosenovoa at gmail.com>
> wrote:
>
>> Hi Maciej,
>>
>> Thanks for your reply. The problem is that Im using gurobipy (for the
>> interaction with the mathematical programming solver Gurobi) to solve a
>> master problem and then using the output to solve the subproblems. Now,
>> gurobipy is not compatible wiith PyPy.
>>
>>
>> On Tue, Nov 24, 2015 at 3:23 PM, Maciej Fijalkowski <fijall at gmail.com>
>> wrote:
>>
>>> Hi Luis.
>>>
>>> Multiprocessing works under pypy, so just run your program under pypy
>>> with no changes and see what happens
>>>
>>> On Tue, Nov 24, 2015 at 7:30 PM, Luis Jos? Novoa
>>> <luisjosenovoa at gmail.com> wrote:
>>> > Hi everyone.
>>> >
>>> > Im trying to solve some problems in parallel using the multiprocessing
>>> > module in Python 2.7. A general model is built using CPython and then
>>> the
>>> > subproblems are solved in parallel returning results in a queue. This
>>> is
>>> > currently working fine. I would like to solve the subproblems using
>>> PyPy to
>>> > increase speed.
>>> > I found
>>> >
>>> http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy
>>> ,
>>> > but there it says that the procedure only works with CPython 3.4.
>>> >
>>> > I wonder is there is any clean direct way to do this.
>>> >
>>> > Appreciate any help.
>>> >
>>> > --
>>> > Luis J. Novoa
>>> >
>>> > _______________________________________________
>>> > pypy-dev mailing list
>>> > pypy-dev at python.org
>>> > https://mail.python.org/mailman/listinfo/pypy-dev
>>> >
>>>
>>
>>
>>
>> --
>> Luis J. Novoa
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151125/623372f9/attachment.html>

From arigo at tunes.org  Fri Nov 27 05:18:05 2015
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 27 Nov 2015 11:18:05 +0100
Subject: [pypy-dev] Leysin Winter sprint?
Message-ID: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>

Hi all,

Due to public pressure :-)  I'm trying to organize this winter's
sprint, in Leysin, Switzerland.  This is a fully public sprint of 7
days, with possible skiing.

If you'd like to come, please reply to this e-mail with dates that are
ok for you.  I will give some priority to core people, but anyone else
is welcome to have preferences too.  Traditionally, it is around
mid-January, but this year as far as I understand there is some push
to have it in February or March instead---which is fine too.  The
following week should be avoided if possible, as it is holidays for
the canton's schools: 20-28 Feb 2016.


A bient?t,

Armin.

From matti.picus at gmail.com  Fri Nov 27 05:30:06 2015
From: matti.picus at gmail.com (Matti Picus)
Date: Fri, 27 Nov 2015 12:30:06 +0200
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
Message-ID: <565830AE.9040809@gmail.com>


On 27/11/15 12:18, Armin Rigo wrote:
> Hi all,
>
> Due to public pressure :-)  I'm trying to organize this winter's
> sprint, in Leysin, Switzerland.  This is a fully public sprint of 7
> days, with possible skiing.
>
> If you'd like to come, please reply to this e-mail with dates that are
> ok for you.  I will give some priority to core people, but anyone else
> is welcome to have preferences too.  Traditionally, it is around
> mid-January, but this year as far as I understand there is some push
> to have it in February or March instead---which is fine too.  The
> following week should be avoided if possible, as it is holidays for
> the canton's schools: 20-28 Feb 2016.
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
I plan on attending, I would prefer early February but am flexible.
Matti

From cfbolz at gmx.de  Fri Nov 27 05:35:01 2015
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 27 Nov 2015 11:35:01 +0100
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
Message-ID: <c36afe67-c0d0-4e4a-bc79-61bc67a31fba@email.android.com>

Hi all,

I'll try to attend next year, I would prefer February (I'm away in a yet to be determined week in January). 

Cheers, 
Carl Friedrich

On November 27, 2015 11:18:05 AM GMT+01:00, Armin Rigo <arigo at tunes.org> wrote:
>Hi all,
>
>Due to public pressure :-)  I'm trying to organize this winter's
>sprint, in Leysin, Switzerland.  This is a fully public sprint of 7
>days, with possible skiing.
>
>If you'd like to come, please reply to this e-mail with dates that are
>ok for you.  I will give some priority to core people, but anyone else
>is welcome to have preferences too.  Traditionally, it is around
>mid-January, but this year as far as I understand there is some push
>to have it in February or March instead---which is fine too.  The
>following week should be avoided if possible, as it is holidays for
>the canton's schools: 20-28 Feb 2016.
>
>
>A bient?t,
>
>Armin.
>_______________________________________________
>pypy-dev mailing list
>pypy-dev at python.org
>https://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151127/b0225c80/attachment.html>

From planrichi at gmail.com  Fri Nov 27 06:16:58 2015
From: planrichi at gmail.com (Richard Plangger)
Date: Fri, 27 Nov 2015 12:16:58 +0100
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
Message-ID: <56583BAA.7030909@gmail.com>

Hi,

I would like to come. Date: I have no preference and I'm flexible as well.


Cheers,
Richard

On 11/27/2015 11:18 AM, Armin Rigo wrote:
> Hi all,
> 
> Due to public pressure :-)  I'm trying to organize this winter's
> sprint, in Leysin, Switzerland.  This is a fully public sprint of 7
> days, with possible skiing.
> 
> If you'd like to come, please reply to this e-mail with dates that are
> ok for you.  I will give some priority to core people, but anyone else
> is welcome to have preferences too.  Traditionally, it is around
> mid-January, but this year as far as I understand there is some push
> to have it in February or March instead---which is fine too.  The
> following week should be avoided if possible, as it is holidays for
> the canton's schools: 20-28 Feb 2016.
> 
> 
> A bient?t,
> 
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
> 

From remi.meier at inf.ethz.ch  Fri Nov 27 11:47:10 2015
From: remi.meier at inf.ethz.ch (Remi Meier)
Date: Fri, 27 Nov 2015 17:47:10 +0100
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
Message-ID: <CAJ8fdYyb8eWo0XUFq0NZpd-Uy=OaWTV85Ho+poepbEx7jpPavg@mail.gmail.com>

Hi,

I would like to come again if possible. Jan and Feb work for me, but
I'm away starting March 1st.

Cheers,
Remi

On 27 November 2015 at 11:18, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Due to public pressure :-)  I'm trying to organize this winter's
> sprint, in Leysin, Switzerland.  This is a fully public sprint of 7
> days, with possible skiing.
>
> If you'd like to come, please reply to this e-mail with dates that are
> ok for you.  I will give some priority to core people, but anyone else
> is welcome to have preferences too.  Traditionally, it is around
> mid-January, but this year as far as I understand there is some push
> to have it in February or March instead---which is fine too.  The
> following week should be avoided if possible, as it is holidays for
> the canton's schools: 20-28 Feb 2016.
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From fijall at gmail.com  Sun Nov 29 04:28:47 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 29 Nov 2015 11:28:47 +0200
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAJ8fdYyb8eWo0XUFq0NZpd-Uy=OaWTV85Ho+poepbEx7jpPavg@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
 <CAJ8fdYyb8eWo0XUFq0NZpd-Uy=OaWTV85Ho+poepbEx7jpPavg@mail.gmail.com>
Message-ID: <CAK5idxQHMJdR6Ksn2vhxMTNsYcCdzkBeoDE-3_rihkzFUB7dDw@mail.gmail.com>

Hi. I would prefer Feb and I will have a much stronger opinion on the
exact dates in a week or so

On Fri, Nov 27, 2015 at 6:47 PM, Remi Meier <remi.meier at inf.ethz.ch> wrote:
> Hi,
>
> I would like to come again if possible. Jan and Feb work for me, but
> I'm away starting March 1st.
>
> Cheers,
> Remi
>
> On 27 November 2015 at 11:18, Armin Rigo <arigo at tunes.org> wrote:
>> Hi all,
>>
>> Due to public pressure :-)  I'm trying to organize this winter's
>> sprint, in Leysin, Switzerland.  This is a fully public sprint of 7
>> days, with possible skiing.
>>
>> If you'd like to come, please reply to this e-mail with dates that are
>> ok for you.  I will give some priority to core people, but anyone else
>> is welcome to have preferences too.  Traditionally, it is around
>> mid-January, but this year as far as I understand there is some push
>> to have it in February or March instead---which is fine too.  The
>> following week should be avoided if possible, as it is holidays for
>> the canton's schools: 20-28 Feb 2016.
>>
>>
>> A bient?t,
>>
>> Armin.
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From arigo at tunes.org  Mon Nov 30 02:42:55 2015
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 30 Nov 2015 08:42:55 +0100
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAK5idxQHMJdR6Ksn2vhxMTNsYcCdzkBeoDE-3_rihkzFUB7dDw@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
 <CAJ8fdYyb8eWo0XUFq0NZpd-Uy=OaWTV85Ho+poepbEx7jpPavg@mail.gmail.com>
 <CAK5idxQHMJdR6Ksn2vhxMTNsYcCdzkBeoDE-3_rihkzFUB7dDw@mail.gmail.com>
Message-ID: <CAMSv6X0sdg5qp6gHuYRh4iucLvusXezEqJfrHhzh92vkW36muw@mail.gmail.com>

Hi all,

Laura pointed me to the Swiss Python Summit
(http://www.python-summit.ch/), a small conference on February 5th.
It is, however, a single day and at the opposite end of Switzerland
(4h15 by train).  Still, maybe we could set the sprint up the
following week (~7-13) just in case someone wants to attend both.


A bient?t,

Armin.

From fijall at gmail.com  Mon Nov 30 03:17:44 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 30 Nov 2015 10:17:44 +0200
Subject: [pypy-dev] Leysin Winter sprint?
In-Reply-To: <CAMSv6X0sdg5qp6gHuYRh4iucLvusXezEqJfrHhzh92vkW36muw@mail.gmail.com>
References: <CAMSv6X3eFrL3RZXjVTMGgqSE-4hd-=Db+S0bDgghpdhZrkscxg@mail.gmail.com>
 <CAJ8fdYyb8eWo0XUFq0NZpd-Uy=OaWTV85Ho+poepbEx7jpPavg@mail.gmail.com>
 <CAK5idxQHMJdR6Ksn2vhxMTNsYcCdzkBeoDE-3_rihkzFUB7dDw@mail.gmail.com>
 <CAMSv6X0sdg5qp6gHuYRh4iucLvusXezEqJfrHhzh92vkW36muw@mail.gmail.com>
Message-ID: <CAK5idxTQo2eNqWkZ=ETM20cewAre6+mr51dk6-2oUz_ovZMDcA@mail.gmail.com>

Those dates really don't work out for me. I would prefer near the end
of Feb/beginning of Mar or something like that

On Mon, Nov 30, 2015 at 9:42 AM, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Laura pointed me to the Swiss Python Summit
> (http://www.python-summit.ch/), a small conference on February 5th.
> It is, however, a single day and at the opposite end of Switzerland
> (4h15 by train).  Still, maybe we could set the sprint up the
> following week (~7-13) just in case someone wants to attend both.
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev