From tracker at bugs.pypy.org  Sun Dec  1 23:27:24 2013
From: tracker at bugs.pypy.org (mmaenpaa)
Date: Sun, 01 Dec 2013 22:27:24 +0000
Subject: [pypy-issue] [issue1347] Perf problem with many regular expressions
In-Reply-To: <1355435752.78.0.072023264159.issue1347@bugs.pypy.org>
Message-ID: <1385936844.32.0.998838961088.issue1347@bugs.pypy.org>


mmaenpaa <mika.j.maenpaa at iki.fi> added the comment:

It seems that Pypy's jit has real problems when dealing with large number of 
regular expressions that are used often. CPython and Pypy without jit 
don't have problems and they seem to scale linearly when increasing number of 
regular expressions.

Steps to test slowdown when creating and using many regular expressions at the
same time:

1. Create and compile n number of random regular expressions
2. Create predetermined number of random strings
3. Measure time it takes to test all generated strings against all 
   regular expressions with re.match

When testing with 20000 random strings, Pypy starts to slowdown when dealing 
with more than 10 regular expressions. When dealing with more than 25 regular 
expressions, Pypy is actually faster with jit disabled.

Attached test program was run with Pypy 2.2.1 and CPython 2.7.3 on 64-bit 
Debian Wheezy. Jit was disabled with "--jit off" command line parameter.

$ python run-regexp-bug.py 20000
random strings: 20000
n    pypy(s)    cpython(s) pypy-nojit(s)
1    0.006      0.010      0.020     
5    0.029      0.039      0.065     
10   0.054      0.078      0.125     
25   0.403      0.197      0.298     
50   0.789      0.391      0.616     
75   1.405      0.576      0.944     
100  2.049      0.763      1.193     
200  7.454      1.606      2.459     
300  18.311     2.313      3.816     
400  30.136     3.079      4.944     
500  44.634     3.849      6.282     
600  63.083     4.871      8.198     
700  79.475     5.488      8.837     
800  98.820     6.152      10.070    
900  122.998    7.306      11.248    
1000 143.714    8.037      12.766    
1100 171.839    8.479      13.758    
1200 200.731    9.274      15.087

----------
nosy: +mmaenpaa
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1347>
________________________________________

From tracker at bugs.pypy.org  Mon Dec  2 14:22:22 2013
From: tracker at bugs.pypy.org (handloomweaver)
Date: Mon, 02 Dec 2013 13:22:22 +0000
Subject: [pypy-issue] [issue1648] Seg Fault 11 using numpy.zeros PYPY2.2
In-Reply-To: <1385990542.02.0.379761913121.issue1648@bugs.pypy.org>
Message-ID: <1385990542.02.0.379761913121.issue1648@bugs.pypy.org>


New submission from handloomweaver <a.w.macmillan at gmail.com>:

>>>> import numpy as np
>>>> x = np.zeros((2,2), dtype=[('a', np.int32), ('b', np.float64, (3,3))])
>>>> x
Segmentation fault: 11

on OSX 10.8 & Ubuntu 12

----------
messages: 6387
nosy: handloomweaver, pypy-issue
priority: bug
release: 2.2
status: unread
title: Seg Fault 11 using numpy.zeros PYPY2.2

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1648>
________________________________________

From tracker at bugs.pypy.org  Tue Dec  3 06:20:51 2013
From: tracker at bugs.pypy.org (mattip)
Date: Tue, 03 Dec 2013 05:20:51 +0000
Subject: [pypy-issue] [issue1648] Seg Fault 11 using numpy.zeros PYPY2.2
In-Reply-To: <1385990542.02.0.379761913121.issue1648@bugs.pypy.org>
Message-ID: <1386048051.24.0.0768774714341.issue1648@bugs.pypy.org>


mattip <matti.picus at gmail.com> added the comment:

Could you try a later version?
There was an issue with str(record_array) which was fixed for 2.2.1 by the 
voidtype_strformat branch. Note that the _creation_ works fine, it's just the 
printing that causes the segfault.

----------
nosy: +mattip
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1648>
________________________________________

From tracker at bugs.pypy.org  Tue Dec  3 14:59:59 2013
From: tracker at bugs.pypy.org (handloomweaver)
Date: Tue, 03 Dec 2013 13:59:59 +0000
Subject: [pypy-issue] [issue1648] Seg Fault 11 using numpy.zeros PYPY2.2
In-Reply-To: <1385990542.02.0.379761913121.issue1648@bugs.pypy.org>
Message-ID: <1386079199.14.0.57595772966.issue1648@bugs.pypy.org>


handloomweaver <a.w.macmillan at gmail.com> added the comment:

Tested on 2.21 binary on OSX. Same result: Segmentation fault: 11

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1648>
________________________________________

From tracker at bugs.pypy.org  Tue Dec  3 15:30:07 2013
From: tracker at bugs.pypy.org (Amaury Forgeot d'Arc)
Date: Tue, 03 Dec 2013 14:30:07 +0000
Subject: [pypy-issue] [issue1648] Seg Fault 11 using numpy.zeros PYPY2.2
In-Reply-To: <1385990542.02.0.379761913121.issue1648@bugs.pypy.org>
Message-ID: <1386081007.39.0.930954688293.issue1648@bugs.pypy.org>


Amaury Forgeot d'Arc <amauryfa at gmail.com> added the comment:

It seems that 2.2.1 does not contain voidtype_strformat.

My 2.2.1-linux64 is at revision 87aa9de10f9c.
With this version, on pyinteractive.py I get a "'VoidType' object has no attribute 'str_format'"
This is fixed on head.

----------
nosy: +amaury

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1648>
________________________________________

From tracker at bugs.pypy.org  Tue Dec  3 16:07:40 2013
From: tracker at bugs.pypy.org (handloomweaver)
Date: Tue, 03 Dec 2013 15:07:40 +0000
Subject: [pypy-issue] [issue1649] np.load bug on pypy 2.21
In-Reply-To: <1386083260.7.0.0343797506723.issue1649@bugs.pypy.org>
Message-ID: <1386083260.7.0.0343797506723.issue1649@bugs.pypy.org>


New submission from handloomweaver <a.w.macmillan at gmail.com>:

>> np.load('blah.npy')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File
"/Users/handloomweaver/.virtualenvs/pypy221/site-packages/numpy/lib/npyio.py",
line 391, in load
    return format.read_array(fid)
  File
"/Users/handloomweaver/virtualenvs/pypy221/site-packages/numpy/lib/format.py",
line 451, in read_array
    if dtype.hasobject:
AttributeError: 'dtype' object has no attribute 'hasobject'

----------
messages: 6391
nosy: handloomweaver, pypy-issue
priority: bug
status: unread
title: np.load bug on pypy 2.21

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1649>
________________________________________

From tracker at bugs.pypy.org  Tue Dec  3 18:38:25 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Tue, 03 Dec 2013 17:38:25 +0000
Subject: [pypy-issue] [issue1649] np.load bug on pypy 2.21
In-Reply-To: <1386083260.7.0.0343797506723.issue1649@bugs.pypy.org>
Message-ID: <CACw2m-aZbS+9WJ2ru-gtkJu71U+aBZpgWtwAT9YnkqSrC1nEEg@mail.gmail.com>


Brian Kearns <bdkearns at gmail.com> added the comment:

this has been fixed since 2.2.1, try the nightly.

On Tue, Dec 3, 2013 at 7:07 AM, handloomweaver <tracker at bugs.pypy.org>wrote:

>
> New submission from handloomweaver <a.w.macmillan at gmail.com>:
>
> >> np.load('blah.npy')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File
>
> "/Users/handloomweaver/.virtualenvs/pypy221/site-packages/numpy/lib/npyio.py",
> line 391, in load
>     return format.read_array(fid)
>   File
>
> "/Users/handloomweaver/virtualenvs/pypy221/site-packages/numpy/lib/format.py",
> line 451, in read_array
>     if dtype.hasobject:
> AttributeError: 'dtype' object has no attribute 'hasobject'
>
> ----------
> messages: 6391
> nosy: handloomweaver, pypy-issue
> priority: bug
> status: unread
> title: np.load bug on pypy 2.21
>
> ________________________________________
> PyPy bug tracker <tracker at bugs.pypy.org>
> <https://bugs.pypy.org/issue1649>
> ________________________________________
> _______________________________________________
> pypy-issue mailing list
> pypy-issue at python.org
> https://mail.python.org/mailman/listinfo/pypy-issue
>

----------
nosy: +bdk
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1649>
________________________________________

From tracker at bugs.pypy.org  Wed Dec  4 11:41:49 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Wed, 04 Dec 2013 10:41:49 +0000
Subject: [pypy-issue] [issue1347] Perf problem with many regular expressions
In-Reply-To: <1355435752.78.0.072023264159.issue1347@bugs.pypy.org>
Message-ID: <1386153709.76.0.352413197195.issue1347@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

After looking at it, it seems that the problem comes from having a single place
in Python source code (the line calling regexp.search()) that can actually
invoke a large number of different regexps.  So far, this is done with a chain
of 'guard_value' that linearly search which of the jit-compiled regexps it is. 
My guess is that it doesn't occur often in real code, but I agree that it's
nevertheless an issue that needs fixing.  This is a special case of
https://bugs.pypy.org/issue1565.

----------
nosy: +arigo

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1347>
________________________________________

From tracker at bugs.pypy.org  Thu Dec  5 08:29:50 2013
From: tracker at bugs.pypy.org (Bryan)
Date: Thu, 05 Dec 2013 07:29:50 +0000
Subject: [pypy-issue] [issue1650] numpy: summing over an empty,
	mutli-dimensional array produces incorrect, nondeterministic results
In-Reply-To: <1386228590.09.0.883459829085.issue1650@bugs.pypy.org>
Message-ID: <1386228590.09.0.883459829085.issue1650@bugs.pypy.org>


New submission from Bryan <bryan.head+pypy at gmail.com>:

To reproduce:

1) Start pypy (results are usually more nondeterministic with ipython rather than straight 
pypy, but it doesn't matter which)
2) Run the following:
>>>> import numpy as np
>>>> np.array([]).reshape(0,2).sum(0)

Here are some of the results I've gotten:
array([  2.68156159e+154,  -2.31584192e+077])
array([ 0.,  0.]) # which is correct
array([  1.72723371e-77,   1.72723371e-77])
array([  1.28822975e-231,   1.28822975e-231])

and so forth.

It looks like some kind of overflow problem. With the regular pypy repl, you usually get long 
sequences of the same answer, though restarting the repl consistently produces different 
answers.

Note that Python + main numpy produces the correct result of array([ 0., 0.]) consistently.

I apologize if this is the wrong place for this bug report. Let me know if there's somewhere 
more appropriate to submit it.

I'm using the pypy fork of numpy 1.8.0.dev-6b8df83 and pypy 2.2.1 on OS X 10.9. Let me know if 
you'd like any other information.

----------
messages: 6394
nosy: pypy-issue, qiemem
priority: bug
status: unread
title: numpy: summing over an empty, mutli-dimensional array produces incorrect, nondeterministic results

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1650>
________________________________________

From tracker at bugs.pypy.org  Thu Dec  5 10:22:13 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Thu, 05 Dec 2013 09:22:13 +0000
Subject: [pypy-issue] [issue1651] Windows: pypyw.exe?
In-Reply-To: <1386235333.33.0.571219454461.issue1651@bugs.pypy.org>
Message-ID: <1386235333.33.0.571219454461.issue1651@bugs.pypy.org>


New submission from Armin Rigo <armin.rigo at gmail.com>:

On Windows, we need to compile *two* wrapper executables around the
"libpypy.dll": pypy.exe and pypyw.exe, similar to python.exe and pythonw.exe. 
The latter doesn't force the console environment.

----------
messages: 6395
nosy: arigo, pypy-issue
priority: feature
status: unread
title: Windows: pypyw.exe?

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1651>
________________________________________

From tracker at bugs.pypy.org  Fri Dec  6 01:35:22 2013
From: tracker at bugs.pypy.org (Spider)
Date: Fri, 06 Dec 2013 00:35:22 +0000
Subject: [pypy-issue] [issue1652] Array search ( Sorting/index heavy) is
	2.24x slower on	PyPy than Python2 and Python3
In-Reply-To: <1386279511.75.0.122444070392.issue1652@bugs.pypy.org>
Message-ID: <1386290122.77.0.315809898375.issue1652@bugs.pypy.org>


Spider <spider at skuggor.se> added the comment:

Some further tuning of the code ( making it less stupid)  only increases the
pain for PyPy :

python3: 
56.57909187000041

pypy:
332.630371809

( 1000 iterations now. )
    15845    0.193    0.000    0.193    0.000 :0(append)
      713    0.021    0.000    0.021    0.000 :0(index)
     2085    0.031    0.000    0.031    0.000 :0(max)
     2314    0.064    0.000    0.064    0.000 :0(sort)

----------
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1652>
________________________________________

From tracker at bugs.pypy.org  Fri Dec  6 02:13:10 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Fri, 06 Dec 2013 01:13:10 +0000
Subject: [pypy-issue] [issue1653] performance regressions in benchmarks
In-Reply-To: <1386292390.88.0.0820526190869.issue1653@bugs.pypy.org>
Message-ID: <1386292390.88.0.0820526190869.issue1653@bugs.pypy.org>


New submission from Brian Kearns <bdkearns at gmail.com>:

Some of the benchmarks have had regressions that don't seem to have been 
investigated.

Go seems the most troubling, both the jump in beginning of July and recent 
noise. Raytrace also starts exhibiting noise around the same time.

http://speed.pypy.org/timeline/?
exe=1%2C5&base=2%2B472&ben=go&env=1&revs=200&equid=off

----------
messages: 6398
nosy: bdk, pypy-issue
priority: performance bug
status: unread
title: performance regressions in benchmarks

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1653>
________________________________________

From tracker at bugs.pypy.org  Fri Dec  6 03:08:28 2013
From: tracker at bugs.pypy.org (Bryan)
Date: Fri, 06 Dec 2013 02:08:28 +0000
Subject: [pypy-issue] [issue1650] numpy: summing over an empty,
	mutli-dimensional array produces incorrect, nondeterministic results
In-Reply-To: <1386228590.09.0.883459829085.issue1650@bugs.pypy.org>
Message-ID: <1386295708.69.0.0278311389929.issue1650@bugs.pypy.org>


Bryan <bryan.head+pypy at gmail.com> added the comment:

It looks like the return array simply isn't being zeroed.

----------
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1650>
________________________________________

From tracker at bugs.pypy.org  Fri Dec  6 03:12:06 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Fri, 06 Dec 2013 02:12:06 +0000
Subject: [pypy-issue] [issue1650] numpy: summing over an empty,
	mutli-dimensional array produces incorrect, nondeterministic results
In-Reply-To: <1386228590.09.0.883459829085.issue1650@bugs.pypy.org>
Message-ID: <1386295926.98.0.96641453501.issue1650@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

Thanks for the report. Fixed in 3fca57d5b11e.

----------
nosy: +bdk
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1650>
________________________________________

From tracker at bugs.pypy.org  Fri Dec  6 09:37:42 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Fri, 06 Dec 2013 08:37:42 +0000
Subject: [pypy-issue] [issue1653] performance regressions in benchmarks
In-Reply-To: <1386292390.88.0.0820526190869.issue1653@bugs.pypy.org>
Message-ID: <1386319062.15.0.1640966374.issue1653@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

If I remember correctly, the recent noise was introduced by the jit-counter
branch, "expectedly" because it is now more prone to random collisions that can
have particularly an effect on some kinds of small examples (e.g. by getting
first one version or another compiled as a loop, then the rest attached as a
less-optimized bridge).  I think that the level of random fluctuations is still
ok overall.

I haven't investigated the jump of Go in the beginning of July.

----------
nosy: +arigo
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1653>
________________________________________

From tracker at bugs.pypy.org  Sat Dec  7 00:25:46 2013
From: tracker at bugs.pypy.org (anon)
Date: Fri, 06 Dec 2013 23:25:46 +0000
Subject: [pypy-issue] [issue1654] math.factorial is slow
In-Reply-To: <1386372346.45.0.97461475817.issue1654@bugs.pypy.org>
Message-ID: <1386372346.45.0.97461475817.issue1654@bugs.pypy.org>


New submission from anon <b10419697 at klzlk.com>:

math.factorial uses a very simple algorithm multiplying by each value in
range(1,x+1) in turn. A better method would be to do binary splitting. I am
unfamiliar with PyPy internals, however I believe app_math.py implements
factorial in pure Python.

Provided below is a patch that uses binary splitting. It seems much faster for
large factorials, but maybe uses slightly more memory and seems 10% slower for
very small factorials.

def factorial(x):
    """Find x!."""
    if isinstance(x, float):
        fl = int(x)
        if fl != x:
            raise ValueError("float arguments must be integral")
        x = fl
    
    #Experimentally this gap seems good
    gap = max(100, x>>7)
    def _fac(low, high):
        if low+gap >= high:
            t = 1
            for i in range(low, high):
                t *= i
            return t
        
        mid = (low + high) >> 1
        return _fac(low, mid) * _fac(mid, high)
    
    return _fac(1, x+1)

If someone can improve it further, that would be great. The code is public domain.

----------
messages: 6402
nosy: anon, pypy-issue
priority: performance bug
status: unread
title: math.factorial is slow

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1654>
________________________________________

From tracker at bugs.pypy.org  Sat Dec  7 00:30:10 2013
From: tracker at bugs.pypy.org (anon)
Date: Fri, 06 Dec 2013 23:30:10 +0000
Subject: [pypy-issue] [issue1654] math.factorial is slow
In-Reply-To: <1386372610.4.0.971079626124.issue1654@bugs.pypy.org>
Message-ID: <1386372610.4.0.971079626124.issue1654@bugs.pypy.org>


New submission from anon <b10419697 at klzlk.com>:

math.factorial uses a very simple algorithm multiplying by each value in
range(1,x+1) in turn. A better method would be to do binary splitting. I am
unfamiliar with PyPy internals, however I believe app_math.py implements
factorial in pure Python.

Provided below is a patch that uses binary splitting. It seems much faster for
large factorials, but maybe uses slightly more memory and seems 10% slower for
very small factorials.

def factorial(x):
    """Find x!."""
    if isinstance(x, float):
        fl = int(x)
        if fl != x:
            raise ValueError("float arguments must be integral")
        x = fl
    if x < 0:
        raise ValueError("x must be >= 0")
    
    #Experimentally this gap seems good
    gap = max(100, x>>7)
    def _fac(low, high):
        if low+gap >= high:
            t = 1
            for i in range(low, high):
                t *= i
            return t
        
        mid = (low + high) >> 1
        return _fac(low, mid) * _fac(mid, high)
    
    return _fac(1, x+1)

If someone can improve it further, that would be great. The code is public domain.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1654>
________________________________________

From tracker at bugs.pypy.org  Sat Dec  7 00:31:39 2013
From: tracker at bugs.pypy.org (anon)
Date: Fri, 06 Dec 2013 23:31:39 +0000
Subject: [pypy-issue] [issue1654] math.factorial is slow
In-Reply-To: <1386372699.48.0.63527104701.issue1654@bugs.pypy.org>
Message-ID: <1386372699.48.0.63527104701.issue1654@bugs.pypy.org>


New submission from anon <b10419697 at klzlk.com>:

math.factorial uses a very simple algorithm multiplying by each value in
range(1,x+1) in turn. A better method would be to do binary splitting. I am
unfamiliar with PyPy internals, however I believe app_math.py implements
factorial in pure Python.

Provided below is a patch that uses binary splitting. It seems much faster for
large factorials, but maybe uses slightly more memory and seems 10% slower for
very small factorials.

def factorial(x):
    """Find x!."""
    if isinstance(x, float):
        fl = int(x)
        if fl != x:
            raise ValueError("float arguments must be integral")
        x = fl
    if x < 0:
        raise ValueError("x must be >= 0")
    
    #Experimentally this gap seems good
    gap = max(100, x>>7)
    def _fac(low, high):
        if low+gap >= high:
            t = 1
            for i in range(low, high):
                t *= i
            return t
        
        mid = (low + high) >> 1
        return _fac(low, mid) * _fac(mid, high)
    
    return _fac(1, x+1)

If someone can improve it further, that would be great. My portion of the code
is public domain.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1654>
________________________________________

From tracker at bugs.pypy.org  Sat Dec  7 00:37:18 2013
From: tracker at bugs.pypy.org (anon)
Date: Fri, 06 Dec 2013 23:37:18 +0000
Subject: [pypy-issue] [issue1654] math.factorial is slow
In-Reply-To: <1386372699.48.0.63527104701.issue1654@bugs.pypy.org>
Message-ID: <1386373038.3.0.577714627421.issue1654@bugs.pypy.org>


anon <b10419697 at klzlk.com> added the comment:

Note the deleted posts contained mistakes.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1654>
________________________________________

From tracker at bugs.pypy.org  Sat Dec  7 10:23:35 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Sat, 07 Dec 2013 09:23:35 +0000
Subject: [pypy-issue] [issue1654] math.factorial is slow
In-Reply-To: <1386372699.48.0.63527104701.issue1654@bugs.pypy.org>
Message-ID: <1386408215.57.0.352872704648.issue1654@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

Yay, thanks! :-)  It's already much, much faster than the version we had (which
is also the version in CPython apparently).  I added a check to use directly the
old version of the code for x <= 100.  Checked in 73fd0ad3e2e2.

----------
nosy: +arigo
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1654>
________________________________________

From tracker at bugs.pypy.org  Sun Dec  8 09:11:38 2013
From: tracker at bugs.pypy.org (Roman Valov)
Date: Sun, 08 Dec 2013 08:11:38 +0000
Subject: [pypy-issue] [issue1655] ctypes errcheck protocol different from
	cpython	implementation
In-Reply-To: <1386490298.95.0.113241491767.issue1655@bugs.pypy.org>
Message-ID: <1386490298.95.0.113241491767.issue1655@bugs.pypy.org>


New submission from Roman Valov <roman.valov at gmail.com>:

Hi, folks.

Currently I'm working on bindings for C-based library using ctypes and once 
bindings were complete I've tried to test them using PyPy. The library I'm 
working on have a lots of function definitions that return pointer to the list 
with number of items in the list returned via passed-by-ref argument.

Here is example of C source:

#include <stdio.h>

int data[] = { -1, -2, -3, -4 };

int *get_data(char const *tag, int *len)
{
	printf("tag: %s\n", tag);

	*len = sizeof(data) / sizeof(data[0]);
	return data;
}

Once saved as sample.c, it should be compiled to shared library:

gcc sample.c -o libsample.so -fPIC -shared


Next, I'd wrapped such functions using both paramflags and errcheck 
functionality of ctypes function prototypes. Here is it:

#!/usr/bin/python

from ctypes import cdll, CFUNCTYPE, POINTER, c_int, c_char_p

lib = cdll.LoadLibrary('./libsample.so')

def ret_list_p(icount):
    def sz_array_p(obj, func, args):
        print (repr(obj))
        print (repr(args))
        return [ obj[i] for i in range(args[icount].value) ]
    return sz_array_p

get_data_prototype = CFUNCTYPE(POINTER(c_int), c_char_p, POINTER(c_int))
get_data_paramflag = ((1,), (2,))
get_data_signature = ('get_data', lib)

get_data = get_data_prototype( get_data_signature, get_data_paramflag )

print get_data('noerrcheck')

get_data.errcheck = ret_list_p(1)

print get_data('doerrcheck')



Once this code is invoked with CPython implementation I've got following result:

tag: noerrcheck
4
tag: doerrcheck
<__main__.LP_c_int object at 0x7ffb124775f0>
('doerrcheck', c_int(4))
[-1, -2, -3, -4]

but with PyPy I'm faced with errors due to errcheck got python wrapper args, but 
not args binded function was called with:

tag: noerrcheck
4
tag: doerrcheck
<_ctypes.pointer.LP_c_int object at 0x00007f27b7d62250>
('doerrcheck',)
Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "./sample.py", line 24, in <module>
    print get_data('doerrcheck')
  File "/usr/lib/pypy/lib_pypy/_ctypes/function.py", line 343, in __call__
    result = self._do_errcheck(result, args)
  File "/usr/lib/pypy/lib_pypy/_ctypes/function.py", line 387, in _do_errcheck
    v = self._errcheck_(result, self, args)
  File "./sample.py", line 11, in sz_array_p
    return [ obj[i] for i in range(args[icount].value) ]
IndexError: tuple index out of range

----------
files: sample.py
messages: 6407
nosy: pypy-issue, reddot
priority: bug
status: unread
title: ctypes errcheck protocol different from cpython implementation

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1655>
________________________________________

From tracker at bugs.pypy.org  Sun Dec  8 10:02:32 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Sun, 08 Dec 2013 09:02:32 +0000
Subject: [pypy-issue] [issue1655] ctypes errcheck protocol different from
	cpython	implementation
In-Reply-To: <1386490298.95.0.113241491767.issue1655@bugs.pypy.org>
Message-ID: <1386493352.39.0.211277555756.issue1655@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

Hi!  Thanks for the bug report.  It's unclear if we'll ever fix this, as it may
be relying on "obscure details" of ctypes, but we'll try to.  It involves
reading carefully the docs for what you are or are not allowed to do in an
errcheck, and what the "paramflags" is at all.

I suppose I should mention CFFI here as a clearer way to write this down: it
doesn't have the same obscure bits of functionality, but encourages you to write
wrapper functions in pure Python.  These do explicitly what you want, rather
than relying on callbacks invoked for you.

----------
nosy: +arigo
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1655>
________________________________________

From tracker at bugs.pypy.org  Sun Dec  8 10:47:08 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Sun, 08 Dec 2013 09:47:08 +0000
Subject: [pypy-issue] [issue1655] ctypes errcheck protocol different from
	cpython	implementation
In-Reply-To: <1386490298.95.0.113241491767.issue1655@bugs.pypy.org>
Message-ID: <1386496028.75.0.531173871781.issue1655@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

Fixed in 160192d5cbdc.

----------
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1655>
________________________________________

From tracker at bugs.pypy.org  Sun Dec  8 11:45:07 2013
From: tracker at bugs.pypy.org (Roman Valov)
Date: Sun, 08 Dec 2013 10:45:07 +0000
Subject: [pypy-issue] [issue1655] ctypes errcheck protocol different from
	cpython	implementation
In-Reply-To: <1386490298.95.0.113241491767.issue1655@bugs.pypy.org>
Message-ID: <1386499507.2.0.394369485582.issue1655@bugs.pypy.org>


Roman Valov <roman.valov at gmail.com> added the comment:

Respect for the quick fix! Tests on real code show no issues.

----------
status: resolved -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1655>
________________________________________

From tracker at bugs.pypy.org  Mon Dec  9 22:07:45 2013
From: tracker at bugs.pypy.org (samlh)
Date: Mon, 09 Dec 2013 21:07:45 +0000
Subject: [pypy-issue] [issue1653] performance regressions in benchmarks
In-Reply-To: <1386292390.88.0.0820526190869.issue1653@bugs.pypy.org>
Message-ID: <1386623265.09.0.374871041418.issue1653@bugs.pypy.org>


samlh <samuel.l.harrington at gmail.com> added the comment:

As I recall, the jump in July was
https://bitbucket.org/pypy/pypy/commits/8194ed86bd544fe86230bf0ecaa43c33c48e7baa
and https://bitbucket.org/pypy/pypy/commits/aa29cd273b2da3d6ebb33c29a9442d62cf653bfb

----------
nosy: +samlh

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1653>
________________________________________

From tracker at bugs.pypy.org  Thu Dec 12 12:09:48 2013
From: tracker at bugs.pypy.org (lesshaste)
Date: Thu, 12 Dec 2013 11:09:48 +0000
Subject: [pypy-issue] [issue1657] numpy failing with AttributeError:
	'ndarray' object has	no attribute 'flags'
In-Reply-To: <1386846588.25.0.265408681978.issue1657@bugs.pypy.org>
Message-ID: <1386846588.25.0.265408681978.issue1657@bugs.pypy.org>


New submission from lesshaste <drraph at gmail.com>:

Some simple code fails with the error 

Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "counterfeit.py", line 9, in <module>
    F = np.matrix(list(itertools.product([0,1],repeat = n))).transpose()
  File "/usr/local/lib/pypy2.7/dist-packages/numpy/matrixlib/defmatrix.py", line
279, in __new__
    if (ndim == 2) and arr.flags.fortran:
AttributeError: 'ndarray' object has no attribute 'flags'

----------
files: counterfeit.py
messages: 6414
nosy: lesshaste, pypy-issue
priority: bug
status: unread
title: numpy failing with AttributeError: 'ndarray' object has no attribute 'flags'

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1657>
________________________________________

From tracker at bugs.pypy.org  Thu Dec 12 12:19:19 2013
From: tracker at bugs.pypy.org (lesshaste)
Date: Thu, 12 Dec 2013 11:19:19 +0000
Subject: [pypy-issue] [issue1657] numpy failing with AttributeError:
	'ndarray' object has	no attribute 'flags'
In-Reply-To: <1386846588.25.0.265408681978.issue1657@bugs.pypy.org>
Message-ID: <1386847159.9.0.450602850795.issue1657@bugs.pypy.org>


lesshaste <drraph at gmail.com> added the comment:

This is using git clone https://bitbucket.org/pypy/numpy.git from December 12, 2013.

----------
release:  -> 2.2
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1657>
________________________________________

From tracker at bugs.pypy.org  Thu Dec 12 18:54:48 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Thu, 12 Dec 2013 17:54:48 +0000
Subject: [pypy-issue] [issue1657] numpy failing with AttributeError:
	'ndarray' object has	no attribute 'flags'
In-Reply-To: <1386846588.25.0.265408681978.issue1657@bugs.pypy.org>
Message-ID: <1386870888.05.0.0958291824606.issue1657@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

try the nightly, minimal version of array.flags exists there

----------
nosy: +bdk

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1657>
________________________________________

From tracker at bugs.pypy.org  Fri Dec 13 00:15:22 2013
From: tracker at bugs.pypy.org (mario)
Date: Thu, 12 Dec 2013 23:15:22 +0000
Subject: [pypy-issue] [issue892] Pypy is slower on longs than it should
In-Reply-To: <1317663715.18.0.663914446646.issue892@bugs.pypy.org>
Message-ID: <1386890122.93.0.150184514982.issue892@bugs.pypy.org>


mario <mario.pernici at gmail.com> added the comment:

In the attached file rbigint_xmul.py there is a modified _x_mul for 
rbigint.py in pypy-2.2.1,
ported from a patch I submitted in http://bugs.python.org/issue3944

On my computer (x86_64) multiplication seems to be at least as fast as 
CPython for numbers with few words,  about 2x faster for numbers 
with about 15 words, about 1.8x faster for numbers with 150 words.

----------
nosy: +pernici -pjenvey

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue892>
________________________________________

From tracker at bugs.pypy.org  Fri Dec 13 06:20:39 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Fri, 13 Dec 2013 05:20:39 +0000
Subject: [pypy-issue] [issue1657] numpy failing with AttributeError:
	'ndarray' object has	no attribute 'flags'
In-Reply-To: <1386846588.25.0.265408681978.issue1657@bugs.pypy.org>
Message-ID: <1386912039.82.0.760927110593.issue1657@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

even after array.flags, other problems existed. just fixed these on trunk, will be 
in next nightly.

----------
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1657>
________________________________________

From tracker at bugs.pypy.org  Sat Dec 14 00:52:17 2013
From: tracker at bugs.pypy.org (Philip Jenvey)
Date: Fri, 13 Dec 2013 23:52:17 +0000
Subject: [pypy-issue] [issue1658] Add an accompanying env variable for --jit
In-Reply-To: <1386978737.74.0.901291547677.issue1658@bugs.pypy.org>
Message-ID: <1386978737.74.0.901291547677.issue1658@bugs.pypy.org>


New submission from Philip Jenvey <pjenvey at underboss.org>:

The pypy-c --jit cmd line option could probably use an accompanying env variable, 
e.g. PYPY_JIT=off

----------
messages: 6419
nosy: pypy-issue
priority: feature
status: unread
title: Add an accompanying env variable for  --jit

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1658>
________________________________________

From tracker at bugs.pypy.org  Sat Dec 14 12:08:28 2013
From: tracker at bugs.pypy.org (Anders Kaseorg)
Date: Sat, 14 Dec 2013 11:08:28 +0000
Subject: [pypy-issue] [issue1659] numpy: 0-d arrays should be indexable with
	np.newaxis
In-Reply-To: <1387019308.91.0.554365090856.issue1659@bugs.pypy.org>
Message-ID: <1387019308.91.0.554365090856.issue1659@bugs.pypy.org>


New submission from Anders Kaseorg <andersk at mit.edu>:

In CPython, np.array(1)[np.newaxis] returns array([1]).  In PyPy, it raises 
?IndexError: 0-d arrays can't be indexed?.

This also shows up in code like this:
a = np.zeros(10)
b = np.array([3,4,5])
a[b] = 1
# CPython: a == array([0, 0, 0, 1, 1, 1, 0, 0, 0, 0])
# PyPy: IndexError: 0-d arrays can't be indexed

(Is it useful to report bugs in PyPy?s numpy implementation at this stage?)

Version info:
Python 2.7.3 (0edc9b68247e, Dec 13 2013, 23:00:16)
[PyPy 2.3.0-alpha0 with GCC 4.6.3] on linux2

----------
messages: 6420
nosy: andersk, pypy-issue
priority: bug
status: unread
title: numpy: 0-d arrays should be indexable with np.newaxis

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1659>
________________________________________

From tracker at bugs.pypy.org  Sat Dec 14 18:55:33 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Sat, 14 Dec 2013 17:55:33 +0000
Subject: [pypy-issue] [issue1659] numpy: 0-d arrays should be indexable with
	np.newaxis
In-Reply-To: <1387019308.91.0.554365090856.issue1659@bugs.pypy.org>
Message-ID: <1387043733.19.0.677256089642.issue1659@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

yes, thanks for the report, fixed in a8a77877670d and 55f7a6da11c7, will be in the 
next nightly.

----------
nosy: +bdk
status: unread -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1659>
________________________________________

From tracker at bugs.pypy.org  Sun Dec 15 05:29:28 2013
From: tracker at bugs.pypy.org (Anders Kaseorg)
Date: Sun, 15 Dec 2013 04:29:28 +0000
Subject: [pypy-issue] [issue1660] numpy: triu_indices,
	tril_indices fails with AttributeError: 'ufunc' object has
	no	attribute 'outer'
In-Reply-To: <1387081768.76.0.606972101704.issue1660@bugs.pypy.org>
Message-ID: <1387081768.76.0.606972101704.issue1660@bugs.pypy.org>


New submission from Anders Kaseorg <andersk at mit.edu>:

Although http://buildbot.pypy.org/numpy-status/latest.html claims that triu_indices and tril_indices should work, 
they don?t:

Python 2.7.3 (55f7a6da11c7, Dec 14 2013, 23:00:23)
[PyPy 2.3.0-alpha0 with GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``it is the expected behavior,
except when you don't expect it''
>>>> import numpy as np
>>>> np.triu_indices(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/anders/pypy-nightly-env/site-packages/numpy/lib/twodim_base.py", line 900, in triu_indices
    return mask_indices(n, triu, k)
  File "/home/anders/pypy-nightly-env/site-packages/numpy/lib/twodim_base.py", line 723, in mask_indices
    a = mask_func(m, k)
  File "/home/anders/pypy-nightly-env/site-packages/numpy/lib/twodim_base.py", line 450, in triu
    out = multiply((1 - tri(m.shape[0], m.shape[1], k - 1, dtype=m.dtype)), m)
  File "/home/anders/pypy-nightly-env/site-packages/numpy/lib/twodim_base.py", line 388, in tri
    m = greater_equal(subtract.outer(arange(N), arange(M)), -k)
AttributeError: 'ufunc' object has no attribute 'outer'

----------
messages: 6422
nosy: andersk, pypy-issue
priority: bug
status: unread
title: numpy: triu_indices, tril_indices fails with AttributeError: 'ufunc' object has no attribute 'outer'

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1660>
________________________________________

From tracker at bugs.pypy.org  Sun Dec 15 05:31:55 2013
From: tracker at bugs.pypy.org (Anders Kaseorg)
Date: Sun, 15 Dec 2013 04:31:55 +0000
Subject: [pypy-issue] [issue1661] numpy: rec.fromarrays fails with
	TypeError: data type not	understood
In-Reply-To: <1387081915.47.0.292368350449.issue1661@bugs.pypy.org>
Message-ID: <1387081915.47.0.292368350449.issue1661@bugs.pypy.org>


New submission from Anders Kaseorg <andersk at mit.edu>:

Python 2.7.3 (55f7a6da11c7, Dec 14 2013, 23:00:23)
[PyPy 2.3.0-alpha0 with GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``fijal: can I initiate you on the
(easy but) non-trivial topic of writing C function types? :-)''
>>>> import numpy as np
>>>> np.rec.fromarrays((np.array([10, 20, 30]), np.array([40, 50, 60])), dtype=[('a', int), ('b', int)])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/anders/pypy-nightly-env/site-packages/numpy/core/records.py", line 564, in fromarrays
    _array = recarray(shape, descr)
  File "/home/anders/pypy-nightly-env/site-packages/numpy/core/records.py", line 402, in __new__
    self = ndarray.__new__(subtype, shape, (record, descr), order=order)
TypeError: data type not understood (value of type type not expected here)

----------
messages: 6423
nosy: andersk, pypy-issue
priority: bug
status: unread
title: numpy: rec.fromarrays fails with TypeError: data type not understood

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1661>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 17 19:30:23 2013
From: tracker at bugs.pypy.org (Kevin Duffy)
Date: Tue, 17 Dec 2013 18:30:23 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>


New submission from Kevin Duffy <kmdext at apple.com>:

There are a few places in datetime.py where the 'is' operator is used but it should really be 
'=='. This can cause comparisons to fail even when two tzinfo's are equivalent.

Simple patch attached.

----------
files: pypy-datetime.patch
messages: 6424
nosy: kmdext, pypy-issue
priority: bug
release: 2.2
status: unread
title: 'is' should be '==' in datetime.py

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 17 19:47:14 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Tue, 17 Dec 2013 18:47:14 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387306034.51.0.0254754711631.issue1662@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

can you provide failing tests to accompany this? 
(pypy/module/test_lib_pypy/test_datetime.py)

----------
nosy: +bdk
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 17 22:21:54 2013
From: tracker at bugs.pypy.org (Kevin Duffy)
Date: Tue, 17 Dec 2013 21:21:54 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387315313.98.0.511428394647.issue1662@bugs.pypy.org>


Kevin Duffy <kmdext at apple.com> added the comment:

Whoops, forgot my test patch. Here is the run without my datetime.py patch:

========================================================================================================================================================================== short test summary info 
==========================================================================================================================================================================
FAIL test_datetime.py::test_astimezone_eq
FAIL test_datetime.py::test_time_cmp_eq
FAIL test_datetime.py::test_datetime_sub_eq
==================================================================================================================================================================== 3 failed, 12 passed in 0.13 seconds 
====================================================================================================================================================================


And with the patch:

../../../pytest.py  test_datetime.py 
============================================================================================================================================================================ test session starts 
============================================================================================================================================================================
platform darwin -- Python 2.7.6 -- pytest-2.2.4.dev2
pytest-2.2.4.dev2 from /Users/kduffy/code/pypy/pytest.pyc
collected 15 items 

test_datetime.py ...............

========================================================================================================================================================================= 15 passed in 0.10 seconds 
=========================================================================================================================================================================

Attached a cumulative patch

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 17 22:43:17 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Tue, 17 Dec 2013 21:43:17 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387316597.76.0.227834384242.issue1662@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

These tests should also pass on CPython 2.7 (these do not). One can run the tests 
against CPython easily by switching the "from lib_pypy import datetime" to "import 
datetime". As is, without your patch, your tests fail on both CPython 2.7 and 
PyPy.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 00:45:02 2013
From: tracker at bugs.pypy.org (Ian Ozsvald)
Date: Tue, 17 Dec 2013 23:45:02 +0000
Subject: [pypy-issue] [issue1663] np.sum on arrays of bools doesn't count
	past 127 correctly
In-Reply-To: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>
Message-ID: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>


New submission from Ian Ozsvald <ian at ianozsvald.com>:

np.sum(np.array([<booleans>]) with over 127 True values results in an incorrect
count. Using Python's sum instead produces the correct answer. np.sum gives the
correct answer for smaller counts of True values. Two examples are below - the
tipping point seems to be a list of True values that sum to 128 and multiples of
128.

Version: PyPy Nightly for today:
pypy-c-jit-68443-d5e489e07679-linux64
and numpy branch checked-out today.

Python 2.7.3 (d5e489e07679, Dec 16 2013, 23:00:17)
[PyPy 2.3.0-alpha0 with GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``"messy" is not a judgement, but
just a fact of complicatedness''
>>>> import numpy as np
>>>> np.version
<module 'numpy.version' from
'/home/ian/Downloads/pypy-c-jit-68443-d5e489e07679-linux64/env_numpy/site-packages/numpy/version.pyc'>
>>>> np.sum(np.array([True,False]))
1
>>>> np.sum(np.array([True,False]*1000))  # Incorrect behaviour
-24
>>>> sum(np.array([True,False]))
1
>>>> sum(np.array([True,False]*1000))
1000

It feels as though the counter is a single byte (as suggested below using
array.itemsize), it counts up to 127 and then counts down back to 0 (when it
should be counting to 256 and beyond), then cycles again:

>>>> np.sum(np.array([True,False]))
1
>>>> np.sum(np.array([True,False]*127))
127
>>>> np.sum(np.array([True,False]*128))
-128
>>>> np.sum(np.array([True,False]*129))
-127
>>>> np.sum(np.array([True,False]*255))
-1
>>>> np.sum(np.array([True,False]*256))
0
>>>> np.sum(np.array([True,False]*257))
1
>>>> np.sum(np.array([True,False]*384))
-128
>>>> np.sum(np.array([True,False]*385))
-127

>>>> np.array([True,False])
array([ True, False], dtype=bool)
>>>> arr=np.array([True,False])
>>>> arr.itemsize
1

----------
nosy: +ianozsvald
status: unread -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1663>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 01:37:13 2013
From: tracker at bugs.pypy.org (Kevin Duffy)
Date: Wed, 18 Dec 2013 00:37:13 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387327033.08.0.477358485725.issue1662@bugs.pypy.org>


Kevin Duffy <kmdext at apple.com> added the comment:

Right, they should pass on both interpreters. When I consulted the CPython 2.7.6 implementation in 
Modules/datetimemodule.c, astimezone() does do a pointer comparison so I can remove my patch for astimezone(). If you look 
at time_richcompare() and datetime_subtract() in CPython it does not have the equivalent of the PyPy tzinfo comparison (if 
my.tzinfo is other.tzinfo) -  it always calls utcoffset(). I still believe changing the 'is' to a '==' is more correct in 
these cases. One reason is if you look at datetime.__cmp__ and time.__sub__ in PyPy, they use a '==' not an 'is'.

Since CPython doesn't do the exact equivalent code (tzinfo comparison), I'm not sure how to get the tests to fail on PyPy 
without the patch AND succeed on CPython without doing an interpreter check. The difference between the interpreters is one 
should call utcoffset and one shouldn't.

Attaching an updated patch that removes changes to astimezone and updated tests that at least pass on both interpreters.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 02:07:17 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Wed, 18 Dec 2013 01:07:17 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387328837.8.0.118538444102.issue1662@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

CPython _does_ have the equivalent of mytz is ottz, look at 
classify_two_utcoffsets (called from time_richcompare). Seems to do pointer 
compare on get_tzinfo_member(o1) and get_tzinfo_member(o2) before calling 
utcoffset.

In these cases, what is "more correct" is "what CPython does". If you have an 
issue with what CPython does, you should take it up with them. If you can 
demonstrate a test that passes on CPython and fails on PyPy, that would be a bug 
and a justification for modifying PyPy behavior.

The tests in the newest patch pass on both CPython and current PyPy, and so do 
not represent a bug.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 02:37:36 2013
From: tracker at bugs.pypy.org (Kevin Duffy)
Date: Wed, 18 Dec 2013 01:37:36 +0000
Subject: [pypy-issue] [issue1662] 'is' should be '==' in datetime.py
In-Reply-To: <1387305023.54.0.0521495933834.issue1662@bugs.pypy.org>
Message-ID: <1387330656.81.0.767728813912.issue1662@bugs.pypy.org>


Kevin Duffy <kmdext at apple.com> added the comment:

Looks like I didn't dig deep enough into that stack. I see the pointer comparison there 
now. Fair enough, I'll close out this bug.

----------
status: chatting -> invalid

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1662>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 03:27:35 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Wed, 18 Dec 2013 02:27:35 +0000
Subject: [pypy-issue] [issue918] implement numpy.matrix in micronumpy
In-Reply-To: <1319321974.08.0.127911146223.issue918@bugs.pypy.org>
Message-ID: <1387333655.44.0.844202264653.issue918@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

works in latest pypy nightly and numpy fork

----------
nosy: +bdk
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue918>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 03:28:45 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Wed, 18 Dec 2013 02:28:45 +0000
Subject: [pypy-issue] [issue1466] for slices,
	__array__interface['data'][0] yields adress of base object's data
In-Reply-To: <1366870624.4.0.139778689303.issue1466@bugs.pypy.org>
Message-ID: <1387333725.53.0.206257825436.issue1466@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

works in latest pypy/numpy

----------
nosy: +bdk
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1466>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 04:32:55 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Wed, 18 Dec 2013 03:32:55 +0000
Subject: [pypy-issue] [issue1581] numpy: strptime does not like string_
In-Reply-To: <1376157623.52.0.570103140202.issue1581@bugs.pypy.org>
Message-ID: <1387337575.56.0.945187468168.issue1581@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

fixed in 2ec4090a85d4 + e8735afef7f7, will be in the next nightly pypy

----------
nosy: +bdk
status: unread -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1581>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 10:14:43 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Wed, 18 Dec 2013 09:14:43 +0000
Subject: [pypy-issue] [issue1663] np.sum on arrays of bools doesn't count
	past 127 correctly
In-Reply-To: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>
Message-ID: <1387358083.73.0.9638943288.issue1663@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

Should check if there are similar issues with arrays of "short", or with arrays
of "int" with more than 2**31 items.

----------
nosy: +arigo

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1663>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 10:15:25 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Wed, 18 Dec 2013 09:15:25 +0000
Subject: [pypy-issue] [issue1663] np.sum on arrays of bools doesn't count
	past 127 correctly
In-Reply-To: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>
Message-ID: <1387358125.93.0.117444018072.issue1663@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

...and also, if numpy.sum() on an array of "float" computes the sum using a
"double" instead, which would give more precision.

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1663>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 13:56:30 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Wed, 18 Dec 2013 12:56:30 +0000
Subject: [pypy-issue] [issue1008] numpypy: commit the code for empty_like,
	zeros_like, ones_like
In-Reply-To: <1326903282.11.0.305182879123.issue1008@bugs.pypy.org>
Message-ID: <1387371390.83.0.200530109472.issue1008@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

added zeros_like, and the others exist/work in the now-used numpy fork. marking 
resolved.

----------
nosy: +bdk
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1008>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 19:07:03 2013
From: tracker at bugs.pypy.org (Ian Ozsvald)
Date: Wed, 18 Dec 2013 18:07:03 +0000
Subject: [pypy-issue] [issue1663] np.sum on arrays of bools doesn't count
	past 127 correctly
In-Reply-To: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>
Message-ID: <1387390023.82.0.430988506683.issue1663@bugs.pypy.org>


Ian Ozsvald <ian at ianozsvald.com> added the comment:

np.sum does sum correctly if I force a more appropriate dtype. The numpy doc notes:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.sum.html
"dtype : dtype, optional
    The type of the returned array and of the accumulator in which the elements
are summed. By default, the dtype of a is used. An exception is when a has an
integer type with less precision than the default platform integer. In that
case, the default platform integer is used instead."

If I force a longer dtype then it sums correctly:
>>>> np.sum(np.array([True]*128))
-128
>>>> np.sum(np.array([True]*128), dtype=np.int64)
128

>>>> np.sum(np.array([True]*65536))
0
>>>> np.sum(np.array([True]*65536), dtype=np.int16)  # not long enough
0
>>>> np.sum(np.array([True]*65536), dtype=np.int32)  # long enough
65536
>>>> np.sum(np.array([True]*65536), dtype=np.int64)  # long enough
65536

Maybe np.sum's behaviour should be to use np.int_ if the dtype of the array to
be summed is shorter than np.int_?

It looks as though int summation works correctly:
>>>> print "%e" % np.sum(np.array([int(1e12)] * int(1e6)))
1.000000e+18 
>>>> print "%e" % np.sum(np.array([int(1e12)] * int(1e7)))  # an int64 overflows
as expected (same result as CPython numpy)
-8.446744e+18
>>>> print "%e" % sys.maxint
9.223372e+18

Floats from numpy preserve their precision (this mirrors CPython numpy's behaviour):
>>>> type(np.sum(np.array([np.float16(0.1)])))
<type 'numpy.float16'>
>>>> type(np.sum(np.array([np.float32(0.1)])))
<type 'numpy.float32'>
>>>> type(np.sum(np.array([np.float64(0.1)])))
<type 'numpy.float64'>

Python Floats are summed using a float64 (this mirrors CPython numpy's behaviour):
>>>> type(np.sum(np.array([float(1)])))
<type 'numpy.float64'>
>>>> type(np.sum(np.array([float(1e70)] * int(1e6))))
<type 'numpy.float64'>

Versions:
PyPy pypy-c-jit-68443-d5e489e07679-linux64 with numpy 1.8.0.dev-6b8df83
CPython 2.7.3 with numpy 1.8.0

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1663>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 20:49:00 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Wed, 18 Dec 2013 19:49:00 +0000
Subject: [pypy-issue] [issue1663] np.sum on arrays of bools doesn't count
	past 127 correctly
In-Reply-To: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>
Message-ID: <1387396140.18.0.158558533189.issue1663@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

fixed in 912fe4190438, will be in the next nightly pypy.

----------
nosy: +bdk
status: chatting -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1663>
________________________________________

From tracker at bugs.pypy.org  Wed Dec 18 22:25:09 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Wed, 18 Dec 2013 21:25:09 +0000
Subject: [pypy-issue] [issue1663] np.sum on arrays of bools doesn't count
	past 127 correctly
In-Reply-To: <1387323902.25.0.478221714214.issue1663@bugs.pypy.org>
Message-ID: <1387401909.23.0.950053515634.issue1663@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

@ianozsvald: thanks for the link (and thanks bdk for the quick fixing).  Indeed,
only integer types smaller than np.int_ should get promoted to np.int_ --- not
to a larger integer, and not float->double.

----------
status: resolved -> chatting

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1663>
________________________________________

From tracker at bugs.pypy.org  Sun Dec 22 09:53:36 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Sun, 22 Dec 2013 08:53:36 +0000
Subject: [pypy-issue] [issue1086] cpyext: calling PyIter_Next() on a
	"dictionaryiterator"	object crashes when the dict was modified
In-Reply-To: <1331480896.67.0.851470392018.issue1086@bugs.pypy.org>
Message-ID: <1387702416.21.0.78917722858.issue1086@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

Tried to reproduce by compiling this C code with cpyext, but it works fine. 
Closing for now as probably fixed in the meantime:

    http://bpaste.net/show/160931/

----------
nosy: +arigo
status: unread -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1086>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 24 12:15:39 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Tue, 24 Dec 2013 11:15:39 +0000
Subject: [pypy-issue] [issue1640] Posix 2001 ftime() is deprecated
In-Reply-To: <1384712643.82.0.535601733935.issue1640@bugs.pypy.org>
Message-ID: <1387883739.95.0.694753483044.issue1640@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

If what CPython 2.7.x does leads clang on FreeBSD to produce a lot of warnings,
then it's an issue that needs to be fixed on CPython first.  You should report
it there (or figure out if someone already did).

We may accept changes in parallel with CPython, but only if there is really a
problem.  This would mean only when using FreeBSD and clang.  On Linux the
problem doesn't appear to exist, and so should not be fixed --- again, in PyPy;
if CPython changes its mind about it we'll follow.

----------
nosy: +arigo

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1640>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 24 12:19:57 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Tue, 24 Dec 2013 11:19:57 +0000
Subject: [pypy-issue] [issue1640] Posix 2001 ftime() is deprecated
In-Reply-To: <1384712643.82.0.535601733935.issue1640@bugs.pypy.org>
Message-ID: <1387883997.74.0.311467832133.issue1640@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

> So if PyPy is to model CPy 2, we might use gettimeofday() when avail, otherwise
> fallback to ftime().

It seems to be precisely what rpython/rtyper/module/ll_time.py is doing...  What
is the problem?

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1640>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 24 13:25:37 2013
From: tracker at bugs.pypy.org (mattip)
Date: Tue, 24 Dec 2013 12:25:37 +0000
Subject: [pypy-issue] [issue1664] numpypy bit shift is broken
In-Reply-To: <1387887937.28.0.436411675183.issue1664@bugs.pypy.org>
Message-ID: <1387887937.28.0.436411675183.issue1664@bugs.pypy.org>


New submission from mattip <matti.picus at gmail.com>:

Python 2.7.3 (4702ccfceeb9, Dec 23 2013, 21:59:39)
[PyPy 2.3.0-alpha0 with MSC v.1500 32 bit] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>> import _numpypy.multiarray as numpy
>>>> a=numpy.array(123,dtype='uint64')
>>>> a
array([123])
>>>> a >> 1
SEGFAULT

----------
messages: 6444
nosy: mattip, pypy-issue
priority: bug
release: ???
status: unread
title: numpypy bit shift is broken

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1664>
________________________________________

From tracker at bugs.pypy.org  Tue Dec 24 19:10:54 2013
From: tracker at bugs.pypy.org (Brian Kearns)
Date: Tue, 24 Dec 2013 18:10:54 +0000
Subject: [pypy-issue] [issue1664] numpypy bit shift is broken
In-Reply-To: <1387887937.28.0.436411675183.issue1664@bugs.pypy.org>
Message-ID: <1387908654.58.0.466412977113.issue1664@bugs.pypy.org>


Brian Kearns <bdkearns at gmail.com> added the comment:

fixed in fc6ac3a6a746

----------
nosy: +bdk
status: unread -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1664>
________________________________________

From tracker at bugs.pypy.org  Fri Dec 27 06:14:06 2013
From: tracker at bugs.pypy.org (danblack)
Date: Fri, 27 Dec 2013 05:14:06 +0000
Subject: [pypy-issue] [issue1665] Module re behavior different from CPython
	in Match.group
In-Reply-To: <1388121246.71.0.937149594906.issue1665@bugs.pypy.org>
Message-ID: <1388121246.71.0.937149594906.issue1665@bugs.pypy.org>


New submission from danblack <daniel.subs at internode.on.net>:

$ pypy -V
Python 2.7.3 (352c78d2e80f4a812ae1d8cdbe8c01a7f2e6fbc0, Aug 19 2013, 10:43:46)
[PyPy 2.1.0 with GCC 4.8.1 20130603 (Red Hat 4.8.1-1)]
(pypyenv) $ ipython
Python 2.7.3 (352c78d2e80f4a812ae1d8cdbe8c01a7f2e6fbc0, Aug 19 2013, 10:43:46)
In [1]: import re
In [2]: m = re.search('(?P<alpha>abc)','abcdef')
In [3]: m.group('CAT')
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-3-506c488db4df> in <module>()
----> 1 m.group('CAT')

KeyError: 'CAT'

CPython (2.7.5)
In [1]: import re
In [2]: m = re.search('(?P<alpha>abc)','abcdef')
In [3]: m.group('CAT')
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-3-506c488db4df> in <module>()
----> 1 m.group('CAT')

IndexError: no such group

----------
messages: 6446
nosy: danblack, pypy-issue
priority: bug
release: 2.1
status: unread
title: Module re behavior different from CPython in Match.group

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1665>
________________________________________

From tracker at bugs.pypy.org  Fri Dec 27 10:51:43 2013
From: tracker at bugs.pypy.org (Armin Rigo)
Date: Fri, 27 Dec 2013 09:51:43 +0000
Subject: [pypy-issue] [issue1665] Module re behavior different from CPython
	in Match.group
In-Reply-To: <1388121246.71.0.937149594906.issue1665@bugs.pypy.org>
Message-ID: <1388137903.02.0.0324079557835.issue1665@bugs.pypy.org>


Armin Rigo <armin.rigo at gmail.com> added the comment:

Thanks!  Fixed in 180796aa92c5.

----------
nosy: +arigo
status: unread -> resolved

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1665>
________________________________________