From tuom.larsen at gmail.com Mon Apr 4 09:10:10 2016 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Mon, 4 Apr 2016 15:10:10 +0200 Subject: [pypy-dev] Shift and typed array Message-ID: Hello! Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', [0])` (item size is 8 bytes). In Python, when an integer does not fit machine width it gets promoted to "long" integer of arbitrary size. So this will fail: a[0] = 2**63 << 1 To fix this, one could instead write: a[0] = (2**63 << 1) & (2**64 - 1) My question is, when I know that the result will be stored in `array.array` anyway, how to prevent the promotion to long integers? What is the most performat way to perform such calculations? Is PyPy able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? I mean, in C I wouldn't have to worry about it as everything above the 63rd bit will be simply cut off. I would like to help PyPy to generate the best possible code, does anyone have some suggestions please? Thanks! From fijall at gmail.com Mon Apr 4 09:32:37 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 4 Apr 2016 15:32:37 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: one option would be to use integers from _numpypy module: from numpy import int64 after installing numpy. There are obscure ways to get it without installing numpy. Another avenue would be to use __pypy__.intop.int_mul etc. Feel free to complain "no, I want real types that I can work with" :-) Cheers, fijal On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: > Hello! > > Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', > [0])` (item size is 8 bytes). In Python, when an integer does not fit > machine width it gets promoted to "long" integer of arbitrary size. So > this will fail: > > a[0] = 2**63 << 1 > > To fix this, one could instead write: > > a[0] = (2**63 << 1) & (2**64 - 1) > > My question is, when I know that the result will be stored in > `array.array` anyway, how to prevent the promotion to long integers? > What is the most performat way to perform such calculations? Is PyPy > able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? > > I mean, in C I wouldn't have to worry about it as everything above the > 63rd bit will be simply cut off. I would like to help PyPy to generate > the best possible code, does anyone have some suggestions please? > > Thanks! > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Mon Apr 4 09:38:39 2016 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Mon, 4 Apr 2016 15:38:39 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: You mean I should first store the result into numpy's `int64`, and then to `array.array`? Like: x = int64(2**63 << 1) a[0] = x Or: x = int64(2**63) x[0] = x << 1 What the "real types" goes, is this the only option? Thanks in any case! On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: > one option would be to use integers from _numpypy module: > > from numpy import int64 after installing numpy. > > There are obscure ways to get it without installing numpy. Another > avenue would be to use __pypy__.intop.int_mul etc. > > Feel free to complain "no, I want real types that I can work with" :-) > > Cheers, > fijal > > On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >> Hello! >> >> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >> [0])` (item size is 8 bytes). In Python, when an integer does not fit >> machine width it gets promoted to "long" integer of arbitrary size. So >> this will fail: >> >> a[0] = 2**63 << 1 >> >> To fix this, one could instead write: >> >> a[0] = (2**63 << 1) & (2**64 - 1) >> >> My question is, when I know that the result will be stored in >> `array.array` anyway, how to prevent the promotion to long integers? >> What is the most performat way to perform such calculations? Is PyPy >> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >> >> I mean, in C I wouldn't have to worry about it as everything above the >> 63rd bit will be simply cut off. I would like to help PyPy to generate >> the best possible code, does anyone have some suggestions please? >> >> Thanks! >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev From fijall at gmail.com Mon Apr 4 09:45:13 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 4 Apr 2016 15:45:13 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: so numpy64 will give you wrap-around arithmetics. What else are you looking for? :-) On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: > You mean I should first store the result into numpy's `int64`, and > then to `array.array`? Like: > > x = int64(2**63 << 1) > a[0] = x > > Or: > > x = int64(2**63) > x[0] = x << 1 > > What the "real types" goes, is this the only option? > > Thanks in any case! > > > On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >> one option would be to use integers from _numpypy module: >> >> from numpy import int64 after installing numpy. >> >> There are obscure ways to get it without installing numpy. Another >> avenue would be to use __pypy__.intop.int_mul etc. >> >> Feel free to complain "no, I want real types that I can work with" :-) >> >> Cheers, >> fijal >> >> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>> Hello! >>> >>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>> machine width it gets promoted to "long" integer of arbitrary size. So >>> this will fail: >>> >>> a[0] = 2**63 << 1 >>> >>> To fix this, one could instead write: >>> >>> a[0] = (2**63 << 1) & (2**64 - 1) >>> >>> My question is, when I know that the result will be stored in >>> `array.array` anyway, how to prevent the promotion to long integers? >>> What is the most performat way to perform such calculations? Is PyPy >>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>> >>> I mean, in C I wouldn't have to worry about it as everything above the >>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>> the best possible code, does anyone have some suggestions please? >>> >>> Thanks! >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Mon Apr 4 09:48:02 2016 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Mon, 4 Apr 2016 15:48:02 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: I would like to avoid numpy, if possible. Even if it might be bundled with PyPy (is it?) I still would like the code to run in CPython with no dependencies. On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: > so numpy64 will give you wrap-around arithmetics. What else are you > looking for? :-) > > On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >> You mean I should first store the result into numpy's `int64`, and >> then to `array.array`? Like: >> >> x = int64(2**63 << 1) >> a[0] = x >> >> Or: >> >> x = int64(2**63) >> x[0] = x << 1 >> >> What the "real types" goes, is this the only option? >> >> Thanks in any case! >> >> >> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>> one option would be to use integers from _numpypy module: >>> >>> from numpy import int64 after installing numpy. >>> >>> There are obscure ways to get it without installing numpy. Another >>> avenue would be to use __pypy__.intop.int_mul etc. >>> >>> Feel free to complain "no, I want real types that I can work with" :-) >>> >>> Cheers, >>> fijal >>> >>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>> Hello! >>>> >>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>> this will fail: >>>> >>>> a[0] = 2**63 << 1 >>>> >>>> To fix this, one could instead write: >>>> >>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>> >>>> My question is, when I know that the result will be stored in >>>> `array.array` anyway, how to prevent the promotion to long integers? >>>> What is the most performat way to perform such calculations? Is PyPy >>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>> >>>> I mean, in C I wouldn't have to worry about it as everything above the >>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>> the best possible code, does anyone have some suggestions please? >>>> >>>> Thanks! >>>> _______________________________________________ >>>> pypy-dev mailing list >>>> pypy-dev at python.org >>>> https://mail.python.org/mailman/listinfo/pypy-dev From fijall at gmail.com Mon Apr 4 09:49:17 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 4 Apr 2016 15:49:17 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: right, so there is no way to get wrap-around arithmetics in cpython without modifications On Mon, Apr 4, 2016 at 3:48 PM, Tuom Larsen wrote: > I would like to avoid numpy, if possible. Even if it might be bundled > with PyPy (is it?) I still would like the code to run in CPython with > no dependencies. > > On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: >> so numpy64 will give you wrap-around arithmetics. What else are you >> looking for? :-) >> >> On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >>> You mean I should first store the result into numpy's `int64`, and >>> then to `array.array`? Like: >>> >>> x = int64(2**63 << 1) >>> a[0] = x >>> >>> Or: >>> >>> x = int64(2**63) >>> x[0] = x << 1 >>> >>> What the "real types" goes, is this the only option? >>> >>> Thanks in any case! >>> >>> >>> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>>> one option would be to use integers from _numpypy module: >>>> >>>> from numpy import int64 after installing numpy. >>>> >>>> There are obscure ways to get it without installing numpy. Another >>>> avenue would be to use __pypy__.intop.int_mul etc. >>>> >>>> Feel free to complain "no, I want real types that I can work with" :-) >>>> >>>> Cheers, >>>> fijal >>>> >>>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>>> Hello! >>>>> >>>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>>> this will fail: >>>>> >>>>> a[0] = 2**63 << 1 >>>>> >>>>> To fix this, one could instead write: >>>>> >>>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>>> >>>>> My question is, when I know that the result will be stored in >>>>> `array.array` anyway, how to prevent the promotion to long integers? >>>>> What is the most performat way to perform such calculations? Is PyPy >>>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>>> >>>>> I mean, in C I wouldn't have to worry about it as everything above the >>>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>>> the best possible code, does anyone have some suggestions please? >>>>> >>>>> Thanks! >>>>> _______________________________________________ >>>>> pypy-dev mailing list >>>>> pypy-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/pypy-dev From fijall at gmail.com Mon Apr 4 09:49:34 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 4 Apr 2016 15:49:34 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: but if you dont actually overflow, the cost is really low btw On Mon, Apr 4, 2016 at 3:49 PM, Maciej Fijalkowski wrote: > right, so there is no way to get wrap-around arithmetics in cpython > without modifications > > On Mon, Apr 4, 2016 at 3:48 PM, Tuom Larsen wrote: >> I would like to avoid numpy, if possible. Even if it might be bundled >> with PyPy (is it?) I still would like the code to run in CPython with >> no dependencies. >> >> On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: >>> so numpy64 will give you wrap-around arithmetics. What else are you >>> looking for? :-) >>> >>> On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >>>> You mean I should first store the result into numpy's `int64`, and >>>> then to `array.array`? Like: >>>> >>>> x = int64(2**63 << 1) >>>> a[0] = x >>>> >>>> Or: >>>> >>>> x = int64(2**63) >>>> x[0] = x << 1 >>>> >>>> What the "real types" goes, is this the only option? >>>> >>>> Thanks in any case! >>>> >>>> >>>> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>>>> one option would be to use integers from _numpypy module: >>>>> >>>>> from numpy import int64 after installing numpy. >>>>> >>>>> There are obscure ways to get it without installing numpy. Another >>>>> avenue would be to use __pypy__.intop.int_mul etc. >>>>> >>>>> Feel free to complain "no, I want real types that I can work with" :-) >>>>> >>>>> Cheers, >>>>> fijal >>>>> >>>>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>>>> Hello! >>>>>> >>>>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>>>> this will fail: >>>>>> >>>>>> a[0] = 2**63 << 1 >>>>>> >>>>>> To fix this, one could instead write: >>>>>> >>>>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>>>> >>>>>> My question is, when I know that the result will be stored in >>>>>> `array.array` anyway, how to prevent the promotion to long integers? >>>>>> What is the most performat way to perform such calculations? Is PyPy >>>>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>>>> >>>>>> I mean, in C I wouldn't have to worry about it as everything above the >>>>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>>>> the best possible code, does anyone have some suggestions please? >>>>>> >>>>>> Thanks! >>>>>> _______________________________________________ >>>>>> pypy-dev mailing list >>>>>> pypy-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Mon Apr 4 09:52:01 2016 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Mon, 4 Apr 2016 15:52:01 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: Yes, I just want to write code which is portable (as for example that `& ...`) but still favors PyPy. On Mon, Apr 4, 2016 at 3:49 PM, Maciej Fijalkowski wrote: > right, so there is no way to get wrap-around arithmetics in cpython > without modifications > > On Mon, Apr 4, 2016 at 3:48 PM, Tuom Larsen wrote: >> I would like to avoid numpy, if possible. Even if it might be bundled >> with PyPy (is it?) I still would like the code to run in CPython with >> no dependencies. >> >> On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: >>> so numpy64 will give you wrap-around arithmetics. What else are you >>> looking for? :-) >>> >>> On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >>>> You mean I should first store the result into numpy's `int64`, and >>>> then to `array.array`? Like: >>>> >>>> x = int64(2**63 << 1) >>>> a[0] = x >>>> >>>> Or: >>>> >>>> x = int64(2**63) >>>> x[0] = x << 1 >>>> >>>> What the "real types" goes, is this the only option? >>>> >>>> Thanks in any case! >>>> >>>> >>>> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>>>> one option would be to use integers from _numpypy module: >>>>> >>>>> from numpy import int64 after installing numpy. >>>>> >>>>> There are obscure ways to get it without installing numpy. Another >>>>> avenue would be to use __pypy__.intop.int_mul etc. >>>>> >>>>> Feel free to complain "no, I want real types that I can work with" :-) >>>>> >>>>> Cheers, >>>>> fijal >>>>> >>>>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>>>> Hello! >>>>>> >>>>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>>>> this will fail: >>>>>> >>>>>> a[0] = 2**63 << 1 >>>>>> >>>>>> To fix this, one could instead write: >>>>>> >>>>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>>>> >>>>>> My question is, when I know that the result will be stored in >>>>>> `array.array` anyway, how to prevent the promotion to long integers? >>>>>> What is the most performat way to perform such calculations? Is PyPy >>>>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>>>> >>>>>> I mean, in C I wouldn't have to worry about it as everything above the >>>>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>>>> the best possible code, does anyone have some suggestions please? >>>>>> >>>>>> Thanks! >>>>>> _______________________________________________ >>>>>> pypy-dev mailing list >>>>>> pypy-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Mon Apr 4 09:53:42 2016 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Mon, 4 Apr 2016 15:53:42 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: So other than numpy, do you think: a[0] = (2**63 << 1) & (2**64 - 1) is the best way I can do? On Mon, Apr 4, 2016 at 3:52 PM, Tuom Larsen wrote: > Yes, I just want to write code which is portable (as for example that > `& ...`) but still favors PyPy. > > On Mon, Apr 4, 2016 at 3:49 PM, Maciej Fijalkowski wrote: >> right, so there is no way to get wrap-around arithmetics in cpython >> without modifications >> >> On Mon, Apr 4, 2016 at 3:48 PM, Tuom Larsen wrote: >>> I would like to avoid numpy, if possible. Even if it might be bundled >>> with PyPy (is it?) I still would like the code to run in CPython with >>> no dependencies. >>> >>> On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: >>>> so numpy64 will give you wrap-around arithmetics. What else are you >>>> looking for? :-) >>>> >>>> On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >>>>> You mean I should first store the result into numpy's `int64`, and >>>>> then to `array.array`? Like: >>>>> >>>>> x = int64(2**63 << 1) >>>>> a[0] = x >>>>> >>>>> Or: >>>>> >>>>> x = int64(2**63) >>>>> x[0] = x << 1 >>>>> >>>>> What the "real types" goes, is this the only option? >>>>> >>>>> Thanks in any case! >>>>> >>>>> >>>>> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>>>>> one option would be to use integers from _numpypy module: >>>>>> >>>>>> from numpy import int64 after installing numpy. >>>>>> >>>>>> There are obscure ways to get it without installing numpy. Another >>>>>> avenue would be to use __pypy__.intop.int_mul etc. >>>>>> >>>>>> Feel free to complain "no, I want real types that I can work with" :-) >>>>>> >>>>>> Cheers, >>>>>> fijal >>>>>> >>>>>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>>>>> Hello! >>>>>>> >>>>>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>>>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>>>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>>>>> this will fail: >>>>>>> >>>>>>> a[0] = 2**63 << 1 >>>>>>> >>>>>>> To fix this, one could instead write: >>>>>>> >>>>>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>>>>> >>>>>>> My question is, when I know that the result will be stored in >>>>>>> `array.array` anyway, how to prevent the promotion to long integers? >>>>>>> What is the most performat way to perform such calculations? Is PyPy >>>>>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>>>>> >>>>>>> I mean, in C I wouldn't have to worry about it as everything above the >>>>>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>>>>> the best possible code, does anyone have some suggestions please? >>>>>>> >>>>>>> Thanks! >>>>>>> _______________________________________________ >>>>>>> pypy-dev mailing list >>>>>>> pypy-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev From fijall at gmail.com Mon Apr 4 09:58:02 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 4 Apr 2016 15:58:02 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: you can write a if in_pypy wrapper that uses __pypy__.intops or something, but for exactly the same code is hard to do better. We can think about adding some optimizations that do that btw On Mon, Apr 4, 2016 at 3:53 PM, Tuom Larsen wrote: > So other than numpy, do you think: > > a[0] = (2**63 << 1) & (2**64 - 1) > > is the best way I can do? > > On Mon, Apr 4, 2016 at 3:52 PM, Tuom Larsen wrote: >> Yes, I just want to write code which is portable (as for example that >> `& ...`) but still favors PyPy. >> >> On Mon, Apr 4, 2016 at 3:49 PM, Maciej Fijalkowski wrote: >>> right, so there is no way to get wrap-around arithmetics in cpython >>> without modifications >>> >>> On Mon, Apr 4, 2016 at 3:48 PM, Tuom Larsen wrote: >>>> I would like to avoid numpy, if possible. Even if it might be bundled >>>> with PyPy (is it?) I still would like the code to run in CPython with >>>> no dependencies. >>>> >>>> On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: >>>>> so numpy64 will give you wrap-around arithmetics. What else are you >>>>> looking for? :-) >>>>> >>>>> On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >>>>>> You mean I should first store the result into numpy's `int64`, and >>>>>> then to `array.array`? Like: >>>>>> >>>>>> x = int64(2**63 << 1) >>>>>> a[0] = x >>>>>> >>>>>> Or: >>>>>> >>>>>> x = int64(2**63) >>>>>> x[0] = x << 1 >>>>>> >>>>>> What the "real types" goes, is this the only option? >>>>>> >>>>>> Thanks in any case! >>>>>> >>>>>> >>>>>> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>>>>>> one option would be to use integers from _numpypy module: >>>>>>> >>>>>>> from numpy import int64 after installing numpy. >>>>>>> >>>>>>> There are obscure ways to get it without installing numpy. Another >>>>>>> avenue would be to use __pypy__.intop.int_mul etc. >>>>>>> >>>>>>> Feel free to complain "no, I want real types that I can work with" :-) >>>>>>> >>>>>>> Cheers, >>>>>>> fijal >>>>>>> >>>>>>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>>>>>> Hello! >>>>>>>> >>>>>>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>>>>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>>>>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>>>>>> this will fail: >>>>>>>> >>>>>>>> a[0] = 2**63 << 1 >>>>>>>> >>>>>>>> To fix this, one could instead write: >>>>>>>> >>>>>>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>>>>>> >>>>>>>> My question is, when I know that the result will be stored in >>>>>>>> `array.array` anyway, how to prevent the promotion to long integers? >>>>>>>> What is the most performat way to perform such calculations? Is PyPy >>>>>>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>>>>>> >>>>>>>> I mean, in C I wouldn't have to worry about it as everything above the >>>>>>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>>>>>> the best possible code, does anyone have some suggestions please? >>>>>>>> >>>>>>>> Thanks! >>>>>>>> _______________________________________________ >>>>>>>> pypy-dev mailing list >>>>>>>> pypy-dev at python.org >>>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev From tuom.larsen at gmail.com Mon Apr 4 10:00:20 2016 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Mon, 4 Apr 2016 16:00:20 +0200 Subject: [pypy-dev] Shift and typed array In-Reply-To: References: Message-ID: Alright, thanks again! On Mon, Apr 4, 2016 at 3:58 PM, Maciej Fijalkowski wrote: > you can write a if in_pypy wrapper that uses __pypy__.intops or > something, but for exactly the same code is hard to do better. We can > think about adding some optimizations that do that btw > > > On Mon, Apr 4, 2016 at 3:53 PM, Tuom Larsen wrote: >> So other than numpy, do you think: >> >> a[0] = (2**63 << 1) & (2**64 - 1) >> >> is the best way I can do? >> >> On Mon, Apr 4, 2016 at 3:52 PM, Tuom Larsen wrote: >>> Yes, I just want to write code which is portable (as for example that >>> `& ...`) but still favors PyPy. >>> >>> On Mon, Apr 4, 2016 at 3:49 PM, Maciej Fijalkowski wrote: >>>> right, so there is no way to get wrap-around arithmetics in cpython >>>> without modifications >>>> >>>> On Mon, Apr 4, 2016 at 3:48 PM, Tuom Larsen wrote: >>>>> I would like to avoid numpy, if possible. Even if it might be bundled >>>>> with PyPy (is it?) I still would like the code to run in CPython with >>>>> no dependencies. >>>>> >>>>> On Mon, Apr 4, 2016 at 3:45 PM, Maciej Fijalkowski wrote: >>>>>> so numpy64 will give you wrap-around arithmetics. What else are you >>>>>> looking for? :-) >>>>>> >>>>>> On Mon, Apr 4, 2016 at 3:38 PM, Tuom Larsen wrote: >>>>>>> You mean I should first store the result into numpy's `int64`, and >>>>>>> then to `array.array`? Like: >>>>>>> >>>>>>> x = int64(2**63 << 1) >>>>>>> a[0] = x >>>>>>> >>>>>>> Or: >>>>>>> >>>>>>> x = int64(2**63) >>>>>>> x[0] = x << 1 >>>>>>> >>>>>>> What the "real types" goes, is this the only option? >>>>>>> >>>>>>> Thanks in any case! >>>>>>> >>>>>>> >>>>>>> On Mon, Apr 4, 2016 at 3:32 PM, Maciej Fijalkowski wrote: >>>>>>>> one option would be to use integers from _numpypy module: >>>>>>>> >>>>>>>> from numpy import int64 after installing numpy. >>>>>>>> >>>>>>>> There are obscure ways to get it without installing numpy. Another >>>>>>>> avenue would be to use __pypy__.intop.int_mul etc. >>>>>>>> >>>>>>>> Feel free to complain "no, I want real types that I can work with" :-) >>>>>>>> >>>>>>>> Cheers, >>>>>>>> fijal >>>>>>>> >>>>>>>> On Mon, Apr 4, 2016 at 3:10 PM, Tuom Larsen wrote: >>>>>>>>> Hello! >>>>>>>>> >>>>>>>>> Suppose I'm on 64-bit machine and there is an `a = arrar.array('L', >>>>>>>>> [0])` (item size is 8 bytes). In Python, when an integer does not fit >>>>>>>>> machine width it gets promoted to "long" integer of arbitrary size. So >>>>>>>>> this will fail: >>>>>>>>> >>>>>>>>> a[0] = 2**63 << 1 >>>>>>>>> >>>>>>>>> To fix this, one could instead write: >>>>>>>>> >>>>>>>>> a[0] = (2**63 << 1) & (2**64 - 1) >>>>>>>>> >>>>>>>>> My question is, when I know that the result will be stored in >>>>>>>>> `array.array` anyway, how to prevent the promotion to long integers? >>>>>>>>> What is the most performat way to perform such calculations? Is PyPy >>>>>>>>> able to optimize away that `& (2**64 - 1)` when I use `'L'` typecode? >>>>>>>>> >>>>>>>>> I mean, in C I wouldn't have to worry about it as everything above the >>>>>>>>> 63rd bit will be simply cut off. I would like to help PyPy to generate >>>>>>>>> the best possible code, does anyone have some suggestions please? >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> _______________________________________________ >>>>>>>>> pypy-dev mailing list >>>>>>>>> pypy-dev at python.org >>>>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev From petr.hanousek at cesnet.cz Fri Apr 8 04:11:11 2016 From: petr.hanousek at cesnet.cz (Petr Hanousek) Date: Fri, 8 Apr 2016 10:11:11 +0200 Subject: [pypy-dev] compilation problem while testing libffi Message-ID: <5707679F.7060103@cesnet.cz> Dears, I have tried to compile pypy 5.0.1 according to http://pypy.org/download.html#building-from-source. First compiled libffi-3.2.1 and installed to /software/pypy/5.0.0/gcc and then exported the PKG_CONFIG_PATH to the appropripate directory. Result from "python ../../rpython/bin/rpython -Ojit targetpypystandalone" is in the attachment. What could be wrong, please? The /software path is somewhere on AFS (network storage) and I have full rights on it. I assume there is no problem to run the compilation as root. When I try to run by hand the last two steps before error occurs, no error appears. Thank you, Petr -- +-------------------------------------------------------------------+ Petr Hanousek e-mail: petr.hanousek at cesnet.cz User Support phone: +420 950 072 112 CESNET z.s.p.o. mobile: +420 606 665 139 location: Zikova 4, Praha room: 90a Czech Republic +-------------------------------------------------------------------+ -------------- next part -------------- A non-text attachment was scrubbed... Name: compilation.log Type: text/x-log Size: 20590 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3749 bytes Desc: S/MIME Cryptographic Signature URL: From petr.hanousek at cesnet.cz Sat Apr 9 17:18:16 2016 From: petr.hanousek at cesnet.cz (Petr Hanousek) Date: Sat, 9 Apr 2016 23:18:16 +0200 Subject: [pypy-dev] compilation problem while testing libffi In-Reply-To: <5707679F.7060103@cesnet.cz> References: <5707679F.7060103@cesnet.cz> Message-ID: <57097198.4010506@cesnet.cz> So I found the problem and now everything seems to compile OK. I haven't properly set LD_LIBRARY_PATH to my installed libffi. It took me some time to find where is a problem. And it would be very useful to have better clue than just: raise Exception("run_example_code failed!\nlocals = %r" % (locals(),)) [translation:ERROR] Exception: run_example_code failed! If there was written the output from the test: /tmp/usession-release-5.0.1-2/platcheck_28: error while loading shared libraries: libffi.so.6: cannot open shared object file: No such file or directory , everything will be much more clear. Petr On 8.4.2016 10:11, Petr Hanousek wrote: > Dears, > I have tried to compile pypy 5.0.1 according to > http://pypy.org/download.html#building-from-source. First compiled > libffi-3.2.1 and installed to /software/pypy/5.0.0/gcc and then exported > the PKG_CONFIG_PATH to the appropripate directory. Result from "python > ../../rpython/bin/rpython -Ojit targetpypystandalone" is in the > attachment. What could be wrong, please? > The /software path is somewhere on AFS (network storage) and I have full > rights on it. I assume there is no problem to run the compilation as > root. When I try to run by hand the last two steps before error occurs, > no error appears. > Thank you, Petr -- +-------------------------------------------------------------------+ Petr Hanousek e-mail: petr.hanousek at cesnet.cz User Support phone: +420 950 072 112 CESNET z.s.p.o. mobile: +420 606 665 139 location: Zikova 4, Praha room: 90a Czech Republic +-------------------------------------------------------------------+ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3749 bytes Desc: S/MIME Cryptographic Signature URL: From arigo at tunes.org Sun Apr 10 08:32:38 2016 From: arigo at tunes.org (Armin Rigo) Date: Sun, 10 Apr 2016 15:32:38 +0300 Subject: [pypy-dev] compilation problem while testing libffi In-Reply-To: <57097198.4010506@cesnet.cz> References: <5707679F.7060103@cesnet.cz> <57097198.4010506@cesnet.cz> Message-ID: Hi Petr, On 10 April 2016 at 00:18, Petr Hanousek wrote: > So I found the problem and now everything seems to compile OK. Ok, that's nonsense to just eat and ignore stderr. I fixed it in 418eb4be7f64. A bient?t, Armin. From matti.picus at gmail.com Wed Apr 20 00:29:40 2016 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 20 Apr 2016 07:29:40 +0300 Subject: [pypy-dev] PyPY 5.1 release is close, help needed Message-ID: <571705B4.9010202@gmail.com> An HTML attachment was scrubbed... URL: From planrichi at gmail.com Wed Apr 20 05:30:49 2016 From: planrichi at gmail.com (Richard Plangger) Date: Wed, 20 Apr 2016 11:30:49 +0200 Subject: [pypy-dev] PyPY 5.1 release is close, help needed In-Reply-To: <571705B4.9010202@gmail.com> References: <571705B4.9010202@gmail.com> Message-ID: <57174C49.7080105@gmail.com> Hello, > I need help: > - please check that the targz bundles of release 3260adbeba4a, available > here > http://buildbot.pypy.org/nightly/release-5.x > are usable on your platform I ran the benchmark suite on s390x. Additionally I even setup some django service with virtual env to see if there are any complications. It all worked fine. I'm planning to do the same on x86 64 bit. > - Could someone with a large upload bandwidth run the repackages script > pypy/tool/release/repackage.sh > and then upload the renamed packages to bitbucket I can volunteer! I should probably wait some more until I upload it or should I just go ahead? > - Please review the release notice > http://doc.pypy.org/en/latest/release-5.1.0.html There is a typo (s960x should be s390x). I already fixed it on default. Cheers, Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From fijall at gmail.com Wed Apr 20 05:36:08 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 20 Apr 2016 11:36:08 +0200 Subject: [pypy-dev] PyPY 5.1 release is close, help needed In-Reply-To: <571705B4.9010202@gmail.com> References: <571705B4.9010202@gmail.com> Message-ID: can we stress more the memory/warmup improvements? jit-leaner-frontend makes the warmup quite a bit faster, additionally we improved memory (both should be on the order of 20% compared to 5.0) On Wed, Apr 20, 2016 at 6:29 AM, Matti Picus wrote: > We are close to releasing PyPy 5.1. The builds of version 3260adbeba4a look > clean, and the release notice is up-to-date. > > I need help: > - please check that the targz bundles of release 3260adbeba4a, available > here > http://buildbot.pypy.org/nightly/release-5.x > are usable on your platform > > - Could someone with a large upload bandwidth run the repackages script > pypy/tool/release/repackage.sh > and then upload the renamed packages to bitbucket > > - Please review the release notice > http://doc.pypy.org/en/latest/release-5.1.0.html > > Thanks > Matti > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From matti.picus at gmail.com Thu Apr 21 11:09:10 2016 From: matti.picus at gmail.com (Matti Picus) Date: Thu, 21 Apr 2016 18:09:10 +0300 Subject: [pypy-dev] 5.1 has been released Message-ID: <5718ED16.90903@gmail.com> An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Fri Apr 22 01:03:36 2016 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 22 Apr 2016 08:03:36 +0300 Subject: [pypy-dev] 5.1 has been released In-Reply-To: References: <5718ED16.90903@gmail.com> Message-ID: <5719B0A8.3020208@gmail.com> On 22/04/16 07:02, Phyo Arkar wrote: > Congrats PyPy Team! > It keeps getting better and better! > And thank you very much for improving cpyext. > > > Fix edge cases in the cpyext refcounting-compatible semantics (more > work on cpyext compatibility is coming in the cpyext-ext branch, but > isn?t ready yet) > > So thats mean , soon, infuture , cpyext will be fully supported? > For some definition of 'infuture' and 'fully' - yes. The larger goal is to enable the scientific python stack with only minor upstream changes, IMO cpyext-ext will be finished when numpy's test suite passes with no failures on the fork of numpy available on github at github.com/pypy/numpy (not to be confused with the one at bitbucket/pypy/numpy, which uses micronumpy and cffi) Eventually, we will modify micronumpy so that app-level python calls with ndarrays will somehow be hijacked to go through micronumpy rather than the C-API functions, assuming this is even possible. It is a big job, help is welcome Matti From phyo.arkarlwin at gmail.com Fri Apr 22 08:17:15 2016 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Fri, 22 Apr 2016 12:17:15 +0000 Subject: [pypy-dev] 5.1 has been released In-Reply-To: <5719B0A8.3020208@gmail.com> References: <5718ED16.90903@gmail.com> <5719B0A8.3020208@gmail.com> Message-ID: I would like to help , i can get Two Interns who love python a lot . But cpyext needs C skills , right? On Fri, Apr 22, 2016 at 11:34 AM Matti Picus wrote: > On 22/04/16 07:02, Phyo Arkar wrote: > > Congrats PyPy Team! > > It keeps getting better and better! > > And thank you very much for improving cpyext. > > > > > Fix edge cases in the cpyext refcounting-compatible semantics (more > > work on cpyext compatibility is coming in the cpyext-ext branch, but > > isn?t ready yet) > > > > So thats mean , soon, infuture , cpyext will be fully supported? > > > For some definition of 'infuture' and 'fully' - yes. > The larger goal is to enable the scientific python stack with only minor > upstream changes, IMO cpyext-ext will be finished when numpy's test > suite passes with no failures on the fork of numpy available on github > at github.com/pypy/numpy (not to be confused with the one at > bitbucket/pypy/numpy, which uses micronumpy and cffi) > > Eventually, we will modify micronumpy so that app-level python calls > with ndarrays will somehow be hijacked to go through micronumpy rather > than the C-API functions, assuming this is even possible. > > It is a big job, help is welcome > Matti > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From flebber.crue at gmail.com Tue Apr 26 05:53:25 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Tue, 26 Apr 2016 09:53:25 +0000 Subject: [pypy-dev] Error Messages Message-ID: Hi Just wondering if PyPy error messages are or could be of a higher quality than the CPython error messages? One of the goals of Rubinius a former Ruby alternate implementation similar in intent to PyPy was improved developer feedback in error messages. Is this an area of opportunity for PyPy? Cheers Sayth -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Tue Apr 26 11:16:29 2016 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 26 Apr 2016 10:16:29 -0500 Subject: [pypy-dev] Error Messages In-Reply-To: References: Message-ID: I personally think it's fine: 1. CPython has pretty decent error messages. Other than long stack traces with recursion errors, or maybe column offsets, there isn't really anything that could be significantly improved. 2. Changing the actual errors would likely break...a *lot*! -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something?s wrong. http://kirbyfan64.github.io/ On Apr 26, 2016 4:54 AM, "Sayth Renshaw" wrote: > Hi > > Just wondering if PyPy error messages are or could be of a higher quality > than the CPython error messages? > > One of the goals of Rubinius a former Ruby alternate implementation > similar in intent to PyPy was improved developer feedback in error messages. > > Is this an area of opportunity for PyPy? > > Cheers > > Sayth > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfbolz at gmx.de Tue Apr 26 11:22:35 2016 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 26 Apr 2016 17:22:35 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: References: Message-ID: <4c616cb9-88d2-4f2a-b3b9-d1fffa2839c1@email.android.com> Hi all, Some points: - in a couple of places pypy has better error messages than cpython, eg around calling functions with the wrong number of arguments, in complex cases. - we are certainly open to patches that improve the quality of our error messages, if that can be done with reasonable effort. - PyPy does not guarantee the same error messages as cpython, and it's a bug to rely on them. Cheers, Carl Friedrich On April 26, 2016 5:16:29 PM GMT+02:00, Ryan Gonzalez wrote: >I personally think it's fine: > >1. CPython has pretty decent error messages. Other than long stack >traces >with recursion errors, or maybe column offsets, there isn't really >anything >that could be significantly improved. > >2. Changing the actual errors would likely break...a *lot*! > >-- >Ryan >[ERROR]: Your autotools build scripts are 200 lines longer than your >program. Something?s wrong. >http://kirbyfan64.github.io/ >On Apr 26, 2016 4:54 AM, "Sayth Renshaw" >wrote: > >> Hi >> >> Just wondering if PyPy error messages are or could be of a higher >quality >> than the CPython error messages? >> >> One of the goals of Rubinius a former Ruby alternate implementation >> similar in intent to PyPy was improved developer feedback in error >messages. >> >> Is this an area of opportunity for PyPy? >> >> Cheers >> >> Sayth >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> >> > > >------------------------------------------------------------------------ > >_______________________________________________ >pypy-dev mailing list >pypy-dev at python.org >https://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Apr 26 11:30:53 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 27 Apr 2016 01:30:53 +1000 Subject: [pypy-dev] Error Messages In-Reply-To: References: Message-ID: <20160426153052.GL13497@ando.pearwood.info> On Tue, Apr 26, 2016 at 10:16:29AM -0500, Ryan Gonzalez wrote: > I personally think it's fine: > > 1. CPython has pretty decent error messages. Other than long stack traces > with recursion errors, or maybe column offsets, there isn't really anything > that could be significantly improved. Well, I don't know about that... some of the error messages are a bit uninformative ("SyntaxError: invalid syntax", although it may be difficult to do much about that one). But potential improvements should be taken on a case-by-case basis. > 2. Changing the actual errors would likely break...a *lot*! Do you mean the exception types? Yes, changing the exception types would break a lot of code. But changing the error messages shouldn't break anything, since the error messages are not part of the function API. They've changed before, and they will change again. -- Steve From fijall at gmail.com Tue Apr 26 12:16:39 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 26 Apr 2016 18:16:39 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: <20160426153052.GL13497@ando.pearwood.info> References: <20160426153052.GL13497@ando.pearwood.info> Message-ID: On Tue, Apr 26, 2016 at 5:30 PM, Steven D'Aprano wrote: > On Tue, Apr 26, 2016 at 10:16:29AM -0500, Ryan Gonzalez wrote: >> I personally think it's fine: >> >> 1. CPython has pretty decent error messages. Other than long stack traces >> with recursion errors, or maybe column offsets, there isn't really anything >> that could be significantly improved. > > Well, I don't know about that... some of the error messages are a bit > uninformative ("SyntaxError: invalid syntax", although it may be > difficult to do much about that one). But potential improvements > should be taken on a case-by-case basis. > > >> 2. Changing the actual errors would likely break...a *lot*! > > Do you mean the exception types? Yes, changing the exception types would > break a lot of code. But changing the error messages shouldn't break > anything, since the error messages are not part of the function API. > They've changed before, and they will change again. Typically the exception type is the same, but there is a bunch of differences, especially around ValueError vs TypeError, noone should rely on that anyway From steve at pearwood.info Tue Apr 26 12:38:31 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 27 Apr 2016 02:38:31 +1000 Subject: [pypy-dev] Error Messages In-Reply-To: References: <20160426153052.GL13497@ando.pearwood.info> Message-ID: <20160426163830.GN13497@ando.pearwood.info> On Tue, Apr 26, 2016 at 06:16:39PM +0200, Maciej Fijalkowski wrote: > Typically the exception type is the same, but there is a bunch of > differences, especially around ValueError vs TypeError, noone should > rely on that anyway Do you mean that PyPy might change ValueError to TypeError, or vice versa? Like this: # TypeError in CPython len(None) => raises ValueError # ValueError in CPython [].index("a") => raises TypeError That doesn't sound good to me. If I have misunderstood you, and you're just talking about the error strings, then that's fine. -- Steve From fijall at gmail.com Tue Apr 26 12:47:48 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 26 Apr 2016 18:47:48 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: <20160426163830.GN13497@ando.pearwood.info> References: <20160426153052.GL13497@ando.pearwood.info> <20160426163830.GN13497@ando.pearwood.info> Message-ID: Not in *those* cases, but there are cases where CPython would raise different error classes depending e.g. on whether stuff is builtin function or not builtin function or the cpython errors are inconsistent. In those cases the exact error might be a bit different (I must admit I don't remember examples off hand) On Tue, Apr 26, 2016 at 6:38 PM, Steven D'Aprano wrote: > On Tue, Apr 26, 2016 at 06:16:39PM +0200, Maciej Fijalkowski wrote: > >> Typically the exception type is the same, but there is a bunch of >> differences, especially around ValueError vs TypeError, noone should >> rely on that anyway > > Do you mean that PyPy might change ValueError to TypeError, or vice > versa? Like this: > > # TypeError in CPython > len(None) > => raises ValueError > > # ValueError in CPython > [].index("a") > => raises TypeError > > That doesn't sound good to me. > > If I have misunderstood you, and you're just talking about the error > strings, then that's fine. > > > > -- > Steve > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From cfbolz at gmx.de Tue Apr 26 13:02:36 2016 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 26 Apr 2016 19:02:36 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: <20160426163830.GN13497@ando.pearwood.info> References: <20160426153052.GL13497@ando.pearwood.info> <20160426163830.GN13497@ando.pearwood.info> Message-ID: <058281d0-87a3-40df-89d7-b58422bb0d25@email.android.com> Those are simple cases, of course we use the same exception types there. However, if you write exactly the wrong obscure code you sometimes get a different exception type under some conditions. If you report them and they are easily fixable, we will. Carl Friedrich On April 26, 2016 6:38:31 PM GMT+02:00, Steven D'Aprano wrote: >On Tue, Apr 26, 2016 at 06:16:39PM +0200, Maciej Fijalkowski wrote: > >> Typically the exception type is the same, but there is a bunch of >> differences, especially around ValueError vs TypeError, noone should >> rely on that anyway > >Do you mean that PyPy might change ValueError to TypeError, or vice >versa? Like this: > ># TypeError in CPython >len(None) >=> raises ValueError > ># ValueError in CPython >[].index("a") >=> raises TypeError > >That doesn't sound good to me. > >If I have misunderstood you, and you're just talking about the error >strings, then that's fine. > > > >-- >Steve >_______________________________________________ >pypy-dev mailing list >pypy-dev at python.org >https://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Tue Apr 26 17:38:30 2016 From: arigo at tunes.org (Armin Rigo) Date: Tue, 26 Apr 2016 23:38:30 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: <058281d0-87a3-40df-89d7-b58422bb0d25@email.android.com> References: <20160426153052.GL13497@ando.pearwood.info> <20160426163830.GN13497@ando.pearwood.info> <058281d0-87a3-40df-89d7-b58422bb0d25@email.android.com> Message-ID: Hi, On 26 April 2016 at 19:02, Carl Friedrich Bolz wrote: > Those are simple cases, of course we use the same exception types there. > However, if you write exactly the wrong obscure code you sometimes get a > different exception type under some conditions. That's about TypeError versus AttributeError for some attributes of built-in types in corner cases. That's not about ValueError. For example: >>> def f(): pass >>> del f.__closure__ You get a TypeError in CPython, but an AttributeError in PyPy. That's because PyPy doesn't have the same zoo of built-in attribute types of CPython. For most purposes it doesn't make a difference, but it turns out that in this case half of these types raise AttributeError and the other half raises TypeError in CPython. A bient?t, Armin. From steve at pearwood.info Tue Apr 26 19:22:09 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 27 Apr 2016 09:22:09 +1000 Subject: [pypy-dev] Error Messages In-Reply-To: References: <20160426153052.GL13497@ando.pearwood.info> <20160426163830.GN13497@ando.pearwood.info> <058281d0-87a3-40df-89d7-b58422bb0d25@email.android.com> Message-ID: <20160426232209.GO13497@ando.pearwood.info> On Tue, Apr 26, 2016 at 11:38:30PM +0200, Armin Rigo wrote: > Hi, > > On 26 April 2016 at 19:02, Carl Friedrich Bolz wrote: > > Those are simple cases, of course we use the same exception types there. > > However, if you write exactly the wrong obscure code you sometimes get a > > different exception type under some conditions. > > That's about TypeError versus AttributeError for some attributes of > built-in types in corner cases. That's not about ValueError. For example: > > >>> def f(): pass > >>> del f.__closure__ > > You get a TypeError in CPython, but an AttributeError in PyPy. That's > because PyPy doesn't have the same zoo of built-in attribute types of > CPython. For most purposes it doesn't make a difference, but it turns > out that in this case half of these types raise AttributeError and the > other half raises TypeError in CPython. Ah, that makes sense! -- Steve From arigo at tunes.org Wed Apr 27 03:26:19 2016 From: arigo at tunes.org (Armin Rigo) Date: Wed, 27 Apr 2016 09:26:19 +0200 Subject: [pypy-dev] Reflex home page? Message-ID: Hi Wim, We are linking from the PyPy documentation to these pages: http://root.cern.ch/drupal/content/reflex http://root.cern.ch/drupal/content/generating-reflex-dictionaries They are 404 now. What should they be fixed to? A bient?t, Armin. From fijall at gmail.com Wed Apr 27 04:14:35 2016 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 27 Apr 2016 10:14:35 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: References: <20160426153052.GL13497@ando.pearwood.info> <20160426163830.GN13497@ando.pearwood.info> <058281d0-87a3-40df-89d7-b58422bb0d25@email.android.com> Message-ID: Hi Armin Any chance we can add this to cpython-differences? Couldn't find it last time (and misremembered) On Tue, Apr 26, 2016 at 11:38 PM, Armin Rigo wrote: > Hi, > > On 26 April 2016 at 19:02, Carl Friedrich Bolz wrote: >> Those are simple cases, of course we use the same exception types there. >> However, if you write exactly the wrong obscure code you sometimes get a >> different exception type under some conditions. > > That's about TypeError versus AttributeError for some attributes of > built-in types in corner cases. That's not about ValueError. For example: > >>>> def f(): pass >>>> del f.__closure__ > > You get a TypeError in CPython, but an AttributeError in PyPy. That's > because PyPy doesn't have the same zoo of built-in attribute types of > CPython. For most purposes it doesn't make a difference, but it turns > out that in this case half of these types raise AttributeError and the > other half raises TypeError in CPython. > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From arigo at tunes.org Wed Apr 27 04:31:40 2016 From: arigo at tunes.org (Armin Rigo) Date: Wed, 27 Apr 2016 10:31:40 +0200 Subject: [pypy-dev] Error Messages In-Reply-To: References: <20160426153052.GL13497@ando.pearwood.info> <20160426163830.GN13497@ando.pearwood.info> <058281d0-87a3-40df-89d7-b58422bb0d25@email.android.com> Message-ID: Hi, On 27 April 2016 at 10:14, Maciej Fijalkowski wrote: > Any chance we can add this to cpython-differences? Couldn't find it > last time (and misremembered) Done (434d21e15d0d). Armin From matti.picus at gmail.com Wed Apr 27 06:30:25 2016 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 27 Apr 2016 13:30:25 +0300 Subject: [pypy-dev] pypy 2.7 version 5.1.1 Message-ID: <572094C1.7010404@gmail.com> An HTML attachment was scrubbed... URL: From flebber.crue at gmail.com Wed Apr 27 07:23:23 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Wed, 27 Apr 2016 11:23:23 +0000 Subject: [pypy-dev] Error Messages In-Reply-To: <4c616cb9-88d2-4f2a-b3b9-d1fffa2839c1@email.android.com> References: <4c616cb9-88d2-4f2a-b3b9-d1fffa2839c1@email.android.com> Message-ID: Thanks for the feedback, just wondering really what the potential was. I sort of wish Ryan you hadn't used the word 'fine' its just a word that usually indicates something really needs to be done abs i had better shut up or its me; or it's about to break. Cheers Sayth On Wed, 27 Apr 2016 1:22 am Carl Friedrich Bolz wrote: > Hi all, > > Some points: > > - in a couple of places pypy has better error messages than cpython, eg > around calling functions with the wrong number of arguments, in complex > cases. > > - we are certainly open to patches that improve the quality of our error > messages, if that can be done with reasonable effort. > > - PyPy does not guarantee the same error messages as cpython, and it's a > bug to rely on them. > > Cheers, > > Carl Friedrich > > > On April 26, 2016 5:16:29 PM GMT+02:00, Ryan Gonzalez > wrote: >> >> I personally think it's fine: >> >> 1. CPython has pretty decent error messages. Other than long stack traces >> with recursion errors, or maybe column offsets, there isn't really anything >> that could be significantly improved. >> >> 2. Changing the actual errors would likely break...a *lot*! >> >> -- >> Ryan >> [ERROR]: Your autotools build scripts are 200 lines longer than your >> program. Something?s wrong. >> http://kirbyfan64.github.io/ >> On Apr 26, 2016 4:54 AM, "Sayth Renshaw" wrote: >> >>> Hi >>> >>> Just wondering if PyPy error messages are or could be of a higher >>> quality than the CPython error messages? >>> >>> One of the goals of Rubinius a former Ruby alternate implementation >>> similar in intent to PyPy was improved developer feedback in error messages. >>> >>> Is this an area of opportunity for PyPy? >>> >>> Cheers >>> >>> Sayth >>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev at python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >>> >>> ------------------------------ >> >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Wed Apr 27 08:11:56 2016 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Wed, 27 Apr 2016 07:11:56 -0500 Subject: [pypy-dev] Reflex home page? In-Reply-To: References: Message-ID: Only page I can find is: https://root.cern.ch/download/doc/ROOTUsersGuideHTML/ch07s10.html Looks like they just re-did the ROOT website. -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something?s wrong. http://kirbyfan64.github.io/ On Apr 27, 2016 2:27 AM, "Armin Rigo" wrote: > Hi Wim, > > We are linking from the PyPy documentation to these pages: > > http://root.cern.ch/drupal/content/reflex > http://root.cern.ch/drupal/content/generating-reflex-dictionaries > > They are 404 now. What should they be fixed to? > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wlavrijsen at lbl.gov Wed Apr 27 11:42:02 2016 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Wed, 27 Apr 2016 08:42:02 -0700 (PDT) Subject: [pypy-dev] Reflex home page? In-Reply-To: References: Message-ID: Armin, > http://root.cern.ch/drupal/content/reflex > http://root.cern.ch/drupal/content/generating-reflex-dictionaries > > They are 404 now. What should they be fixed to? https://root.cern.ch/how/how-use-reflex for both. Aside, Google approved a GSoC student to make the Cling backend happen. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From rveroy at cs.tufts.edu Wed Apr 27 16:42:59 2016 From: rveroy at cs.tufts.edu (Raoul Veroy) Date: Wed, 27 Apr 2016 15:42:59 -0500 Subject: [pypy-dev] Garbage collection internals Message-ID: Hi all. Sorry if this is the wrong venue for this. First a question about the old reference counting implementation in Pypy. I wanted to learn and work on the garbage collection of Pypy. First, in this paper from 2007 about PyPy , it mentions a reference counting GC transformer, which is not mentioned anymore in the 2.4.x documentation ( https://pypy.readthedocs.org/en/release-2.4.x/garbage_collection.html). The latest documentation doesn't seem to have a version of this garbage_collection page. I wanted to do some studies on reference counting and the implementation does not have to be efficient or fast. What is the best way to get at the old refcount implementation? Would it be to work on the last version to have it? Secondly, I wanted to log a trace of all allocations of objects and all updates to references. Is there infrastructure or tool that exists for this? Thanks. - Raoul -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Thu Apr 28 04:57:05 2016 From: arigo at tunes.org (Armin Rigo) Date: Thu, 28 Apr 2016 10:57:05 +0200 Subject: [pypy-dev] Garbage collection internals In-Reply-To: References: Message-ID: Hi Raoul, On 27 April 2016 at 22:42, Raoul Veroy wrote: > First a question about the old reference counting implementation in Pypy. It is still there, used mainly by tests of the C backend. But it wasn't used for translating the full PyPy for many years now, so you are very likely to get into various problems. > I wanted to do some studies on reference counting and the implementation > does not have to be efficient or fast. Well, a good reference counting implementation would be able to remove a large fraction of the incref/decref operations, by clever code analysis. I don't really understand what kind of study might be interesting on a bad reference counting implementation like ours. > Secondly, I wanted to log a trace of all allocations of objects and all > updates to references. Is there infrastructure or tool that exists for this? No. I tried in the distant past to do that to track a hard bug, with some success, but it involved too many hacks to be merged back into "default". If you want to dig, it was the branch "lltrace", last check-in 2011-02-24. Nowadays there would be additional complications; for example, the JIT often allocates several objects using a single pointer bump. A bient?t, Armin. From planrichi at gmail.com Fri Apr 29 10:54:48 2016 From: planrichi at gmail.com (Richard Plangger) Date: Fri, 29 Apr 2016 16:54:48 +0200 Subject: [pypy-dev] jitlog 2.0 Message-ID: <526144f8-dacb-6b40-0f99-1cb46ff21385@gmail.com> Hello, I have spent a lot of my free time to improve our jitviewer. Finally I'm happy to present some results and I want to get some feedback (bad or good). I run a test server (at http://217.172.172.207:8000) so without installing anything you can just have a quick look. Here [1] is richards.py uploaded to that server. Here some more information: What has changed? ================= We already host vmprof.com for profile data. This is only part of the information that is interesting. I think it would be very helpful to see what the JIT compiler currently produces in a more 'global view'. The combination of profile data available at vmprof.com and the log output of pypy can really help to understand what is going on inside the VM. The new log output is easier to process and allows to fully rebuild the information in jit viewer (with no dependency to the pypy source code). I have added a svg chart that displays the trace tree. It can be examined further which is really something. I think that is a much nicer high level view of our tracing JIT compiler than in the old jit viewer. Technically ----------- I have mostly rewritten the log format. It writes a binary file and currently encodes pretty much every information the PYPYLOG does. It is a custom binary format similar to the one in vmprof. It is intended to reduce the size of jitlog. We discussed that it would be a rather good idea to use a standardized binary protocol instead of creating our own. I agree with that, but there are no real options without significant effort. E.g. capnproto would have required me to write the encoder on the rpython side and adjust the javascript capnproto library that it is able to run in the browser (nodejs only). I decided to stick with the custom very simple binary format and extended it that it is able to version the jitlog. We can support several versions, ideally never dropping support for any official version of the jitlog. I extended vmprof-python [2] to read that binary file format and upload it along with the normal vmprof data. It has a pretty printer which you can use to get information on the console. It's state is very minimal and my current idea is to extend a very minimal api to be able to query the jitlog. I have something in mind like: $ python -m vmprof.prettyprint /tmp/abcdef.jitlog --query 'trace.get_func_name() == "schedule"' > 1x trace matched your query > trace started in the function schedule in path/to/richards.py:501 > label(...) > ... > jump(...) A sample command to upload jitlog data would be: python -m vmprof --jitlog --web --web-url http://217.172.172.207:8000 /richards.py Here are some things I'm aware of and want to change: * No compression of JSON. That would greatly speed up loading! * The live ranges are not connected when assembler is shown (style fix) * The entry count (of a trace) displayed in the trace view is always 0, the data is present but not serialized when invoked with -m vmprof --jitlog What's next =========== I hope someone could take a look at the new-jit-log branch so we can discuss if it. I also want to achieve: * Feature parity with the old jitviewer (what is missing?) * Be able to jump from vmprof profile data to jitlog and back * Show the jumps in the trace view * Merge to default :) * Upload the source code to give a better pictures of what has been produced from which source line * Display merge point information in the resoperation list * Does it work on IE? Cheers, Richard [1] http://217.172.172.207:8000/#/92a1ff97c87d187eee3f60a0a088e669/traces [2] https://github.com/vmprof/vmprof-python -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From arigo at tunes.org Fri Apr 29 11:14:32 2016 From: arigo at tunes.org (Armin Rigo) Date: Fri, 29 Apr 2016 17:14:32 +0200 Subject: [pypy-dev] jitlog 2.0 In-Reply-To: <526144f8-dacb-6b40-0f99-1cb46ff21385@gmail.com> References: <526144f8-dacb-6b40-0f99-1cb46ff21385@gmail.com> Message-ID: Hi Richard, On 29 April 2016 at 16:54, Richard Plangger wrote: > * Feature parity with the old jitviewer (what is missing?) A few "obvious" things from my point of view: 1. Showing the descrs is very useful. Currently, we get "p6 = getfield_gc_r(p0)"... thank you, but that's not enough to deduce anything about what is being read from what :-) 2. We don't know where a "jump" instruction is jumping to. 3. It would help to navigate if there was a way to go from a bridge back to the loop (or bridge) where it originates. Currently, even if we start from the loop and click on the bridge, I don't see a way to come back, and I can't use the browser "back" button either. Thanks for doing this! What I'm not listing here is all the new things that are definitely better than they used to be :-) A bient?t, Armin. From matti.picus at gmail.com Sat Apr 30 13:56:06 2016 From: matti.picus at gmail.com (Matti Picus) Date: Sat, 30 Apr 2016 20:56:06 +0300 Subject: [pypy-dev] 5.1 has been released In-Reply-To: References: <5718ED16.90903@gmail.com> <5719B0A8.3020208@gmail.com> Message-ID: <5724F1B6.5090904@gmail.com> Hi Phyo. I am not sure, since I'm so deep in, it's hard to say what level of expertise is required. I would imagine one would have to know how to write snippets of code in the C-API of python, and certainly would have to be familiar with the PyPY development process. A familiarity with a beloved (to them) C-API based module would probably also help, so they can say when it is working properly. Maybe some others have an opinion? Matti On 22/04/16 15:17, Phyo Arkar wrote: > I would like to help , i can get Two Interns who love python a lot . > But cpyext needs C skills , right? > > On Fri, Apr 22, 2016 at 11:34 AM Matti Picus > wrote: > > On 22/04/16 07:02, Phyo Arkar wrote: > > Congrats PyPy Team! > > It keeps getting better and better! > > And thank you very much for improving cpyext. > > > > > Fix edge cases in the cpyext refcounting-compatible semantics > (more > > work on cpyext compatibility is coming in the cpyext-ext branch, but > > isn?t ready yet) > > > > So thats mean , soon, infuture , cpyext will be fully supported? > > > For some definition of 'infuture' and 'fully' - yes. > The larger goal is to enable the scientific python stack with only > minor > upstream changes, IMO cpyext-ext will be finished when numpy's test > suite passes with no failures on the fork of numpy available on github > at github.com/pypy/numpy (not to be > confused with the one at > bitbucket/pypy/numpy, which uses micronumpy and cffi) > > Eventually, we will modify micronumpy so that app-level python calls > with ndarrays will somehow be hijacked to go through micronumpy rather > than the C-API functions, assuming this is even possible. > > It is a big job, help is welcome > Matti > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev >