From barry at barrys-emacs.org  Tue Oct  2 14:52:37 2018
From: barry at barrys-emacs.org (Barry Scott)
Date: Tue, 2 Oct 2018 19:52:37 +0100
Subject: [pypy-dev] PyCXX and PyPy status
Message-ID: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>

I'm the maintainer of the PyCXX C++ extension interface to Python C API.

PyCXX source code is a SVN repo at: https://svn.code.sf.net/p/cxx/code/trunk/CXX

I have been working to get PyCXX to build and run against PyPy C API.

I am happy to have PyPy specific code in PyCXX where necessary.

Using PyPy 0.6.0 on macOS I managed to build a .so that crashed when imported.

The reason is that the PyPy's C API does not allow tuples to be created.
The pattern in CPython is like this:

explicit Tuple (const Sequence& s)
        {
            sequence_index_type limit( sequence_index_type( s.length() ) );

            set(PyTuple_New (limit), true);
            validate();

            for(sequence_index_type i=0; i < limit; i++)
            {
                if(PyTuple_SetItem (ptr(), i, new_reference_to(s[i])) == -1)
                {
                    ifPyErrorThrowCxxException();
                }
            }
        }

The PyTuple_SetItem() always fails. Its a must have to create tuples.
How do you expect a tuple to be created?

I have a build of the tip from the PyPy HG repo. This has regressed from the 0.6.0 release:

g++ -c -g -Wall -fPIC -fexceptions -frtti -I. -ISrc -I/home/barry/tmpdir/usession-default-barry/build/pypy-qqq/include -DNDEBUG  -oobj/simple.obj  Demo/Python2/simple.cxx
In file included from ./CXX/Objects.hxx:40,
                 from Demo/Python2/simple.cxx:15:
./CXX/Python2/Objects.hxx: In member function ?void Py::MapBase<T>::delItem(const string&)?:
./CXX/Python2/Objects.hxx:3041:17: error: there are no arguments to ?PyMapping_DelItemString? that depend on a template parameter, so a declaration of ?PyMapping_DelItemString? must be available [-fpermissive]
             if (PyMapping_DelItemString (ptr(), const_cast<char*>(s.c_str())) == -1)
                 ^~~~~~~~~~~~~~~~~~~~~~~
./CXX/Python2/Objects.hxx:3041:17: note: (if you use ?-fpermissive?, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
./CXX/Python2/Objects.hxx: In member function ?void Py::MapBase<T>::delItem(const Py::Object&)?:
./CXX/Python2/Objects.hxx:3049:17: error: there are no arguments to ?PyMapping_DelItem? that depend on a template parameter, so a declaration of ?PyMapping_DelItem? must be available [-fpermissive]
             if (PyMapping_DelItem (ptr(), *s) == -1)
                 ^~~~~~~~~~~~~~~~~

It seems that PyMapping_DelItem and PyMapping_DelString have been removed since 0.6.0.
Can they be restored?

Oh and where does the magic "44" come from that I had to use in the <extension>.pypy-44.so file name?

Barry


From armin.rigo at gmail.com  Tue Oct  2 16:55:10 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Tue, 2 Oct 2018 22:55:10 +0200
Subject: [pypy-dev] PyCXX and PyPy status
In-Reply-To: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
References: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
Message-ID: <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>

Hi Barry,

On Tue, 2 Oct 2018 at 21:09, Barry Scott <barry at barrys-emacs.org> wrote:
> Using PyPy 0.6.0 on macOS I managed to build a .so that crashed when imported.

I will assume that you mean PyPy2 6.0.0, and not PyPy3 nor the version 0.6.0.

> The reason is that the PyPy's C API does not allow tuples to be created.

You cannot change tuple objects after they "escaped" to become general
PyPy objects.  You can only call PyTuple_SetItem() on a tuple object
that exists as a "PyObject *" but not yet as a PyPy object.  That
means, mostly, you should only call PyTuple_SetItem() on the fresh
result of PyTuple_New() before the "PyObject *" is sent somewhere
else.  In your example code, maybe the set() function makes the tuple
object escape in this way.  Note that escaping tuples before they are
fully initialized can also sometimes create potential crashes on
CPython.

>              if (PyMapping_DelItemString (ptr(), const_cast<char*>(s.c_str())) == -1)
>              if (PyMapping_DelItem (ptr(), *s) == -1)

I don't know how it ever worked, because PyMapping_DelItem and
PyMapping_DelItemString are not implemented, neither right now nor at
the time of release 6.0.0.

If you are using these two functions, please open an issue
(https://bitbucket.org/pypy/pypy/issues?status=new&status=open) and
we'll implement them.


A bient?t,

Armin.

From ronan.lamy at gmail.com  Wed Oct  3 15:30:38 2018
From: ronan.lamy at gmail.com (Ronan Lamy)
Date: Wed, 3 Oct 2018 20:30:38 +0100
Subject: [pypy-dev] PyCXX and PyPy status
In-Reply-To: <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>
References: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
 <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>
Message-ID: <2da701ad-0224-3bf5-ebfe-aadac3cd49c9@gmail.com>

Le 02/10/18 ? 21:55, Armin Rigo a ?crit?:

>>               if (PyMapping_DelItemString (ptr(), const_cast<char*>(s.c_str())) == -1)
>>               if (PyMapping_DelItem (ptr(), *s) == -1)
> 
> I don't know how it ever worked, because PyMapping_DelItem and
> PyMapping_DelItemString are not implemented, neither right now nor at
> the time of release 6.0.0.
> 
> If you are using these two functions, please open an issue
> (https://bitbucket.org/pypy/pypy/issues?status=new&status=open) and
> we'll implement them.

I just implemented these in 
https://bitbucket.org/pypy/pypy/commits/a5735c0b1edd6e5e16dc7015facf2131fed6188d

From barry at barrys-emacs.org  Sun Oct  7 12:05:25 2018
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 7 Oct 2018 17:05:25 +0100
Subject: [pypy-dev] PyCXX and PyPy status
In-Reply-To: <2da701ad-0224-3bf5-ebfe-aadac3cd49c9@gmail.com>
References: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
 <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>
 <2da701ad-0224-3bf5-ebfe-aadac3cd49c9@gmail.com>
Message-ID: <1D954F6A-B503-4AD4-921A-D0D02F71BBC9@barrys-emacs.org>



> On 3 Oct 2018, at 20:30, Ronan Lamy <ronan.lamy at gmail.com> wrote:
> 
> Le 02/10/18 ? 21:55, Armin Rigo a ?crit :
> 
>>>              if (PyMapping_DelItemString (ptr(), const_cast<char*>(s.c_str())) == -1)
>>>              if (PyMapping_DelItem (ptr(), *s) == -1)
>> I don't know how it ever worked, because PyMapping_DelItem and
>> PyMapping_DelItemString are not implemented, neither right now nor at
>> the time of release 6.0.0.
>> If you are using these two functions, please open an issue
>> (https://bitbucket.org/pypy/pypy/issues?status=new&status=open) and
>> we'll implement them.
> 
> I just implemented these in https://bitbucket.org/pypy/pypy/commits/a5735c0b1edd6e5e16dc7015facf2131fed6188d <https://bitbucket.org/pypy/pypy/commits/a5735c0b1edd6e5e16dc7015facf2131fed6188d>


Thanks I'm compiling again.

I see that the .so will not be found unless I use the .pypy-41.so suffix.
Where does the 41 come from?

I'm getting further. I have changed the details of how I create tuples
to make progress. I will go back and figure out the impact on my users later.

I'm now getting a lot further.

Module has completed init.
Old style classes seem to be working.
But new style classes are not working.

When I make a new style class its type has the following tp_flags:

(gdb) p/x table->tp_flags
$4 = 0x201eb

But when the instance comes back to me they are:

(gdb) p/x self->ob_type->tp_flags
$11 = 0x1208

Surely the flags should not have been changed?

Since Py_TPFLAGS_BASETYPE has gone from 1 to 0.
I crash because I assume the wrong type of class and access a pointer
that is not there.

Thoughts?

Barry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181007/6974e7e5/attachment.html>

From armin.rigo at gmail.com  Mon Oct  8 17:40:20 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Mon, 8 Oct 2018 23:40:20 +0200
Subject: [pypy-dev] PyCXX and PyPy status
In-Reply-To: <1D954F6A-B503-4AD4-921A-D0D02F71BBC9@barrys-emacs.org>
References: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
 <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>
 <2da701ad-0224-3bf5-ebfe-aadac3cd49c9@gmail.com>
 <1D954F6A-B503-4AD4-921A-D0D02F71BBC9@barrys-emacs.org>
Message-ID: <CAMSv6X1CnxbcUsLFJrGbxFyQQBGESmJUf9bWqc9LpLuqyCz3-A@mail.gmail.com>

Hi,

On Sun, 7 Oct 2018 at 18:05, Barry Scott <barry at barrys-emacs.org> wrote:
> (gdb) p/x table->tp_flags
> $4 = 0x201eb
>
> But when the instance comes back to me they are:
>
> (gdb) p/x self->ob_type->tp_flags
> $11 = 0x1208
>
> Surely the flags should not have been changed?

Some flags change in CPython too.  The change you're seeing might be
correct, or it might not be---although I agree that the bits in the
two values are very different.  But it's hard for us to know at this
point without knowing what you are doing exactly.  You may be using
only the standard parts of the CPython C API, or instead be using
completely internal details.  Can you try to extract a small CPython
extension module as C code, containing the usual kind of PyTypeObject
definitions and calls to some PyXxx() functions, and which behaves
differently on CPython and on PyPy?  That would help us understand the
problem.


A bient?t,

Armin.

From Markus.Wiener at rwth-aachen.de  Fri Oct 12 10:52:25 2018
From: Markus.Wiener at rwth-aachen.de (Wiener, Markus)
Date: Fri, 12 Oct 2018 14:52:25 +0000
Subject: [pypy-dev] Difference to CPython (Maybe a bug in PyPy)
Message-ID: <4d68b5eb832241c2a9983a077b1c1e28@rwth-aachen.de>

Hello,


i found (maybe) a bug in PyPy.


Try:


> python python_vs_pypy.py


and


> pypy python_vs_pypy.py


Regards,

Markus Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181012/df03d869/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: python_vs_pypy.py
Type: text/x-python
Size: 614 bytes
Desc: python_vs_pypy.py
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181012/df03d869/attachment.py>

From armin.rigo at gmail.com  Fri Oct 12 12:28:19 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Fri, 12 Oct 2018 18:28:19 +0200
Subject: [pypy-dev] Difference to CPython (Maybe a bug in PyPy)
In-Reply-To: <4d68b5eb832241c2a9983a077b1c1e28@rwth-aachen.de>
References: <4d68b5eb832241c2a9983a077b1c1e28@rwth-aachen.de>
Message-ID: <CAMSv6X0RPauP4-S6BwSzuk2rWTHhn-9Tcsoaz3BUs46Gm_dhfg@mail.gmail.com>

Hi,

On Fri, 12 Oct 2018 at 17:00, Wiener, Markus
<Markus.Wiener at rwth-aachen.de> wrote:
> i found (maybe) a bug in PyPy.

Your code is relying on an explicitly undefined behavior described for
dicts in https://docs.python.org/3/library/stdtypes.html#dictionary-view-objects
.  The same applies to sets, although I'm not sure it's as clearly
spelled out as for dicts.  Specifically, you cannot mutate the set
``a`` in the middle of a loop ``for j in a``.

In fact, I think that if your code was written using dicts instead of
sets, then its behavior would change in CPython 3.6 because dicts are
ordered there.  Sets are not---as far as I know (and I guess I should
say "not yet"), so CPython 3.6 didn't change for your exact code.  But
PyPy's sets are ordered, and gives different results.


A bient?t,

Armin.

From norbert.raimund.leisner at arcor.de  Fri Oct 19 06:14:18 2018
From: norbert.raimund.leisner at arcor.de (norbert.raimund.leisner at arcor.de)
Date: Fri, 19 Oct 2018 12:14:18 +0200 (CEST)
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
Message-ID: <1535465547.83558.1539944058508@mail.vodafone.de>

Hello support-team,

it is a computerchess-related topic e.g. running Python 0.99.3 https://github.com/pychess/pychess/releases/download/0.99.3/pychess-engine-20180928.pyz outsite its own graphical user interface https://github.com/pychess/pychess/releases/download/0.99.3/pychess-0.99.3-win32.msi on an external front-end like WinBoard Interface http://www.open-aurec.com/wbforum/viewtopic.php?f=19&t=51528 http://www.open-aurec.com/wbforum/viewtopic.php?f=19&t=51528  with protocol CECP http://hgm.nubati.net/CECP.html 

A detailed description (with all necessary configurations) how to accomplish this problem is hosted on my Google Drive chess collection https://drive.google.com/drive/folders/0B5Ao6h_yMCl6ay04RjVtUDVTUG8 > folder Pychess_20180928

G?nther Simon from Germany http://www.rwbc-chess.de/ http://www.rwbc-chess.de/   reported that PyPy for Windows caused problems to run PyChess engine in contrast to WinPython or Python for Windows 3.

Best wishes,

Norbert


PyChess 0.99.3 - WinBoard GUI


* Download WinPython 3.6.30Qt5 x64 (390 MB) https://github.com/winpython/winpython/releases/download/1.9.20171031/WinPython-64bit-3.6.3.0Qt5.exe https://github.com/winpython/winpython/releases/download/1.9.20171031/WinPython-64bit-3.6.3.0Qt5.exe and install it on a drive e.g. C:\

This procedure can take more than 30 minutes.


* Create a directory on C:\Engines\WB\PyChess_20180928 with the following working components:

> InBetween (configuration settings)

> PyChess_20180928 Editor (text document)

> PyChess_20180928 = renamed InBetween http://web.archive.org/web/20100922051419/http://home.online.no/~malin/sjakk/download/InBetween.zip http://web.archive.org/web/20100922051419/http://home.online.no/~malin/sjakk/download/InBetween.zip

> PyChess Zip application


* Run WinBoard GUI > Load first engine > PyChess_20180928.exe (without any commandline-parameters) under protocol WB-2


It will not run with PyPy https://pypy.org/download.html https://pypy.org/download.html on Windows, because there are only 32-bit PyPy installs for Windows available, while at least latest pychess.pyz seems to rely on some 64-bit Python stuff, which throws some known exceptions (according to G?nther?s Google research) in this case.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181019/b555f35c/attachment.html>

From armin.rigo at gmail.com  Fri Oct 19 06:26:36 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Fri, 19 Oct 2018 12:26:36 +0200
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <1535465547.83558.1539944058508@mail.vodafone.de>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
Message-ID: <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>

Hi,

On Fri, 19 Oct 2018 at 12:23, <norbert.raimund.leisner at arcor.de> wrote:
> It will not run with PyPy https://pypy.org/download.html on Windows, because there are only 32-bit PyPy installs for Windows available

PyPy is not available for 64-bit Windows.  See
http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation
for technical information.  It's not high priority for us, which means
we won't do it unless we're getting a sizeable grant or contract.


A bient?t,

Armin.

From barry at barrys-emacs.org  Sun Oct 21 10:47:44 2018
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 21 Oct 2018 15:47:44 +0100
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
Message-ID: <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>


> On 19 Oct 2018, at 11:26, Armin Rigo <armin.rigo at gmail.com> wrote:
> 
> Hi,
> 
> On Fri, 19 Oct 2018 at 12:23, <norbert.raimund.leisner at arcor.de> wrote:
>> It will not run with PyPy https://pypy.org/download.html on Windows, because there are only 32-bit PyPy installs for Windows available
> 
> PyPy is not available for 64-bit Windows.  See
> http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation
> for technical information.  It's not high priority for us, which means
> we won't do it unless we're getting a sizeable grant or contract.

How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.

Barry


From armin.rigo at gmail.com  Sun Oct 21 14:04:20 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sun, 21 Oct 2018 20:04:20 +0200
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
Message-ID: <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>

Hi,

On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
> > On 19 Oct 2018, at 11:26, Armin Rigo <armin.rigo at gmail.com> wrote:
> > PyPy is not available for 64-bit Windows.
>
> How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.

That's because MacOS and Fedora are not Windows.


Armin

From barry at barrys-emacs.org  Mon Oct 22 17:13:39 2018
From: barry at barrys-emacs.org (Barry)
Date: Mon, 22 Oct 2018 22:13:39 +0100
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
Message-ID: <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>



> On 21 Oct 2018, at 19:04, Armin Rigo <armin.rigo at gmail.com> wrote:
> 
> Hi,
> 
> On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
>>> On 19 Oct 2018, at 11:26, Armin Rigo <armin.rigo at gmail.com> wrote:
>>> PyPy is not available for 64-bit Windows.
>> 
>> How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.
> 
> That's because MacOS and Fedora are not Windows.

Do you why windows is unique in this respect?

Barry

> 
> Armin
> 

From aidanlakshman at gmail.com  Mon Oct 22 18:44:54 2018
From: aidanlakshman at gmail.com (aidanlakshman at gmail.com)
Date: Mon, 22 Oct 2018 18:44:54 -0400
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
 <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
Message-ID: <898CF9E9-D66A-42ED-B941-67B67FD3B897@gmail.com>

Probably because 32-bit is still supported by Windows, whereas Fedora discontinued supporting 32 bit in 2016, and OSX stopped supporting 32 bit recently as well. Easier to have just one version of Windows rather than supporting two versions, and 32-bit is compatible with all Windows, whereas 64-bit is not. Not sure if that?s the actual reason, but it seems to be the most likely one to me. 

If you?re asking why Windows still supports 32-bit, I can?t really help you there. 


-----------------
Aidan 

> On Oct 22, 2018, at 17:13, Barry <barry at barrys-emacs.org> wrote:
> 
> 
> 
>> On 21 Oct 2018, at 19:04, Armin Rigo <armin.rigo at gmail.com> wrote:
>> 
>> Hi,
>> 
>> On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
>>>> On 19 Oct 2018, at 11:26, Armin Rigo <armin.rigo at gmail.com> wrote:
>>>> PyPy is not available for 64-bit Windows.
>>> 
>>> How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.
>> 
>> That's because MacOS and Fedora are not Windows.
> 
> Do you why windows is unique in this respect?
> 
> Barry
> 
>> 
>> Armin
>> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

From matti.picus at gmail.com  Tue Oct 23 00:26:31 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Tue, 23 Oct 2018 07:26:31 +0300
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
 <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
Message-ID: <d3acc22a-3c49-d300-1778-56a50abb3239@gmail.com>


On 23/10/18 12:13 am, Barry wrote:
>
>> On 21 Oct 2018, at 19:04, Armin Rigo <armin.rigo at gmail.com> wrote:
>>
>> Hi,
>>
>> On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
>>>> On 19 Oct 2018, at 11:26, Armin Rigo <armin.rigo at gmail.com> wrote:
>>>> PyPy is not available for 64-bit Windows.
>>> How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.
>> That's because MacOS and Fedora are not Windows.
> Do you why windows is unique in this respect?
>
> Barry

As Armin said previously, PyPy supporting win64 is work, since 
sizeof(long) != sizeof(void*)

Seehttp://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation
for technical information.  It's not high priority for us, which means
we won't do it unless we're getting a sizeable grant or contract.



From barry at barrys-emacs.org  Tue Oct 23 03:40:06 2018
From: barry at barrys-emacs.org (Barry)
Date: Tue, 23 Oct 2018 08:40:06 +0100
Subject: [pypy-dev] Fwd: [#3588935] Re: PyPy 3 for Windows 64-bit systems -
 no download link available
References: <83d9ddd86c6fc6bac859a110fbf3893b@ubersmith>
Message-ID: <86AED470-9522-41E5-9AA7-942AB6E7443C@barrys-emacs.org>

Each time i email pypy i get a email from this hosting company.

Can you unsubscribe them?

Barry


Begin forwarded message:

> From: support-level1 at contact4.westhost.com
> Date: 22 October 2018 at 22:37:31 BST
> To: barry at barrys-emacs.org
> Subject: [#3588935] Re: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link available
> Reply-To: support-level1 at contact4.westhost.com
> 
> Hello,
> 
> Thank you for contacting the WestHost Support Department. This message has been automatically generated in response to your e-mail regarding: "Re: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link available". Your request has been assigned a ticket number of [westhost.com #3588935].  A representative will be contacting you shortly.
> 
> Please include this text: [westhost.com #3588935] in any future e-mails about this issue. You can also provide additional information on your request by simply replying to this message.
> 
> WestHost Forums
> -------------------------------------------
> While waiting for a response on your ticket, please take a look through our knowledge base at
> http://kb.westhost.com/. We are continually improving it with a variety of walkthroughs and common resolutions!
> 
> Although not an official support method, our WestHost Forums (http://forums.westhost.com/) remain a great resource for getting help from other WestHost clients. The Forums also provide a great medium for discussion on web hosting.
> 
> We appreciate your patience while we respond to your e-mail ticket.  Thank you for choosing WestHost.
> 
> Best regards,
> 
> WestHost Support Department
> 
> When you expect more from your Web host
> http://www.westhost.com/
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181023/c6556f96/attachment.html>

From janzert at janzert.com  Tue Oct 23 10:44:40 2018
From: janzert at janzert.com (Janzert)
Date: Tue, 23 Oct 2018 10:44:40 -0400
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
 <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
Message-ID: <pqnc0i$151$1@blaine.gmane.org>

On 10/22/2018 17:13, Barry wrote:
> 
> 
>> On 21 Oct 2018, at 19:04, Armin Rigo <armin.rigo at gmail.com> wrote:
>>
>> Hi,
>>
>> On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
>>>> On 19 Oct 2018, at 11:26, Armin Rigo <armin.rigo at gmail.com> wrote:
>>>> PyPy is not available for 64-bit Windows.
>>>
>>> How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.
>>
>> That's because MacOS and Fedora are not Windows.
> 
> Do you why windows is unique in this respect?
> 
> Barry
> 

 From what I understand the underlying cause is because 64bit windows 
(or more specifically the visual studio C compiler) uses an LLP64 
(IL32P64) data model* while most 64bit unix based systems use LP64 
(I32LP64).

The previously linked article on getting a 64bit windows pypy talks 
about the size difference for long but doesn't spell out the data model 
explicitly.

Hopefully that helps clear up where the underlying issue comes from.

Brian Haskin

* https://en.wikipedia.org/wiki/64-bit_computing#64-bit_data_models


From ivanstepanovftw at gmail.com  Wed Oct 24 15:46:10 2018
From: ivanstepanovftw at gmail.com (Ivan Stepanov)
Date: Wed, 24 Oct 2018 22:46:10 +0300
Subject: [pypy-dev] cppyy_backend logging issue
Message-ID: <CANmEVT6PbD_XJ8xOV+Eg-DAj3H48av6Lh-yNVshUv_+nN5_5wA@mail.gmail.com>

cppyy doesn't work propertly.

Traceback (most recent call last):
  File
"/usr/lib/python3.7/site-packages/cppyy_backend/_cppyy_generator.py", line
692, in main
    mapping = g.create_mapping(args.sources)
  File
"/usr/lib/python3.7/site-packages/cppyy_backend/_cppyy_generator.py", line
214, in create_mapping
    file_info = self.create_file_mapping(h_file)
  File
"/usr/lib/python3.7/site-packages/cppyy_backend/_cppyy_generator.py", line
244, in create_file_mapping
    logger.log(m * diag.severity + c, "While parsing: {}".format(msg))
  File "/usr/lib/python3.7/logging/__init__.py", line 1395, in log
    raise TypeError("level must be an integer")
TypeError: level must be an integer

System info:
Manjaro (Arch linux)
Python 3.7.0
CPyCppyy 1.3.6
cppyy 1.2.4
cppyy-backend 1.4.2
cppyy-cling 6.14.2.1

Possible solution:
in file cppyy_backend/_cppyy_generator.py:
-    logger.log(m * diag.severity + c, "While parsing: {}".format(msg))
+    logger.log(int(m * diag.severity + c), "While parsing: {}".format(msg))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181024/558ce34a/attachment.html>

From matti.picus at gmail.com  Wed Oct 24 16:51:29 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 24 Oct 2018 23:51:29 +0300
Subject: [pypy-dev] cppyy_backend logging issue
In-Reply-To: <CANmEVT6PbD_XJ8xOV+Eg-DAj3H48av6Lh-yNVshUv_+nN5_5wA@mail.gmail.com>
References: <CANmEVT6PbD_XJ8xOV+Eg-DAj3H48av6Lh-yNVshUv_+nN5_5wA@mail.gmail.com>
Message-ID: <3d5276da-c5b6-9534-4a29-9c8d86fd6503@gmail.com>


On 24/10/18 10:46 pm, Ivan Stepanov wrote:
> cppyy doesn't work propertly.
>
> Traceback (most recent call last):
> ? File 
> "/usr/lib/python3.7/site-packages/cppyy_backend/_cppyy_generator.py", 
> line 692, in main
> ? ? mapping = g.create_mapping(args.sources)
> ? File 
> "/usr/lib/python3.7/site-packages/cppyy_backend/_cppyy_generator.py", 
> line 214, in create_mapping
> ? ? file_info = self.create_file_mapping(h_file)
> ? File 
> "/usr/lib/python3.7/site-packages/cppyy_backend/_cppyy_generator.py", 
> line 244, in create_file_mapping
> ? ? logger.log(m * diag.severity + c, "While parsing: {}".format(msg))
> ? File "/usr/lib/python3.7/logging/__init__.py", line 1395, in log
> ? ? raise TypeError("level must be an integer")
> TypeError: level must be an integer
>
> System info:
> Manjaro (Arch linux)
> Python 3.7.0
> CPyCppyy 1.3.6
> cppyy 1.2.4
> cppyy-backend 1.4.2
> cppyy-cling 6.14.2.1
>
> Possible solution:
> in file cppyy_backend/_cppyy_generator.py:
> - logger.log(m * diag.severity + c, "While parsing: {}".format(msg))
> +? ? logger.log(int(m * diag.severity + c), "While parsing: 
> {}".format(msg))
>

Does this fix it, in the same file cppyy_backend/_cppyy_generator.py:

- m= (logging.ERROR- logging.WARNING)/(Diagnostic.Error- Diagnostic.Warning)
+ m= (logging.ERROR- logging.WARNING) // (Diagnostic.Error- 
Diagnostic.Warning)
Matti

From wlavrijsen at lbl.gov  Wed Oct 24 17:07:59 2018
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 24 Oct 2018 14:07:59 -0700 (PDT)
Subject: [pypy-dev] cppyy_backend logging issue
In-Reply-To: <3d5276da-c5b6-9534-4a29-9c8d86fd6503@gmail.com>
References: <CANmEVT6PbD_XJ8xOV+Eg-DAj3H48av6Lh-yNVshUv_+nN5_5wA@mail.gmail.com>
 <3d5276da-c5b6-9534-4a29-9c8d86fd6503@gmail.com>
Message-ID: <alpine.LSU.2.21.1810241406560.1997@pc3-wlav.lbl.gov>

Matti,

thanks for the pull request. :)

I'm still bogged down on get the CPython side of things running on Windows.
Will have more cycles available once I get that out of my hair.

Cheers,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From armin.rigo at gmail.com  Wed Oct 24 17:55:22 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Wed, 24 Oct 2018 23:55:22 +0200
Subject: [pypy-dev] Fwd: [#3588935] Re: PyPy 3 for Windows 64-bit
 systems - no download link available
In-Reply-To: <86AED470-9522-41E5-9AA7-942AB6E7443C@barrys-emacs.org>
References: <83d9ddd86c6fc6bac859a110fbf3893b@ubersmith>
 <86AED470-9522-41E5-9AA7-942AB6E7443C@barrys-emacs.org>
Message-ID: <CAMSv6X15mwor6K8P6DuFKS01JfUcK7cat2HtKUocAgYQPFcJsg@mail.gmail.com>

Hi,

On Tue, 23 Oct 2018 at 10:22, Barry <barry at barrys-emacs.org> wrote:
> Each time i email pypy i get a email from this hosting company.
>
> Can you unsubscribe them?

It's not so easy because we have no direct clue about which e-mail
address ends up on that random issue tracker.  But I found it anyway
and removed the offender.


A bient?t,

Armin.

From Martin.Gfeller at swisscom.com  Thu Oct 25 07:43:05 2018
From: Martin.Gfeller at swisscom.com (Martin.Gfeller at swisscom.com)
Date: Thu, 25 Oct 2018 11:43:05 +0000
Subject: [pypy-dev] Fwd: [#3588935] Re: PyPy 3 for Windows 64-bit systems
Message-ID: <01ab486afc734b338926361c7075547f@SG001745.corproot.net>

Dear Brian

The Wikipedia article seems to imply that using Cygwin as a compiler on Windows 64 would resolve the problem? 

Dear Armin, it can't be possibly that easy?

Best regards
Martin



> Date: Tue, 23 Oct 2018 10:44:40 -0400
> From: Janzert <janzert at janzert.com>
> To: pypy-dev at python.org
> Subject: Re: [pypy-dev] PyPy 3 for Windows 64-bit systems - no
	download link available
> Message-ID: <pqnc0i$151$1 at blaine.gmane.org>

> From what I understand the underlying cause is because 64bit windows (or more specifically the visual studio C compiler) uses an LLP64
> (IL32P64) data model* while most 64bit unix based systems use LP64 (I32LP64).

>  The previously linked article on getting a 64bit windows pypy talks about the size difference for long but doesn't spell out the data model explicitly.

> Hopefully that helps clear up where the underlying issue comes from.

> Brian Haskin

> * https://en.wikipedia.org/wiki/64-bit_computing#64-bit_data_models


From armin.rigo at gmail.com  Thu Oct 25 10:53:20 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Thu, 25 Oct 2018 16:53:20 +0200
Subject: [pypy-dev] Fwd: [#3588935] Re: PyPy 3 for Windows 64-bit systems
In-Reply-To: <01ab486afc734b338926361c7075547f@SG001745.corproot.net>
References: <01ab486afc734b338926361c7075547f@SG001745.corproot.net>
Message-ID: <CAMSv6X2fWYQke+6NuTmMBuxAtOUfder5SpT7E4JVB8x01hs5tQ@mail.gmail.com>

Hi,

On Thu, 25 Oct 2018 at 16:48, <Martin.Gfeller at swisscom.com> wrote:
> The Wikipedia article seems to imply that using Cygwin as a compiler on Windows 64 would resolve the problem?

It might, but then you would have to compile all the CPython C
extension modules using Cygwin as well, which is (1) unexpected and
(2) likely to break some
of them not expecting a different 64-bit model.  In addition, I have
no clue about which calling convention Cygwin-on-64bit-Windows is
using, which is important for our JIT.  So no, that's unlikely to be
the right approach to 64-bit Windows.


A bient?t,

Armin.

From barry at barrys-emacs.org  Sat Oct 27 10:22:47 2018
From: barry at barrys-emacs.org (Barry Scott)
Date: Sat, 27 Oct 2018 15:22:47 +0100
Subject: [pypy-dev] PyCXX and PyPy status
In-Reply-To: <CAMSv6X1CnxbcUsLFJrGbxFyQQBGESmJUf9bWqc9LpLuqyCz3-A@mail.gmail.com>
References: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
 <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>
 <2da701ad-0224-3bf5-ebfe-aadac3cd49c9@gmail.com>
 <1D954F6A-B503-4AD4-921A-D0D02F71BBC9@barrys-emacs.org>
 <CAMSv6X1CnxbcUsLFJrGbxFyQQBGESmJUf9bWqc9LpLuqyCz3-A@mail.gmail.com>
Message-ID: <D4103C77-82AE-4B42-B89F-44541EFEF6C5@barrys-emacs.org>



> On 8 Oct 2018, at 22:40, Armin Rigo <armin.rigo at gmail.com> wrote:
> 
> Hi,
> 
> On Sun, 7 Oct 2018 at 18:05, Barry Scott <barry at barrys-emacs.org> wrote:
>> (gdb) p/x table->tp_flags
>> $4 = 0x201eb
>> 
>> But when the instance comes back to me they are:
>> 
>> (gdb) p/x self->ob_type->tp_flags
>> $11 = 0x1208
>> 
>> Surely the flags should not have been changed?
> 
> Some flags change in CPython too.  The change you're seeing might be
> correct, or it might not be---although I agree that the bits in the
> two values are very different.  But it's hard for us to know at this
> point without knowing what you are doing exactly.  You may be using
> only the standard parts of the CPython C API, or instead be using
> completely internal details.

To my knowledge I'm not using internal APIs.

>  Can you try to extract a small CPython
> extension module as C code, containing the usual kind of PyTypeObject
> definitions and calls to some PyXxx() functions, and which behaves
> differently on CPython and on PyPy?  That would help us understand the
> problem.

Where is your test code for the this feature? I could review/modify that
to help find a difference that may explain the problem.

If you have no test coverage for new-style-classes I would rather not write that code for you.

Instead I can document how to build the PyCXX test code to run against PyPy for you.
It is not difficult.

Barry





> 
> 
> A bient?t,
> 
> Armin.
> 


From armin.rigo at gmail.com  Sat Oct 27 11:28:32 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Sat, 27 Oct 2018 17:28:32 +0200
Subject: [pypy-dev] PyCXX and PyPy status
In-Reply-To: <D4103C77-82AE-4B42-B89F-44541EFEF6C5@barrys-emacs.org>
References: <EFD50189-9FB4-495F-A910-F8272DF5304E@barrys-emacs.org>
 <CAMSv6X2qAMfbNJBSxXQ5pP6kba3DJpXfSrFqQwciDdZyc_ayEQ@mail.gmail.com>
 <2da701ad-0224-3bf5-ebfe-aadac3cd49c9@gmail.com>
 <1D954F6A-B503-4AD4-921A-D0D02F71BBC9@barrys-emacs.org>
 <CAMSv6X1CnxbcUsLFJrGbxFyQQBGESmJUf9bWqc9LpLuqyCz3-A@mail.gmail.com>
 <D4103C77-82AE-4B42-B89F-44541EFEF6C5@barrys-emacs.org>
Message-ID: <CAMSv6X2PYR54gjWAXBCHCvFGDMcKAZ5kNX7fbO4FUfOPXUBJNQ@mail.gmail.com>

Hi Barry,

On Sat, 27 Oct 2018 at 16:22, Barry Scott <barry at barrys-emacs.org> wrote:
> Where is your test code for the this feature? I could review/modify that
> to help find a difference that may explain the problem.

You are saying that your use case fails, but we don't know which part
of cpyext is causing the failure.  I can point to the whole tests of
cpyext (they are in pypy/module/cpyext/test/; new-style classes tests
are in test_typeobject).  But that seems not useful.

What I'm asking for is any way for us to reproduce the problem.  It
can be step-by-step instructions about building PyCXX (like you
suggested); or it can be a smaller .c use case, which might help
to motivate us to look at it.  It doesn't have to be specially prepared
or integrated with the cpyext tests---we first need to figure out what
the real problem is, before we can write a relevant unit test.


A bient?t,

Armin.

From wlavrijsen at lbl.gov  Sat Oct 27 14:55:11 2018
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Sat, 27 Oct 2018 11:55:11 -0700 (PDT)
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
 <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
Message-ID: <alpine.LSU.2.21.1810271149230.18145@pc3-wlav.lbl.gov>

Hi,

On Mon, 22 Oct 2018, Barry wrote:
>> On 21 Oct 2018, at 19:04, Armin Rigo <armin.rigo at gmail.com> wrote:
>> On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
>>> How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.
>> That's because MacOS and Fedora are not Windows.
> Do you why windows is unique in this respect?

as I'm struggling with Windows at the moment, I may have an answer ...

I find that for CPython3, sys.maxsize is (2**31)-1 on 32-bits Python and
(2**63)-1 for 64-bits Python builds. With sys.maxint being a Python2 thing,
it may be different there, but it seems 3 at least is perfectly consistent.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From bokr at bokr.com  Sun Oct 28 06:04:44 2018
From: bokr at bokr.com (Bengt Richter)
Date: Sun, 28 Oct 2018 11:04:44 +0100
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <alpine.LSU.2.21.1810271149230.18145@pc3-wlav.lbl.gov>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
 <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
 <alpine.LSU.2.21.1810271149230.18145@pc3-wlav.lbl.gov>
Message-ID: <20181028100444.GA2700@hd2-boar>

On +2018-10-27 11:55:11 -0700, wlavrijsen at lbl.gov wrote:
> Hi,
> 
> On Mon, 22 Oct 2018, Barry wrote:
> > > On 21 Oct 2018, at 19:04, Armin Rigo <armin.rigo at gmail.com> wrote:
> > > On Sun, 21 Oct 2018 at 16:47, Barry Scott <barry at barrys-emacs.org> wrote:
> > > > How odd. sys.maxint on macOS and Fedora is (2**63)-1 not (2**31)-1.
> > > That's because MacOS and Fedora are not Windows.
> > Do you why windows is unique in this respect?
> 
> as I'm struggling with Windows at the moment, I may have an answer ...
> 
> I find that for CPython3, sys.maxsize is (2**31)-1 on 32-bits Python and
> (2**63)-1 for 64-bits Python builds. With sys.maxint being a Python2 thing,
> it may be different there, but it seems 3 at least is perfectly consistent.
> 
> Best regards,
>            Wim
> -- 
> WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev

Does (2**63)-1 work because size is unsigned (presumably?)
and (2**63) is then valid as an intermediate value before subtracting 1,
whereas for a signed maxint, the intermediate value would have to
be specialcased or the value written maybe like (2**62-1)+(2**62) ?
Or does it involve a bigint representation somewhere?

Regards,
Bengt Richter

From matti.picus at gmail.com  Sun Oct 28 07:11:25 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Sun, 28 Oct 2018 13:11:25 +0200
Subject: [pypy-dev] PyPy 3 for Windows 64-bit systems - no download link
 available
In-Reply-To: <20181028100444.GA2700@hd2-boar>
References: <1535465547.83558.1539944058508@mail.vodafone.de>
 <CAMSv6X3QjLW0T8KhGN66XtCRx3hH9_EHqVTeTsc2RAduYtbgHg@mail.gmail.com>
 <19659D6F-42CB-4364-9B77-66C0256BF80F@barrys-emacs.org>
 <CAMSv6X2+k9xuJOzq5ZjHr-o_B6=+CQq9JDWCvT7MmXuvJB9u6Q@mail.gmail.com>
 <0AFABE9F-6CA1-4148-95A8-9410259A39ED@barrys-emacs.org>
 <alpine.LSU.2.21.1810271149230.18145@pc3-wlav.lbl.gov>
 <20181028100444.GA2700@hd2-boar>
Message-ID: <84b3d253-2af8-bbf3-bf31-474a1a5c1a57@gmail.com>

 > On +2018-10-27 11:55:11 -0700, wlavrijsen at lbl.gov wrote:
>> as I'm struggling with Windows at the moment, I may have an answer ...
>> I find that for CPython3, sys.maxsize is (2**31)-1 on 32-bits Python and
>> (2**63)-1 for 64-bits Python builds. With sys.maxint being a Python2 thing,
>> it may be different there, but it seems 3 at least is perfectly consistent.
>>
>> Best regards,
>>             Wim
>> -- 
>> WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
> Does (2**63)-1 work because size is unsigned (presumably?)
> and (2**63) is then valid as an intermediate value before subtracting 1,
> whereas for a signed maxint, the intermediate value would have to
> be specialcased or the value written maybe like (2**62-1)+(2**62) ?
> Or does it involve a bigint representation somewhere?
>
> Regards,
> Bengt Richter


The sys.maxint vs sys.maxsize difference is the first thing that appears 
at 
http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation

There is a nice plan there to transition to a win64 python2 (which we 
need since rpython is written in python2) where sys.maxint == 
sys.maxsize == 2**63-1, and to continue from there to a pypy2 that can 
translate rpython.

Matti



From cfbolz at gmx.de  Mon Oct 29 05:59:23 2018
From: cfbolz at gmx.de (Carl Friedrich Bolz-Tereick)
Date: Mon, 29 Oct 2018 10:59:23 +0100
Subject: [pypy-dev] =?utf-8?q?save_the_date=3A_PyPy_Sprint_in_D=C3=BCssel?=
 =?utf-8?q?dorf_February_4-9=2C_2019?=
Message-ID: <668b0c40-85aa-b55d-af96-ef846e573306@gmx.de>

Hi all,

I'll do a more proper sprint announcement soon, but to give some advance
warning, I am organizing a PyPy sprint at the Heinrich-Heine-Universit?t
D?sseldorf from February 4-9, 2019 (both days inclusive).

The last sprint we had there was in 2010 :-).

Cheers,

Carl Friedrich

From tbaldridge at gmail.com  Mon Oct 29 19:41:07 2018
From: tbaldridge at gmail.com (Timothy Baldridge)
Date: Mon, 29 Oct 2018 17:41:07 -0600
Subject: [pypy-dev] Best way to inline a primitive array in RPython?
Message-ID: <CAL36E+vcrnOSZtu9Btoun3F7dRyJnti7059R1rQdk5VF+dq0Hg@mail.gmail.com>

Looking at rbuffer, rgc, and the lltype namespaces, it's not completely
clear to me how I would go about writing a struct with a resizable
primitive array inlined.

Something like:

typedef struct foo_t {
int a, b;
int _data[0];
}

foo_t tmp = malloc(sizeof(foo_t) + 64);

I can see that the GC has the ability to allocate "extra" memory beyond
what's required for a struct, and that lltype has the ability to inline
some lists into a struct, but how does this work in practice? And maybe I'm
missing something in the docs, so simply pointing me to some docs is also
fine.


-- 
?One of the main causes of the fall of the Roman Empire was that?lacking
zero?they had no way to indicate successful termination of their C
programs.?
(Robert Firth)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181029/9041aa50/attachment.html>

From rene at pylots.com  Tue Oct 30 03:29:15 2018
From: rene at pylots.com (Rene Nejsum)
Date: Tue, 30 Oct 2018 08:29:15 +0100
Subject: [pypy-dev] Best way to inline a primitive array in RPython?
In-Reply-To: <CAL36E+vcrnOSZtu9Btoun3F7dRyJnti7059R1rQdk5VF+dq0Hg@mail.gmail.com>
References: <CAL36E+vcrnOSZtu9Btoun3F7dRyJnti7059R1rQdk5VF+dq0Hg@mail.gmail.com>
Message-ID: <9F84FF85-C97E-4A2E-9101-EB327F3B8814@pylots.com>

Hi Timothy/

The code below is kind of a C-trick (or hack) based on the knowledge of how the C-compiler lays out structs in memory.

In your example it would be:

struct layout
+------+
| a
+------+
| b
+------+
| _data[0]
+------+

Adding the +64, gives you 64 extra bytes (64/4) = 16 int's if sizeof(int) is 4 (32-bit)), making the layout look like:
+------+
| a
+------+
| b
+------+
| _data[0]
+------+
| +1
+------+
| +2
+------+
|
.
.
.
+------+
| +16
+------+

So now you can safely address tmp._data[15] = 0x12345678 ; // last element in struct

I don't know your need, but I don't think C was intended to be used like this, best practice is:

1) Hard allocate memory
#define SIZE 16
typedef struct foo_t {
int a, b;
int _data[SIZE];
}

2) Dynamic allocate memory
typedef struct foo_t {
int a, b;
int* _data;
struct tmp = foo_t;
tmp._data = malloc(64)

3) Hack alloc
- see above :-)

> On 30 Oct 2018, at 00.41, Timothy Baldridge <tbaldridge at gmail.com> wrote:
> 
> Looking at rbuffer, rgc, and the lltype namespaces, it's not completely clear to me how I would go about writing a struct with a resizable primitive array inlined.
> 
> Something like:
> 
> typedef struct foo_t {
> int a, b;
> int _data[0];
> }
> 
> foo_t tmp = malloc(sizeof(foo_t) + 64);

malloc() returns a pointer, so It should be foo_t* tmp = (foo_t*) malloc(sizeof(foo_t) + 64);

> I can see that the GC has the ability to allocate "extra" memory beyond what's required for a struct, and that lltype has the ability to inline some lists into a struct, but how does this work in practice? And maybe I'm missing something in the docs, so simply pointing me to some docs is also fine. 

Not sure where to find this, it is sometimes used, but I am not sure I would recommend it, unless you know exactly what your are doing and like keeping track of allocated memory :-)

br
/Rene

> 
> 
> -- 
> ?One of the main causes of the fall of the Roman Empire was that?lacking zero?they had no way to indicate successful termination of their C programs.?
> (Robert Firth)
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev


From armin.rigo at gmail.com  Tue Oct 30 03:58:06 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Tue, 30 Oct 2018 08:58:06 +0100
Subject: [pypy-dev] Best way to inline a primitive array in RPython?
In-Reply-To: <CAL36E+vcrnOSZtu9Btoun3F7dRyJnti7059R1rQdk5VF+dq0Hg@mail.gmail.com>
References: <CAL36E+vcrnOSZtu9Btoun3F7dRyJnti7059R1rQdk5VF+dq0Hg@mail.gmail.com>
Message-ID: <CAMSv6X3o9+MNqMwpbzxN7uVgssMNuqBy-29hcj+iUKa+Tb9WJQ@mail.gmail.com>

Hi Timothy,

On Tue, 30 Oct 2018 at 00:42, Timothy Baldridge <tbaldridge at gmail.com> wrote:
> typedef struct foo_t {
> int a, b;
> int _data[0];
> }
>
> foo_t tmp = malloc(sizeof(foo_t) + 64);

You can do that if you use the lltype.GcStruct type directly, not
using "regular" RPython code.  See the main example in
rpython.rtyper.lltypesystem.rstr: the types STR and UNICODE.  They are
defined as lltype.GcStruct('name', ..some_regular_fields..,
inlined_array_field), and allocated with lltype.malloc(STR,
length_of_array).

Note that you also need to prevent the JIT from seeing any such type:
it has got special case for STR and UNICODE but doesn't handle the
general case.


A bient?t,

Armin.

From tbaldridge at gmail.com  Tue Oct 30 07:37:28 2018
From: tbaldridge at gmail.com (Timothy Baldridge)
Date: Tue, 30 Oct 2018 05:37:28 -0600
Subject: [pypy-dev] Best way to inline a primitive array in RPython?
In-Reply-To: <CAMSv6X3o9+MNqMwpbzxN7uVgssMNuqBy-29hcj+iUKa+Tb9WJQ@mail.gmail.com>
References: <CAL36E+vcrnOSZtu9Btoun3F7dRyJnti7059R1rQdk5VF+dq0Hg@mail.gmail.com>
 <CAMSv6X3o9+MNqMwpbzxN7uVgssMNuqBy-29hcj+iUKa+Tb9WJQ@mail.gmail.com>
Message-ID: <CAL36E+t31LUZpE1KyPbnc2htvt+0ZGqviFZau4d0xj3EeDZTWw@mail.gmail.com>

Thanks, I'll check it out.

On Tue, Oct 30, 2018 at 1:58 AM Armin Rigo <armin.rigo at gmail.com> wrote:

> Hi Timothy,
>
> On Tue, 30 Oct 2018 at 00:42, Timothy Baldridge <tbaldridge at gmail.com>
> wrote:
> > typedef struct foo_t {
> > int a, b;
> > int _data[0];
> > }
> >
> > foo_t tmp = malloc(sizeof(foo_t) + 64);
>
> You can do that if you use the lltype.GcStruct type directly, not
> using "regular" RPython code.  See the main example in
> rpython.rtyper.lltypesystem.rstr: the types STR and UNICODE.  They are
> defined as lltype.GcStruct('name', ..some_regular_fields..,
> inlined_array_field), and allocated with lltype.malloc(STR,
> length_of_array).
>
> Note that you also need to prevent the JIT from seeing any such type:
> it has got special case for STR and UNICODE but doesn't handle the
> general case.
>
>
> A bient?t,
>
> Armin.
>


-- 
?One of the main causes of the fall of the Roman Empire was that?lacking
zero?they had no way to indicate successful termination of their C
programs.?
(Robert Firth)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20181030/f0f02045/attachment.html>