From mikes351 at yahoo.com  Sun Jan  1 23:26:05 2017
From: mikes351 at yahoo.com (Michael A.)
Date: Sun, 01 Jan 2017 23:26:05 -0500
Subject: [Wheel-builders] newbie question
Message-ID: <094f6333-987f-494d-8c00-b08b9723bf14@email.android.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/wheel-builders/attachments/20170101/b4f6ee9c/attachment.html>

From matthew.brett at gmail.com  Mon Jan  2 07:10:48 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Mon, 2 Jan 2017 12:10:48 +0000
Subject: [Wheel-builders] newbie question
In-Reply-To: <094f6333-987f-494d-8c00-b08b9723bf14@email.android.com>
References: <094f6333-987f-494d-8c00-b08b9723bf14@email.android.com>
Message-ID: <CAH6Pt5ovWL_J0tnV-+WR1YwoV31UONesp_GDg7fJC0TAabSmSg@mail.gmail.com>

Hi,

On Mon, Jan 2, 2017 at 4:26 AM, Michael A. via Wheel-builders
<wheel-builders at python.org> wrote:
> Ok it said newbie questions allowed. Im still in pretty basic syntax, just
> trying what python is capable of doing for me. I come across wheel on the
> help page and was wodering what all its capable of doing. I use 3.5 and seen
> videos of sti?l developing and thats c?ol considing everything is sti?l
> rather new. Does making wheels make .exe files for stand alone projects? Can
> i make wheels and be able to send executable files to other people with
> different operating systems? Is this like cx_Freeze I've seen used on
> youtube? Give it to me in less Geek terms if possible on usage and how to
> use it best for my scripts.

The wheel format is a zip file format that packages up everything you
need to install a particular Python package on a particular operating
system.   For example, have a look at:

https://pypi.python.org/pypi/Cython/0.25.2

You'll see there that the team responsible for the "Cython" package
have built these zip archives for Windows 32- and 64-bit, OSX and for
Intel Linux.  The Python install tool "pip" knows how to unpack these
zip files so that Python can 'import" these packages, as in (from the
command line):

pip install --user cython  # fetches the correct wheel from the pypi site above

Then:

python
>>> import cython   # finds the files installed above

So, short summary, the wheel is a binary format (in fact, a zip file
with a particular layout) that "pip" can use to install Python
packages.

To answer your specific questions:

* if your Python package doesn't have any compiled modules, yes you
can make a wheel that will work for other operating systems,
otherwise, no, you have to make a wheel for each operating system you
support, usually by building the wheel on that operating system;
* no, wheels aren't the right format to pack up a whole Python
application with all its dependencies, wheels are specifically for
containing individual packages (dependencies).

Cheers,

Matthew

From leonardo.bianconi at eldorado.org.br  Mon Jan  9 11:33:52 2017
From: leonardo.bianconi at eldorado.org.br (Leonardo Bianconi)
Date: Mon, 9 Jan 2017 16:33:52 +0000
Subject: [Wheel-builders] Wheel files for PPC64le
Message-ID: <8d5b0b78fc144cf39f3942494a01a130@serv031.corp.eldorado.org.br>

Hi All.

I have being analyzing the projects that makes wheel files be possible
for x86_64 and i686 architectures.
I have been changed some of them, and could create the numpy wheel
file for PPC64le. So far, I created a wheel file on Ubuntu 14.10 and I got
It installed and working on Ubuntu (16.10 and 14/04), SUSE
Tumbleweed and CentOS7.
I have sent an initial patch to the auditwheel project
(https://github.com/pypa/auditwheel/pull/59), but the maintainer
informed me that it needs discussion, and here is the place to start it.

So, what is necessary to make it work for PPC64le? What should I check
to know that it is possible? Is it necessary to write a new PEP?


Regards,
Leonardo.

From njs at pobox.com  Thu Jan 12 04:32:01 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 12 Jan 2017 01:32:01 -0800
Subject: [Wheel-builders] Wheel files for PPC64le
In-Reply-To: <8d5b0b78fc144cf39f3942494a01a130@serv031.corp.eldorado.org.br>
References: <8d5b0b78fc144cf39f3942494a01a130@serv031.corp.eldorado.org.br>
Message-ID: <CAPJVwBnFnrsUdmF4O-PFcY0boLoc3-tBpUA4K-Do4gGrA_2wvg@mail.gmail.com>

Hi Leonardo,

Yeah, you'll need to write a PEP. PEP 513 is the spec for i686/x86_64,
and should give you some idea of what to expect:

    https://www.python.org/dev/peps/pep-0513/

Yours might be much shorter, depending on how much stuff you can
hand-wave with "see PEP 513" :-).

Writing a PEP isn't too hard -- see:

    https://www.python.org/dev/peps/pep-0001/
    https://www.python.org/dev/peps/pep-0012/

The standard list for discussing packaging-related PEPs is actually
distutils-sig, not wheel-builders, so you'll want to move discussion
over there going forward.

The main technical issue here is that you'll want to have a somewhat
convincing story for (a) how to build packages that will be compatible
across a reasonable range of systems, (b) some way for pip to decide
whether it's running on one of those systems that your packages will
work on. I don't know enough about PPC64le to guess what this will
look like, but you can see PEP 513 for how we did it there.

A bonus would be if you have some nice story for how end-users can
actually build these packages, like the manylinux docker image. But
the manylinux docker image itself assumes the existence of
infrastructure like Travis-CI, which is not really something that
exists for ppc64le, I think. I guess you could try running docker in
qemu on travis :-).

-n

On Mon, Jan 9, 2017 at 8:33 AM, Leonardo Bianconi
<leonardo.bianconi at eldorado.org.br> wrote:
> Hi All.
>
> I have being analyzing the projects that makes wheel files be possible
> for x86_64 and i686 architectures.
> I have been changed some of them, and could create the numpy wheel
> file for PPC64le. So far, I created a wheel file on Ubuntu 14.10 and I got
> It installed and working on Ubuntu (16.10 and 14/04), SUSE
> Tumbleweed and CentOS7.
> I have sent an initial patch to the auditwheel project
> (https://github.com/pypa/auditwheel/pull/59), but the maintainer
> informed me that it needs discussion, and here is the place to start it.
>
> So, what is necessary to make it work for PPC64le? What should I check
> to know that it is possible? Is it necessary to write a new PEP?
>
>
> Regards,
> Leonardo.
> _______________________________________________
> Wheel-builders mailing list
> Wheel-builders at python.org
> https://mail.python.org/mailman/listinfo/wheel-builders



-- 
Nathaniel J. Smith -- https://vorpus.org

From ncoghlan at gmail.com  Fri Jan 13 11:53:43 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 14 Jan 2017 02:53:43 +1000
Subject: [Wheel-builders] Wheel files for PPC64le
In-Reply-To: <CAPJVwBnFnrsUdmF4O-PFcY0boLoc3-tBpUA4K-Do4gGrA_2wvg@mail.gmail.com>
References: <8d5b0b78fc144cf39f3942494a01a130@serv031.corp.eldorado.org.br>
 <CAPJVwBnFnrsUdmF4O-PFcY0boLoc3-tBpUA4K-Do4gGrA_2wvg@mail.gmail.com>
Message-ID: <CADiSq7d6LwuNWb8_m+aaMXWUM6=S+E1XPuBUgOxe9Ez2uFEXLQ@mail.gmail.com>

On 12 January 2017 at 19:32, Nathaniel Smith <njs at pobox.com> wrote:
> Writing a PEP isn't too hard -- see:
>
>     https://www.python.org/dev/peps/pep-0001/
>     https://www.python.org/dev/peps/pep-0012/
>
> The standard list for discussing packaging-related PEPs is actually
> distutils-sig, not wheel-builders, so you'll want to move discussion
> over there going forward.

The ABI compatibility is the tricky part here, so this probably isn't
a bad place to talk about it until some more of the problem points are
worked out.

> The main technical issue here is that you'll want to have a somewhat
> convincing story for (a) how to build packages that will be compatible
> across a reasonable range of systems, (b) some way for pip to decide
> whether it's running on one of those systems that your packages will
> work on. I don't know enough about PPC64le to guess what this will
> look like, but you can see PEP 513 for how we did it there.

The main problem-and-opportunity for ppc64le is that it's so new that
you immediately cut down your long term support distros of interest
to:

- RHEL 7.1+
- SLES 12+
- Ubuntu 14.04+

rather than having to go all the way back to RHEL5 to establish a
relatively common ABI.

The downside is that you'd be blazing new trails in terms of finding
out just *how* common the ppc64le ABIs actually are across those
distros.

> A bonus would be if you have some nice story for how end-users can
> actually build these packages, like the manylinux docker image. But
> the manylinux docker image itself assumes the existence of
> infrastructure like Travis-CI, which is not really something that
> exists for ppc64le, I think. I guess you could try running docker in
> qemu on travis :-)

Cross-compiling from x86_64 would likely be significantly less painful
for all concerned, although you'd still need qemu to test the
resulting wheel files.

https://github.com/dockcross/dockcross looks like it may be a
potentially interesting project in that vein.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From matthew.brett at gmail.com  Sat Jan 14 18:23:36 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sat, 14 Jan 2017 15:23:36 -0800
Subject: [Wheel-builders] A couple of small utilities and a page on build
 tags
Message-ID: <CAH6Pt5rjQg=ROjyx9sdPkh+bMef0PcX6=HNdju-9K5tR3OHLmw@mail.gmail.com>

Hi,

I've found myself having to use pip build tags a few times now, to
update wheels on pypi without doing a new release.  I wrote up a page
explaining how they work, including a small script to add a build tag
to your wheel filename:

https://github.com/MacPython/wiki/wiki/Build-Tags

I've also hit problems using twine to upload wheels that don't have a
properly formed email address for the maintainer.  Here's a little
script to set the author email address for a wheel, to fix uploading
via twine:

https://gist.github.com/matthew-brett/3cb5b19decdc717bd05123b65884e04c

Cheers,

Matthew

From matthew.brett at gmail.com  Sat Jan 14 18:29:07 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sat, 14 Jan 2017 15:29:07 -0800
Subject: [Wheel-builders] Rackspace URL for historical or missing wheels
Message-ID: <CAH6Pt5ommTvcTRD5jV6FFr=vhdmDSL4NcZuG0i2jZuf43ASF9Q@mail.gmail.com>

Hi,

I guess that some of you, like me, may be using travis-ci for testing,
and wheels to install your minimum dependencies.

I've set up a set of curated wheels for old versions of difficult to
build packages like matplotlib, Pillow and so on, over at:

https://5cf40426d9f06eb7461d-6fe47d9331aba7cd62fc36c7196769e4.ssl.cf2.rackcdn.com

I've tried to avoid overlap with pypi, so if the wheel is already on
pypi, you should not find it in the historical wheel URL above.  For
example, numpy and scipy already have historical wheels uploaded to
pypi.

I'm happy to get suggestions for other wheels that would help with
travis-ci runs, that aren't already on pypi.

Cheers,

Matthew

From brenohl at br.ibm.com  Tue Jan 24 08:42:52 2017
From: brenohl at br.ibm.com (Breno Leitao)
Date: Tue, 24 Jan 2017 11:42:52 -0200
Subject: [Wheel-builders] Wheel files for PPC64le
In-Reply-To: <CADiSq7d6LwuNWb8_m+aaMXWUM6=S+E1XPuBUgOxe9Ez2uFEXLQ@mail.gmail.com>
References: <8d5b0b78fc144cf39f3942494a01a130@serv031.corp.eldorado.org.br>
 <CAPJVwBnFnrsUdmF4O-PFcY0boLoc3-tBpUA4K-Do4gGrA_2wvg@mail.gmail.com>
 <CADiSq7d6LwuNWb8_m+aaMXWUM6=S+E1XPuBUgOxe9Ez2uFEXLQ@mail.gmail.com>
Message-ID: <90c09e61-7ae0-9f3f-a295-76866a8d7dbc@br.ibm.com>

Hello,

> On 12 January 2017 at 19:32, Nathaniel Smith <njs at pobox.com> wrote:
>> Writing a PEP isn't too hard -- see:
>>
>>     https://www.python.org/dev/peps/pep-0001/
>>     https://www.python.org/dev/peps/pep-0012/
>>
>> The standard list for discussing packaging-related PEPs is actually
>> distutils-sig, not wheel-builders, so you'll want to move discussion
>> over there going forward.
> 
> The ABI compatibility is the tricky part here, so this probably isn't
> a bad place to talk about it until some more of the problem points are
> worked out.

When you say ABI here, do you mean the first versions of the manylinux1
libraries, as listed in [1], that contains support (or was even compiled by
any distro) for pppc64le?

[1] https://www.python.org/dev/peps/pep-0513/#id40

If this assumption is true, we want to list the mimimum version of these
libraries that supports ABI v2 (little endian) mode?

>> The main technical issue here is that you'll want to have a somewhat
>> convincing story for (a) how to build packages that will be compatible
>> across a reasonable range of systems, (b) some way for pip to decide
>> whether it's running on one of those systems that your packages will
>> work on. I don't know enough about PPC64le to guess what this will
>> look like, but you can see PEP 513 for how we did it there.
> 
> The main problem-and-opportunity for ppc64le is that it's so new that
> you immediately cut down your long term support distros of interest
> to:
> 
> - RHEL 7.1+
> - SLES 12+
> - Ubuntu 14.04+
> 
> rather than having to go all the way back to RHEL5 to establish a
> relatively common ABI. 

Correct.

> 
> The downside is that you'd be blazing new trails in terms of finding
> out just *how* common the ppc64le ABIs actually are across those
> distros.

I would say that Ubuntu 14.04 has the first initial support for ppc64le
ABI, so, it would be the base. The problem happened later, when distro
with packages other than Ubuntu 14.04 backported the ppc64le patches to these
older packages.

Anyway, there is a tool that is able to help us here:
https://www-356.ibm.com/partnerworld/wps/ospat/ospat_v1.html

>> A bonus would be if you have some nice story for how end-users can
>> actually build these packages, like the manylinux docker image. But
>> the manylinux docker image itself assumes the existence of
>> infrastructure like Travis-CI, which is not really something that
>> exists for ppc64le, I think. I guess you could try running docker in
>> qemu on travis :-)

Hmm, can't we plug a remote ppc64le machine into Travis-CI? I understand
you are talking about pypa/manylinux repository, correct?

Thanks for helping us here,
Breno


From leonardo.bianconi at eldorado.org.br  Mon Jan 30 11:42:26 2017
From: leonardo.bianconi at eldorado.org.br (Leonardo Bianconi)
Date: Mon, 30 Jan 2017 16:42:26 +0000
Subject: [Wheel-builders] Wheel files for PPC64le
In-Reply-To: <90c09e61-7ae0-9f3f-a295-76866a8d7dbc@br.ibm.com>
References: <8d5b0b78fc144cf39f3942494a01a130@serv031.corp.eldorado.org.br>
 <CAPJVwBnFnrsUdmF4O-PFcY0boLoc3-tBpUA4K-Do4gGrA_2wvg@mail.gmail.com>
 <CADiSq7d6LwuNWb8_m+aaMXWUM6=S+E1XPuBUgOxe9Ez2uFEXLQ@mail.gmail.com>
 <90c09e61-7ae0-9f3f-a295-76866a8d7dbc@br.ibm.com>
Message-ID: <5200867097914522b4d702d049db8672@serv031.corp.eldorado.org.br>

Regarding the ABI, as the goal is make the wheel files available only for
little endian systems, it will be recognized by the flag "ppc64le", as the
big endian systems have the flag "ppc64" (https://en.wikipedia.org/wiki/Ppc64).
This behavior will make the process of building the wheel files very similar
with x86_64/i686, correct?


So, based on that, initially my plans to make it work is:

1. How to build wheel files?
   Building the wheel files for ppc64le will follow basically the same process
   that is done for x86_64/i686:
   - The binary executable and compiled code must link only with the same
     library list specified for x86_64/i686.
   - Must use the versions of GLIBC, CXXABI_, GLIBCXX_ and GCC provided with
     the Ubuntu 14.04 (The first distro for ppc64le).

2. How will pip decide if the system is eligible to install the wheel file?
   Detecting if the system is eligible to install the wheel file will follow
   the same rules from the module "_manylinux", but with the following change
   proposal:
   - Add the platform "ppc64le" in the platform list as a compatible one.
   - Remove the hard-coded glibc version on "return have_compatible_glibc(2, 5)
     ", and provide a json or configuration file to handle it for each
     architecture, which will be read to obtain the version value.

3. Compiling and testing wheel files.
   In order to compile the wheel files without a ppc64le machine, a virtualized
   environment will be necessary. So we can provide the following items:
   - Ubuntu 14.04 ppc64le ISO.
   - Wiki with instructions to install the required software and versions.
   - Create a docker file to configure the environment automatically (Not sure
     if this is a good solution, as the docker would run in a virtualized
     system).

Any questions about the ABI? If not, may I move this discussion to the
distutils-sig?


> -----Original Message-----
> From: Breno Leitao [mailto:brenohl at br.ibm.com]
> Sent: ter?a-feira, 24 de janeiro de 2017 11:43
> To: Nick Coghlan <ncoghlan at gmail.com>; Nathaniel Smith
> <njs at pobox.com>
> Cc: Leonardo Bianconi <leonardo.bianconi at eldorado.org.br>; wheel-
> builders at python.org
> Subject: Re: Re: [Wheel-builders] Wheel files for PPC64le
> 
> Hello,
> 
> > On 12 January 2017 at 19:32, Nathaniel Smith <njs at pobox.com> wrote:
> >> Writing a PEP isn't too hard -- see:
> >>
> >>     https://www.python.org/dev/peps/pep-0001/
> >>     https://www.python.org/dev/peps/pep-0012/
> >>
> >> The standard list for discussing packaging-related PEPs is actually
> >> distutils-sig, not wheel-builders, so you'll want to move discussion
> >> over there going forward.
> >
> > The ABI compatibility is the tricky part here, so this probably isn't
> > a bad place to talk about it until some more of the problem points are
> > worked out.
> 
> When you say ABI here, do you mean the first versions of the manylinux1
> libraries, as listed in [1], that contains support (or was even compiled by
> any distro) for pppc64le?
> 
> [1] https://www.python.org/dev/peps/pep-0513/#id40
> 
> If this assumption is true, we want to list the mimimum version of these
> libraries that supports ABI v2 (little endian) mode?
> 
> >> The main technical issue here is that you'll want to have a somewhat
> >> convincing story for (a) how to build packages that will be compatible
> >> across a reasonable range of systems, (b) some way for pip to decide
> >> whether it's running on one of those systems that your packages will
> >> work on. I don't know enough about PPC64le to guess what this will
> >> look like, but you can see PEP 513 for how we did it there.
> >
> > The main problem-and-opportunity for ppc64le is that it's so new that
> > you immediately cut down your long term support distros of interest
> > to:
> >
> > - RHEL 7.1+
> > - SLES 12+
> > - Ubuntu 14.04+
> >
> > rather than having to go all the way back to RHEL5 to establish a
> > relatively common ABI.
> 
> Correct.
> 
> >
> > The downside is that you'd be blazing new trails in terms of finding
> > out just *how* common the ppc64le ABIs actually are across those
> > distros.
> 
> I would say that Ubuntu 14.04 has the first initial support for ppc64le
> ABI, so, it would be the base. The problem happened later, when distro
> with packages other than Ubuntu 14.04 backported the ppc64le patches to
> these
> older packages.
> 
> Anyway, there is a tool that is able to help us here:
> https://www-356.ibm.com/partnerworld/wps/ospat/ospat_v1.html
> 
> >> A bonus would be if you have some nice story for how end-users can
> >> actually build these packages, like the manylinux docker image. But
> >> the manylinux docker image itself assumes the existence of
> >> infrastructure like Travis-CI, which is not really something that
> >> exists for ppc64le, I think. I guess you could try running docker in
> >> qemu on travis :-)
> 
> Hmm, can't we plug a remote ppc64le machine into Travis-CI? I understand
> you are talking about pypa/manylinux repository, correct?
> 
> Thanks for helping us here,
> Breno