From cstratak at redhat.com  Mon Mar  4 09:55:20 2019
From: cstratak at redhat.com (Charalampos Stratakis)
Date: Mon, 4 Mar 2019 09:55:20 -0500 (EST)
Subject: [Python-buildbots] Adding a Fedora Rawhide x86_64 to the buildbot
 fleet
In-Reply-To: <791621877.6131904.1551710429204.JavaMail.zimbra@redhat.com>
Message-ID: <2079274997.6136374.1551711320334.JavaMail.zimbra@redhat.com>

Hello everyone,

We've recently got a Fedora Rawhide virtual machine which should be available and up at all times and I'd like to add it to the existing fleet of buildbots.

Thus I'd like to request for a worker name and a password for it. I'm currently following the instructions at [0]
for setting up the buildbot.

https://devguide.python.org/buildworker/

-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat

From vstinner at redhat.com  Tue Mar  5 10:07:57 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Tue, 5 Mar 2019 16:07:57 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
Message-ID: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>

Hi,

I'm working on getting green buildbot workers at
https://buildbot.python.org/all/#/builders but they are many workers
which are "known" to be broken. Recently, Cheryl asked me if she broke
Alpine Linux... well, as far as I recall, this one has always been
broken: Python doesn't support musl C library.

What do you think of having a separated website for unstable buildbots?

Why doing that?

Right now, the server is sending notifications by email and IRC on
some state changes like green => red or purple (exception) => another
color. It generates noise which annoy me when I would like to have a
look at buildbots.

The other "unstable" buildbot servers would either not send any
notification, or use a different email address / different IRC
channel.

List of workers that I would like to move there:

* AIX
* Alpine Linux
* Cygwin
* UBSan

My intent is also to clarify the difference between a "fully supported
platform" and "best effort platform support":
https://pythondev.readthedocs.io/platforms.html

My intent is not to kick some platforms out of Python.

What do you think?

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From ncoghlan at gmail.com  Thu Mar  7 07:19:16 2019
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 7 Mar 2019 22:19:16 +1000
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
Message-ID: <CADiSq7eVBeioygqf8twPNccZJOWuZ_X298J_tXcKc3CMvYvQiQ@mail.gmail.com>

On Wed, 6 Mar 2019 at 01:08, Victor Stinner <vstinner at redhat.com> wrote:
> What do you think?

Fedora has long had a separation between primary, release blocking,
CPU architectures, and separate secondary architectures that generally
work, most of the time, but issues with them aren't going to prevent a
release going out.

Our stable/unstable split is similar, so if using a separate master
would make that easier to manage than the existing "stable" tag [1],
it seems like a reasonable idea to me.

Cheers,
Nick.

[1] https://buildbot.python.org/stable/
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From vstinner at redhat.com  Thu Mar  7 11:33:56 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Thu, 7 Mar 2019 17:33:56 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CADiSq7eVBeioygqf8twPNccZJOWuZ_X298J_tXcKc3CMvYvQiQ@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <CADiSq7eVBeioygqf8twPNccZJOWuZ_X298J_tXcKc3CMvYvQiQ@mail.gmail.com>
Message-ID: <CA+3bQGFSerEbvN4k_aNR1pKEM2p73FDGyDbeMPZ1avtmHKZHUg@mail.gmail.com>

I'm proposing to have a separated server to ensure that no
notification is sent. But I would be fine with the following
compromise: modifiy buildbot configuration (master.cfg) to disable
notifications on broken workers (tests known to fail) and "*-custom"
jobs.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From aixtools at felt.demon.nl  Thu Mar  7 14:48:11 2019
From: aixtools at felt.demon.nl (Michael)
Date: Thu, 7 Mar 2019 20:48:11 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
Message-ID: <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>

On 05/03/2019 16:07, Victor Stinner wrote:
> My intent is also to clarify the difference between a "fully supported
> platform" and "best effort platform support":
> https://pythondev.readthedocs.io/platforms.html
>
> My intent is not to kick some platforms out of Python.
>
> What do you think?

I think "ouch" - as I have been working hard to get all the tests to
pass for AIX. I feel "defeated".

-- Let me take a moment to just say thanks for everyone's help the last
months to get it this far (WARN message status on gccfarm bot due to
issues with linking to a third-party (not IBM AIX!) library to build an
extension.

-- Short comment on the "platforms.html" - sys.platform (re aix3 and
aix4) is dated. Is this something I can assist with via an issue and a PR?

-- oh yes, I thought "unstable" already meant no messages were sent.
Corrected.

-- p.s. I do not see you holding up a release of Python because a test
regresses on AIX (and no explanation can be found) - but I would hop
that AIX will be seen as stable enough to warrant being permitted to
send a message during development to signal a portability issue.

-- p.p.s. When I get an idea and/or assistance with understanding how to
debug the *multiprocessing* test failures the bot faces (but I
(generally) do not get when running manually) I'll dig further.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-buildbots/attachments/20190307/3fcf377c/attachment.sig>

From vstinner at redhat.com  Fri Mar  8 03:11:25 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Fri, 8 Mar 2019 09:11:25 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
Message-ID: <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>

Le jeu. 7 mars 2019 ? 21:19, Michael <aixtools at felt.demon.nl> a ?crit :
> I think "ouch" - as I have been working hard to get all the tests to
> pass for AIX. I feel "defeated".

Congrats :-) But that only for 3.x branch: 2.7 and 3.7 branches are
still red. Can we simply remove AIX jobs on 2.7 and 3.7 branches? Or
does anyone see the point of keeping them?

Tests pass on PPC64 3.x, but all jobs POWER6 AIX are job (including
POWER6 AIX 3.x).

> -- Short comment on the "platforms.html" - sys.platform (re aix3 and
> aix4) is dated. Is this something I can assist with via an issue and a PR?

http://pythondev.readthedocs.io/platforms.html is my website.

https://github.com/vstinner/pythondev is the source.

I have no idea of what is the current value of sys.platform on recent AIX.

> -- oh yes, I thought "unstable" already meant no messages were sent.
> Corrected.

Oh wait, I wasn't aware of that. Is it something new? I confirm that I
saw this code in master.cfg:

        if stability == STABLE:
            mail_status_builders.append(buildername)
            github_status_builders.append(buildername)

I started this email thread because I get emails from custom jobs. It
seems like this test should be updated to explicitly exclude
notifications from custom jobs.


> -- p.s. I do not see you holding up a release of Python because a test
> regresses on AIX (and no explanation can be found) - but I would hop
> that AIX will be seen as stable enough to warrant being permitted to
> send a message during development to signal a portability issue.

The release process is unrelated to my thread. My use case is that I'm
keeping an eye on buildbots as part of the Night's Watch team: I'm
reading most emails sent to buildbot-status and I'm trying to analyze
all failures.

I didn't know that unstable buildbots don't send emails. When an
unstable buildbot switch from green to red, I would like to be
notified as well. It can be a regression in Python, not specific to
the buildbot.

I can handle regresions caused by a change on a buildbot, like an
update of the system.


> -- p.p.s. When I get an idea and/or assistance with understanding how to
> debug the *multiprocessing* test failures the bot faces (but I
> (generally) do not get when running manually) I'll dig further.

Oh. Multiprocessing is complex. There are still many know issues
reported to the bug tracker, *but* multiprocessing tests should be way
more stable on buildbots now, than they were 1 or 2 years ago.

You should be more specific if you want to help you, but maybe start a
new thread instead.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From vstinner at redhat.com  Fri Mar  8 03:47:42 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Fri, 8 Mar 2019 09:47:42 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
Message-ID: <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>

Le ven. 8 mars 2019 ? 09:11, Victor Stinner <vstinner at redhat.com> a ?crit :
> Oh wait, I wasn't aware of that. Is it something new? I confirm that I
> saw this code in master.cfg:
>
>         if stability == STABLE:
>             mail_status_builders.append(buildername)
>             github_status_builders.append(buildername)
>
> I started this email thread because I get emails from custom jobs. It
> seems like this test should be updated to explicitly exclude
> notifications from custom jobs.

I proposed https://github.com/python/buildmaster-config/pull/82 to:

* Disable notifications for custom builders
* Re-enable email notifications for unstable builders

I also proposed https://github.com/python/buildmaster-config/pull/83
to only run AIX, Alpine Linux and Clang UBSan on the master branch.

Victor

From aixtools at felt.demon.nl  Fri Mar  8 04:43:59 2019
From: aixtools at felt.demon.nl (Michael Felt)
Date: Fri, 8 Mar 2019 10:43:59 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
Message-ID: <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>


On 3/8/2019 9:47 AM, Victor Stinner wrote:
> Le ven. 8 mars 2019 ? 09:11, Victor Stinner <vstinner at redhat.com> a ?crit :
>> Oh wait, I wasn't aware of that. Is it something new? I confirm that I
>> saw this code in master.cfg:
>>
>>         if stability == STABLE:
>>             mail_status_builders.append(buildername)
>>             github_status_builders.append(buildername)
>>
>> I started this email thread because I get emails from custom jobs. It
>> seems like this test should be updated to explicitly exclude
>> notifications from custom jobs.
> I proposed https://github.com/python/buildmaster-config/pull/82 to:
>
> * Disable notifications for custom builders
> * Re-enable email notifications for unstable builders
>
> I also proposed https://github.com/python/buildmaster-config/pull/83
> to only run AIX, Alpine Linux and Clang UBSan on the master branch.

I am fine with running only on the master branch - as I long gave up the
fight for back-porting. Hard enough to get people to look at master
(emote: speaking - so also see a smile). For 2.7 the view has been, for
as long as I have been trying, that all my fixes were "new features" so
they were refused. Or some variation of that.

If there is mutual commitment (I'll give mine) to also get the 3.7
branch to PASS, I'll certainly look at that. Some were backported
automagically, others were not.

As to supporting AIX - I obviously, cannot promise 24x7 support - only
best effort for as long as I am able. My goal as a packager is that it
be transparent and what can be installed can be uninstalled. And, imho,
stable Python is in the interest of AIX customers. They are the people I
try to cater to.

And, in that line - if possible, you may add my email address to the
mails the AIX bots send.

Side-question: when do the build-times and success-rate start getting
scored. The PPC64 bot is "stable" in that it does not fail everything.
And if you could accept the change to setup.py to not always give the
preference to ncurses (which is onloy available via third party on AIX)
I hope it would go to full "PASS" status.

FYI - they key difference between the POWER6 and PPC64 bots are not
POWER6 versus POWER8, or AIX 7.1 versus AIX 7.2 - b ut the compiler.
"POWER6" uses xlc (v11) while PPC64 uses "gcc (v7.2 iirc)".
Additionally, my (POWER6) has no "AIXToolbox aka third-party) packages
installed. Anything additional needed is built and packaged using xlc -
to focus the entire runt-time-environment on IBM provided code. And
wheile the bot runs on AIX 7.1 I regularly test manually on AIX 6.1 and
AIX 5.3.

In short, there are some code dependencies that show up when the
compiler is not gcc. They are very hard to find, sadly. I hope finding
them makes CPython more portable and stable. No other reason to pursue them.

Sincerely,

Michael

>
> Victor
>

From vstinner at redhat.com  Fri Mar  8 09:09:48 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Fri, 8 Mar 2019 15:09:48 +0100
Subject: [Python-buildbots] Email notifications enabled again on unstable
 builders
Message-ID: <CA+3bQGFgxqmb-_JLoqCtkkh-n3OPwRkLgAcFami6g-pOeme8+A@mail.gmail.com>

but disabled on custom builders.

I deployed the new configuration:

https://github.com/python/buildmaster-config/commit/e85208dd39ad55741273b662348bc1b1d4c42d74

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From vstinner at redhat.com  Fri Mar  8 09:20:05 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Fri, 8 Mar 2019 15:20:05 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
 <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
Message-ID: <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>

Le ven. 8 mars 2019 ? 10:44, Michael Felt <aixtools at felt.demon.nl> a ?crit :
> I am fine with running only on the master branch - as I long gave up the
> fight for back-porting. Hard enough to get people to look at master
> (emote: speaking - so also see a smile). For 2.7 the view has been, for
> as long as I have been trying, that all my fixes were "new features" so
> they were refused. Or some variation of that.

Right, we rarely backport changes to support a new platform in stable
branch. Example of exception: we backported changes to support OpenSSL
1.1.x, but it's not exactly a "platform" :-)


> As to supporting AIX - I obviously, cannot promise 24x7 support - only
> best effort for as long as I am able. My goal as a packager is that it
> be transparent and what can be installed can be uninstalled. And, imho,
> stable Python is in the interest of AIX customers. They are the people I
> try to cater to.

Nobody is available 24x7 :-)

Don't worry, we will bug you as soon as it turns red again by email
and you can reply whenever you want or ignore such bug :-)


> And, in that line - if possible, you may add my email address to the
> mails the AIX bots send.

We don't have such granularity yet. Again, we will simply notify you
explicitly, ex: on bug reports.


> Side-question: when do the build-times and success-rate start getting
> scored. The PPC64 bot is "stable" in that it does not fail everything.
> And if you could accept the change to setup.py to not always give the
> preference to ncurses (which is onloy available via third party on AIX)
> I hope it would go to full "PASS" status.

I don't understand which change you are proposing. Maybe open a bug report.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From aixtools at felt.demon.nl  Sat Mar  9 08:02:00 2019
From: aixtools at felt.demon.nl (Michael)
Date: Sat, 9 Mar 2019 14:02:00 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
 <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
 <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>
Message-ID: <8a4d16f1-c758-2bf3-47be-15b820a6a631@felt.demon.nl>

On 08/03/2019 15:20, Victor Stinner wrote:
> Le ven. 8 mars 2019 ? 10:44, Michael Felt <aixtools at felt.demon.nl> a ?crit :
>> I am fine with running only on the master branch - as I long gave up the
>> fight for back-porting. Hard enough to get people to look at master
>> (emote: speaking - so also see a smile). For 2.7 the view has been, for
>> as long as I have been trying, that all my fixes were "new features" so
>> they were refused. Or some variation of that.
> Right, we rarely backport changes to support a new platform in stable
> branch. Example of exception: we backported changes to support OpenSSL
> 1.1.x, but it's not exactly a "platform" :-)

I was actually think more of the backporting from master to 3.7 and 3.6.
It was made very clear to me well over a year ago that "nothing" would
be considered for 2.7. However, the issues I have been fixing have been
around forever. I guess I was not smart enough to "request" a backport
for the previous issues.

Not trying to be a burden for anyone - is it worth looking into? I could
try some "simple" merges. Most of the changes are closer to typo's that
anything else.

>
>
>> As to supporting AIX - I obviously, cannot promise 24x7 support - only
>> best effort for as long as I am able. My goal as a packager is that it
>> be transparent and what can be installed can be uninstalled. And, imho,
>> stable Python is in the interest of AIX customers. They are the people I
>> try to cater to.
> Nobody is available 24x7 :-)
(No I am not :p - I know I am nobody - my wife tells me so whenever I go
on a trip -"Nobody loves me!" So I guess "Noone" is available 24x7 :) )
>
> Don't worry, we will bug you as soon as it turns red again by email
> and you can reply whenever you want or ignore such bug :-)
>
>
>> And, in that line - if possible, you may add my email address to the
>> mails the AIX bots send.
> We don't have such granularity yet. Again, we will simply notify you
> explicitly, ex: on bug reports.
>
>
>> Side-question: when do the build-times and success-rate start getting
>> scored. The PPC64 bot is "stable" in that it does not fail everything.
>> And if you could accept the change to setup.py to not always give the
>> preference to ncurses (which is onloy available via third party on AIX)
>> I hope it would go to full "PASS" status.
> I don't understand which change you are proposing. Maybe open a bug report.

I have a bug report, and a PR open for this. Would be "nice" to see it
backported into 3.7 and 3.6 as well.

https://bugs.python.org/issue36210 and
https://github.com/python/cpython/pull/12202

>
> Victor



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-buildbots/attachments/20190309/6423e946/attachment.sig>

From vstinner at redhat.com  Tue Mar 12 05:18:17 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Tue, 12 Mar 2019 10:18:17 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <8a4d16f1-c758-2bf3-47be-15b820a6a631@felt.demon.nl>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
 <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
 <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>
 <8a4d16f1-c758-2bf3-47be-15b820a6a631@felt.demon.nl>
Message-ID: <CA+3bQGFxHbPDUZ+BJj57uQr5VoUVHWs74hMTS71gGay9=GEgnA@mail.gmail.com>

Le sam. 9 mars 2019 ? 14:00, Michael <aixtools at felt.demon.nl> a ?crit :
> I was actually think more of the backporting from master to 3.7 and 3.6.
> It was made very clear to me well over a year ago that "nothing" would
> be considered for 2.7. However, the issues I have been fixing have been
> around forever. I guess I was not smart enough to "request" a backport
> for the previous issues.
>
> Not trying to be a burden for anyone - is it worth looking into? I could
> try some "simple" merges. Most of the changes are closer to typo's that
> anything else.

3.6 now only accept security fixes.

I don't think that it's worth it to touch Python 2.7 which is close to
its death.

I'm not interested to spend time on backporting AIX-specific changes
to 3.7. I dislike introducing too many changes in a stable branch. My
experience tells me that any single change can introduce a regression.
I suggest to focus on the master branch.

I'm telling about Python upstream. You can easily maintain your
CPython fork for patches on 2.7, 3.6 and 3.7. It's very common that
packages of a Linux distribution include downstream patches. I know
that well, I'm working for Red Hat and we have between 12 and 55
patches per Python package :-)

Maybe you will find someone else to do the backport ;-)

Victor

From aixtools at felt.demon.nl  Sun Mar 24 11:25:44 2019
From: aixtools at felt.demon.nl (Michael)
Date: Sun, 24 Mar 2019 16:25:44 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGFxHbPDUZ+BJj57uQr5VoUVHWs74hMTS71gGay9=GEgnA@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
 <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
 <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>
 <8a4d16f1-c758-2bf3-47be-15b820a6a631@felt.demon.nl>
 <CA+3bQGFxHbPDUZ+BJj57uQr5VoUVHWs74hMTS71gGay9=GEgnA@mail.gmail.com>
Message-ID: <7046239a-6641-4cf2-522c-34d3719691e5@felt.demon.nl>

On 12/03/2019 10:18, Victor Stinner wrote:
> Le sam. 9 mars 2019 ? 14:00, Michael <aixtools at felt.demon.nl> a ?crit :
>> I was actually think more of the backporting from master to 3.7 and 3.6.
>> It was made very clear to me well over a year ago that "nothing" would
>> be considered for 2.7. However, the issues I have been fixing have been
>> around forever. I guess I was not smart enough to "request" a backport
>> for the previous issues.
>>
>> Not trying to be a burden for anyone - is it worth looking into? I could
>> try some "simple" merges. Most of the changes are closer to typo's that
>> anything else.
> 3.6 now only accept security fixes.
>
> I don't think that it's worth it to touch Python 2.7 which is close to
> its death.
>
> I'm not interested to spend time on backporting AIX-specific changes
> to 3.7. I dislike introducing too many changes in a stable branch. My
> experience tells me that any single change can introduce a regression.
> I suggest to focus on the master branch.
>
> I'm telling about Python upstream. You can easily maintain your
> CPython fork for patches on 2.7, 3.6 and 3.7. It's very common that
> packages of a Linux distribution include downstream patches. I know
> that well, I'm working for Red Hat and we have between 12 and 55
> patches per Python package :-)
>
> Maybe you will find someone else to do the backport ;-)
>
> Victor
>
Understood - and have, after I better understood the reluctance for any
AIX related patch - took, I think it was yours (Victor) advice - worry
about master, and when that is done (all test pass) - re-evaluate.

I have also already started maintaining my own forks for 2.7, and for a
while 3.4 through 3.6.

I have no issue doing the backports myself. I might need to learn some
tricks with git to make it nearly automated, but I do not want to have
the expectation that they will be considered, when in fact, there seems
to be more aversion than willingness to accept.

I too am a volunteer. I have no commercial benefit from anything I have
done. Nor do I expect any from any future action. No manager of mine is
encouraging me to do this - so no "bonus" at all. I hoped I was
contributing.

The last few messages have discouraged me in ways that months of
discouraging messages from many sources never did. I choose to ignore
those messages. But now the 'fire' is out.

I gave my word, and shall keep it. But only if I feel there is some
level of acceptance. If you really do not want AIX supported - I am
finally ready to accept the hint. I'll leave you all alone (and wish you
well).

So, to get back to core question: If I backport the patches to the
failing tests will they be considered for merge?

Sincerely,

Michael

p.s. I am responding here as my gut says it has the smallest following.
I am not looking for any 'distress' with anyone.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-buildbots/attachments/20190324/9fb4c3ac/attachment.sig>

From vstinner at redhat.com  Mon Mar 25 04:50:51 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Mon, 25 Mar 2019 09:50:51 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <7046239a-6641-4cf2-522c-34d3719691e5@felt.demon.nl>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
 <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
 <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>
 <8a4d16f1-c758-2bf3-47be-15b820a6a631@felt.demon.nl>
 <CA+3bQGFxHbPDUZ+BJj57uQr5VoUVHWs74hMTS71gGay9=GEgnA@mail.gmail.com>
 <7046239a-6641-4cf2-522c-34d3719691e5@felt.demon.nl>
Message-ID: <CA+3bQGG7f7403atOXNqwfBAq6FujSpWcwk6XmF5mRJsxmymjdA@mail.gmail.com>

Hello,

Le dim. 24 mars 2019 ? 16:32, Michael <aixtools at felt.demon.nl> a ?crit :
> Understood - and have, after I better understood the reluctance for any
> AIX related patch - took, I think it was yours (Victor) advice - worry
> about master, and when that is done (all test pass) - re-evaluate.

I told you that AIX is not my priority. I have limited time to work on
Python upstream, and so I make choices to prioritize and organize my
time. I told you how I work so you can adjust your expectations in
term in availability of core developers. I contacted you, since I
guess that other core devs will not, and you was asking questions
about (the lack of) reviews.

There is no such "reluctance" from me.

It seems like your underestimate how much time I spent on your AIX
changes. I had to not review other changes while reviewing your
changes. I had to not fix other bugs while looking at your bugs. Time
is precious :-) You likely also underestimate the cost of just basic
maintenance on Windows, macOS and Linux: just ensure that the CI is
"not broken": it's between 20% and 40% of my whole week. Supporting a
new platform would only increase this ratio for me, whereas I would
like to reduce this ratio. Hopefully, slowly, I'm getting help on
maintainaing these platforms :-) Sometimes, I don't fix regressions, I
just report them. But producing an useful bug report requires a look a
little bit at the bug, it takes me a lot of time (more than what I
would like to spend).


> I have also already started maintaining my own forks for 2.7, and for a
> while 3.4 through 3.6.
>
> I have no issue doing the backports myself. I might need to learn some
> tricks with git to make it nearly automated, but I do not want to have
> the expectation that they will be considered, when in fact, there seems
> to be more aversion than willingness to accept.

All Linux distributions maintain downstream patches for whatever
reasons. There is a lot of tooling to help you in this task. Git is a
good start :-)



> I gave my word, and shall keep it. But only if I feel there is some
> level of acceptance. If you really do not want AIX supported - I am
> finally ready to accept the hint. I'll leave you all alone (and wish you
> well).

I told you how I work, it's a matter of priorities. For example, Red
Hat customers issues are more important than any Python upstream
issues for me ;-)


> I too am a volunteer. I have no commercial benefit from anything I have
> done. Nor do I expect any from any future action. No manager of mine is
> encouraging me to do this - so no "bonus" at all. I hoped I was
> contributing.

I'm not sure why you are writing that.

Are you expecting that your changes deserve to be reviewed because you
are working on your free time on Python? Do you think that other
contributors are paid to write their changes?

If you expected to get rewarded, maybe you chose the wrong project :-)
Python doesn't work like that. The motto is "We are all volunteer" :-)


> So, to get back to core question: If I backport the patches to the
> failing tests will they be considered for merge?

Just to elaborate what I wrote previously: while I had limited time to
review AIX changes on the master branch, fixing issues specific to AIX
on 3.7 is a low priority for me. I only see 3.7 changes as a threat to
3.7 stability. I'm talking about the general case, not the specific
case, so I'm always very careful with changes in stable branches. For
example, we are rejecting more and more bugfixes in 2.7 because of the
high risk of regressions introduced by subtle behavior changes, even
when the reported bug is severe (sadly, more and more severe bugs are
known).

For AIX, my priority remains the master branch, and not all AIX
buildbots are green yet. The following bug seems severe, it's a crash:
https://bugs.python.org/issue36273

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From Paul.Monson at microsoft.com  Tue Mar 26 20:41:33 2019
From: Paul.Monson at microsoft.com (Paul Monson)
Date: Wed, 27 Mar 2019 00:41:33 +0000
Subject: [Python-buildbots] Windows ARM32 buildbots
Message-ID: <CY4PR2101MB08044B1BAAD9408B3E2C40E5F9580@CY4PR2101MB0804.namprd21.prod.outlook.com>

Hi,

I am preparing to set up Windows ARM32 buildbots.
The doc page at https://devguide.python.org/buildworker/
says to send email to this alias to discuss adding a worker and to get a workername and password.

Thanks,
Paul

From aixtools at felt.demon.nl  Wed Mar 27 12:36:36 2019
From: aixtools at felt.demon.nl (Michael Felt)
Date: Wed, 27 Mar 2019 17:36:36 +0100
Subject: [Python-buildbots] Idea: have a different buildbot server for
 unstable workers?
In-Reply-To: <CA+3bQGG7f7403atOXNqwfBAq6FujSpWcwk6XmF5mRJsxmymjdA@mail.gmail.com>
References: <CA+3bQGE3bxqWQNhB3j3q_zNi-U4_vdMzUTpxx4-XENXkzfs8vg@mail.gmail.com>
 <2c8eb5ef-fb4d-6b0d-0bec-fc2b2f205049@felt.demon.nl>
 <CA+3bQGGytk_wsj5yU0xt7EGF-p25iOoh5rnq_gAxhy60_pyVEw@mail.gmail.com>
 <CA+3bQGF9dcR7v5ffxKg4_P8my5y1_sAPvNsDZdF4Nt1XFVjLJA@mail.gmail.com>
 <1f00ea7d-d106-db4b-597c-4b4f8b575b4b@felt.demon.nl>
 <CA+3bQGH5AwCsdh9BTH8_ZtsZUtqbYDU0E4fiAKigBaw9tPOMTw@mail.gmail.com>
 <8a4d16f1-c758-2bf3-47be-15b820a6a631@felt.demon.nl>
 <CA+3bQGFxHbPDUZ+BJj57uQr5VoUVHWs74hMTS71gGay9=GEgnA@mail.gmail.com>
 <7046239a-6641-4cf2-522c-34d3719691e5@felt.demon.nl>
 <CA+3bQGG7f7403atOXNqwfBAq6FujSpWcwk6XmF5mRJsxmymjdA@mail.gmail.com>
Message-ID: <566B7249-A8BE-4F99-A669-1B0AF13DAF2B@felt.demon.nl>



> On 25-Mar-19 09:50, Victor Stinner wrote:
> Hello,
> 
> Le dim. 24 mars 2019 ? 16:32, Michael <aixtools at felt.demon.nl> a ?crit :
>> Understood - and have, after I better understood the reluctance for any
>> AIX related patch - took, I think it was yours (Victor) advice - worry
>> about master, and when that is done (all test pass) - re-evaluate.
> I told you that AIX is not my priority.
Yes you have. For all clarity - I do not expect AIX to become a priority.
> I have limited time to work on Python upstream, and so I make choices to prioritize and organize my
> time. I told you how I work so you can adjust your expectations in
> term in availability of core developers. I contacted you, since I
> guess that other core devs will not, and you was asking questions
> about (the lack of) reviews.
> 
> There is no such "reluctance" from me.
Yes, this has been difficult to learn patience about reviews. I was trying to pace my comments to be pre-rc release. Once that process begins I do not expect any attention (aka priority).
> It seems like your underestimate how much time I spent on your AIX
> changes.
Correct, that is not something I can estimate. And from your comment. I apologize for being a burden.

I also wonder if you underestimate the amount of time I have put into debugging before I come up with anything. Lets just say I am also surprised at the amount of time I have invested in CPython.
>  I had to not review other changes while reviewing your
> changes. I had to not fix other bugs while looking at your bugs. Time
> is precious :-) You likely also underestimate the cost of just basic
> maintenance on Windows, macOS and Linux: just ensure that the CI is
> "not broken": it's between 20% and 40% of my whole week. 
Not that much of my whole week, as I have a job. But for a few months it was nearly 100% of my free time.
> Supporting a
> new platform would only increase this ratio for me, whereas I would
> like to reduce this ratio. Hopefully, slowly, I'm getting help on
> maintainaing these platforms :-) Sometimes, I don't fix regressions, I
> just report them. But producing an useful bug report requires a look a
> little bit at the bug, it takes me a lot of time (more than what I
> would like to spend).
Yes, I am suffering especially in that area re: the "multi-processing" tests. Sometimes they fail, sometimes they do not. 

I had hoped my questions were recognizable by someone familiar (enough) with internals that I could then get some kind of forward motion.
>> I have also already started maintaining my own forks for 2.7, and for a
>> while 3.4 through 3.6.
>> 
>> I have no issue doing the backports myself. I might need to learn some
>> tricks with git to make it nearly automated, but I do not want to have
>> the expectation that they will be considered, when in fact, there seems
>> to be more aversion than willingness to accept.
> All Linux distributions maintain downstream patches for whatever
> reasons. There is a lot of tooling to help you in this task. Git is a
> good start :-)
I am trying to pursue the high-road. I do not submit a patch unless I have verified that it works on AIX 5.3 and, unmodified, can run on AIX 7.1 and AIX 6.1. I do not have ready access to an AIX 7.2 system with "install" access. i.e., I do not want to maintain a "distribution". My goal is to have a single package that installs without requiring multiple additional packages. IMHO, the apporach taken by others - who have different packages for each level/release (of AIX) only make life harder than necessary for admins to remain up to date.
> 
>> I gave my word, and shall keep it. But only if I feel there is some
>> level of acceptance. If you really do not want AIX supported - I am
>> finally ready to accept the hint. I'll leave you all alone (and wish you
>> well).
> I told you how I work, it's a matter of priorities. For example, Red
> Hat customers issues are more important than any Python upstream
> issues for me ;-)
Understood. And you have been very clear on this. I have tried to not ask (directly) you when it comes to AIX. And, whenever possible I also try to ask platform independent questions.
> 
>> I too am a volunteer. I have no commercial benefit from anything I have
>> done. Nor do I expect any from any future action. No manager of mine is
>> encouraging me to do this - so no "bonus" at all. I hoped I was
>> contr, buting.
Because I have not hundreds, but clearly tens of replies from people saying "I am a volunteer", or "they are volunteers". All implying that I, somehow, must not be a "volunteer", i.e., being paid to pursue this. It is my free time, my systems, and my electric bill. So, what I am trying to make clear is that I am investing time and materials and am rather tired of the "they are volunteers" card - as if I am not giving something of myself.

I hope that is more clear. In any case, I shall not bring it up again.
> I'm not sure why you are writing that.
> 
> Are you expecting that your changes deserve to be reviewed because you
> are working on your free time on Python? Do you think that other
> contributors are paid to write their changes?
No. per above - I was getting the impression people thought I was paid. I - like many others - do this "as a volunteer" and hope that my "investment" is appreciated. If I am a burden, then per below, this is the wrong project.
> If you expected to get rewarded, maybe you chose the wrong project :-)
> Python doesn't work like that. The motto is "We are all volunteer" :-)
:-)
> 
>> So, to get back to core question: If I backport the patches to the
>> failing tests will they be considered for merge?
> Just to elaborate what I wrote previously: while I had limited time to
> review AIX changes on the master branch, fixing issues specific to AIX
> on 3.7 is a low priority for me. I only see 3.7 changes as a threat to
> 3.7 stability. I'm talking about the general case, not the specific
> case, so I'm always very careful with changes in stable branches. For
> example, we are rejecting more and more bugfixes in 2.7 because of the
> high risk of regressions introduced by subtle behavior changes, even
> when the reported bug is severe (sadly, more and more severe bugs are
> known).
> 
> For AIX, my priority remains the master branch, and not all AIX
> buildbots are green yet.
a) to save you time, I have deactivated my bot. There is an issue re: xlc, because when I build using gcc the failure goes away. No point is generating noise.

b) the bot on the gccfarm is passing all the tests, but comes up yellow because the admin had installed a third-party package that is not working with python. I have dug into enough to feel it could be a two-way street. In any case, python does not build the optional module "curses". On my bot, where I do not have this additional library installed (ncurses) python builds curses and the curses related tests pass.

I understand that AIX is not a priority - but I did submit a PR to change setup.py to not build ncurses by default on AIX - as ncurses     is always "third-party" and introduces an additional installation dependency. Relying the officially supported libcurses.a takes away several failures (and I would hope permit the gcc based bot to pass).

fyi: for the "bot" I'll probably switch it to use gcc to remove this confusion. 
>  The following bug seems severe, it's a crash:
> https://bugs.python.org/issue36273
c) I'll look at this, and reply when I know more.
> Victor
Thank you for your time taken to read and consider my comments. Your comments are appreciated and I know your time is limited.

Michael


From zachary.ware+pydev at gmail.com  Sat Mar 30 12:36:54 2019
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Sat, 30 Mar 2019 11:36:54 -0500
Subject: [Python-buildbots] Adding a Fedora Rawhide x86_64 to the
 buildbot fleet
In-Reply-To: <2079274997.6136374.1551711320334.JavaMail.zimbra@redhat.com>
References: <791621877.6131904.1551710429204.JavaMail.zimbra@redhat.com>
 <2079274997.6136374.1551711320334.JavaMail.zimbra@redhat.com>
Message-ID: <CAKJDb-MkVrowsxpM0nkDPBf7B0gh95D7s48ChPoo5ibj04+O8w@mail.gmail.com>

On Mon, Mar 4, 2019 at 8:55 AM Charalampos Stratakis
<cstratak at redhat.com> wrote:
> Hello everyone,

Hi, and sorry this has taken so long for me to respond!

> We've recently got a Fedora Rawhide virtual machine which should be available and up at all times and I'd like to add it to the existing fleet of buildbots.
>
> Thus I'd like to request for a worker name and a password for it. I'm currently following the instructions at [0]
> for setting up the buildbot.

You can expect an email from me shortly with your worker name and
password.  Once you have that you'll be able to connect your worker,
but it won't actually do anything until we assign builders to it which
we can do via a PR to the https://github.com/python/buildmaster-config
repository, which houses the non-secret configuration for the buildbot
master.

Thanks for setting up a worker!
--
Zach

From zachary.ware+pydev at gmail.com  Sat Mar 30 12:40:19 2019
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Sat, 30 Mar 2019 11:40:19 -0500
Subject: [Python-buildbots] Windows ARM32 buildbots
In-Reply-To: <CY4PR2101MB08044B1BAAD9408B3E2C40E5F9580@CY4PR2101MB0804.namprd21.prod.outlook.com>
References: <CY4PR2101MB08044B1BAAD9408B3E2C40E5F9580@CY4PR2101MB0804.namprd21.prod.outlook.com>
Message-ID: <CAKJDb-NToA7t233CXcoRcvDLGrm9V52kx2fbzp1e-cu9H5Migw@mail.gmail.com>

On Wed, Mar 27, 2019 at 10:43 AM Paul Monson via Python-Buildbots
<python-buildbots at python.org> wrote:
> Hi,

Hi Paul,

> I am preparing to set up Windows ARM32 buildbots.
> The doc page at https://devguide.python.org/buildworker/
> says to send email to this alias to discuss adding a worker and to get a workername and password.

Expect an off-list email from me shortly with your worker name and
password.  Once you have that you'll be able to connect your worker,
but it won't actually do anything until we assign builders to it which
we can do via a PR to the https://github.com/python/buildmaster-config
repository, which houses the non-secret configuration for the buildbot
master.

Thanks for setting this up!

--
Zach