From zachary.ware+pydev at gmail.com  Mon Sep  4 14:32:54 2017
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 4 Sep 2017 13:32:54 -0500
Subject: [Python-buildbots] OpenSSL versions on builders
Message-ID: <CAKJDb-OzfWcs9pU51YB3UMeSuYt9wz_9TKKirN2kveE39ZSH2A@mail.gmail.com>

Hi folks,

Since OpenSSL 1.0.1 was EOL'd at the end of 2016, we're looking into
whether we can drop support for OpenSSL 1.0.1 and earlier in Python
3.7.  However, it looks like many of our builders are still using as
far back as 1.0.1e.  If you would, please check to make sure your
builders are using the latest OpenSSL available from your distributor
and let us know if it is not at least 1.0.2 so we can gauge how to
proceed.

Thanks,
-- 
Zach

From greg at krypto.org  Mon Sep  4 15:49:13 2017
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 04 Sep 2017 19:49:13 +0000
Subject: [Python-buildbots] OpenSSL versions on builders
In-Reply-To: <CAKJDb-OzfWcs9pU51YB3UMeSuYt9wz_9TKKirN2kveE39ZSH2A@mail.gmail.com>
References: <CAKJDb-OzfWcs9pU51YB3UMeSuYt9wz_9TKKirN2kveE39ZSH2A@mail.gmail.com>
Message-ID: <CAGE7PN+k749eWO7o4UNs4evTTYAdUb4M48f7w7V1LffbP1zwrg@mail.gmail.com>

RHEL 7 (the *current* release), Debian Jessie (oldstable) and Ubuntu 14.04
(old LTS supported in "maintenance" mode until early 2019 -
https://www.ubuntu.com/info/release-end-of-life) all shipped with 1.0.1
based OpenSSL. :(

IMNSHO *I still think we should do this to 3.7*.  OpenSSL >=1.0.2 provides
a much more usable API for modern security standards.  If we set our
standards based on the most conservative OS distro out there, we're just
holding ourselves back.

Isn't MacOS also in a lousy state when it comes to OpenSSL?  Did we switch
to providing our own there already?  We could do the same (ship our own)
when building on stale distros.

For my own buildbots, I'll figure out how to upgrade any that need it.
http://buildbot.python.org/all/buildslaves/gps-ubuntu-exynos5-armv7l will
be the interesting one for me to upgrade but it isn't one of our stable
builders so I won't treat it as urgent. :)

-gps

PS  Possibly relevant for RHEL 7 users -
https://access.redhat.com/solutions/2740151 (i don't have a login so i
can't read it, and it just says "in progress").

On Mon, Sep 4, 2017 at 11:33 AM Zachary Ware <zachary.ware+pydev at gmail.com>
wrote:

> Hi folks,
>
> Since OpenSSL 1.0.1 was EOL'd at the end of 2016, we're looking into
> whether we can drop support for OpenSSL 1.0.1 and earlier in Python
> 3.7.  However, it looks like many of our builders are still using as
> far back as 1.0.1e.  If you would, please check to make sure your
> builders are using the latest OpenSSL available from your distributor
> and let us know if it is not at least 1.0.2 so we can gauge how to
> proceed.
>
> Thanks,
> --
> Zach
> _______________________________________________
> Python-Buildbots mailing list
> Python-Buildbots at python.org
> https://mail.python.org/mailman/listinfo/python-buildbots
>

From rosuav at gmail.com  Mon Sep  4 16:08:38 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 Sep 2017 06:08:38 +1000
Subject: [Python-buildbots] OpenSSL versions on builders
In-Reply-To: <CAGE7PN+k749eWO7o4UNs4evTTYAdUb4M48f7w7V1LffbP1zwrg@mail.gmail.com>
References: <CAKJDb-OzfWcs9pU51YB3UMeSuYt9wz_9TKKirN2kveE39ZSH2A@mail.gmail.com>
 <CAGE7PN+k749eWO7o4UNs4evTTYAdUb4M48f7w7V1LffbP1zwrg@mail.gmail.com>
Message-ID: <CAPTjJmqOwp0nuMojduEwmgePtormh3vYxPp2cUZ4EfuSPkpunQ@mail.gmail.com>

On Tue, Sep 5, 2017 at 5:49 AM, Gregory P. Smith <greg at krypto.org> wrote:
> RHEL 7 (the *current* release), Debian Jessie (oldstable) and Ubuntu 14.04
> (old LTS supported in "maintenance" mode until early 2019 -
> https://www.ubuntu.com/info/release-end-of-life) all shipped with 1.0.1
> based OpenSSL. :(
>
> IMNSHO *I still think we should do this to 3.7*.  OpenSSL >=1.0.2 provides
> a much more usable API for modern security standards.  If we set our
> standards based on the most conservative OS distro out there, we're just
> holding ourselves back.

Does that mean that 3.7 won't be easily able to be deployed, even from
source, on the current RHEL? I've generally followed a policy of "use
the stable OS but then altinstall a newer Python if I want one" -
getting the advantage of a dependable OS distro (Debian in my case,
but same diff) while still using the latest Python for my own personal
work. For Debian and Ubuntu, this change will mean that people have to
switch to the latest stable before building Py3.7; for Red Hat, will
people need to install a second OpenSSL? And if so, is that easy or
hard?

Regarding the buildbot specifically: the Angelico bot is currently
running Debian Jessie, and therefore has 1.0.1. I could upgrade that,
but will wait on a decision wrt 3.7 support first - if Python 3.7 is
going to support Jessie, I'll keep the bot on Jessie.

ChrisA

From christian at python.org  Mon Sep  4 16:16:08 2017
From: christian at python.org (Christian Heimes)
Date: Mon, 4 Sep 2017 13:16:08 -0700
Subject: [Python-buildbots] OpenSSL versions on builders
In-Reply-To: <CAGE7PN+k749eWO7o4UNs4evTTYAdUb4M48f7w7V1LffbP1zwrg@mail.gmail.com>
References: <CAKJDb-OzfWcs9pU51YB3UMeSuYt9wz_9TKKirN2kveE39ZSH2A@mail.gmail.com>
 <CAGE7PN+k749eWO7o4UNs4evTTYAdUb4M48f7w7V1LffbP1zwrg@mail.gmail.com>
Message-ID: <ed8c07e7-6735-eaef-7da6-99c45cf7aeff@python.org>

On 2017-09-04 12:49, Gregory P. Smith wrote:
> RHEL 7 (the *current* release), Debian Jessie (oldstable) and Ubuntu
> 14.04 (old LTS supported in "maintenance" mode until early 2019 -
> https://www.ubuntu.com/info/release-end-of-life) all shipped with 1.0.1
> based OpenSSL. :(

RHEL 7.4 comes with OpenSSL 1.0.2:

https://access.redhat.com/errata/RHBA-2017:1929
https://bugzilla.redhat.com/show_bug.cgi?id=1276310

> IMNSHO *I still think we should do this to 3.7*.? OpenSSL >=1.0.2
> provides a much more usable API for modern security standards.? If we
> set our standards based on the most conservative OS distro out there,
> we're just holding ourselves back.

That's the gist of my upcoming PEP. For one, I want to replace
ssl.match_hostname() with OpenSSL's hostname verification API. It solves
several issues. The API is available since OpenSSL 1.0.2 and LibreSSL 2.5.3.

The PEP is still in proto state, e.g. missing several chapters and has
bunch of grammar and spelling errors:
https://github.com/tiran/peps/blob/sslmod/pep-9999.txt

> Isn't MacOS also in a lousy state when it comes to OpenSSL?? Did we
> switch to providing our own there already?? We could do the same (ship
> our own) when building on stale distros.

Ned has changed our macOS packages to use our own build of OpenSSL. IIRC
it's 1.0.2 and not 1.1.0. 1.1.0 was deemed too new when 3.6.0 came out.
Also Apple no longer ships header files for OpenSSL. Apple's own Python
binary use a private copy of an old LibreSSL version.

Christian

From victor.stinner at gmail.com  Mon Sep  4 16:20:48 2017
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 4 Sep 2017 22:20:48 +0200
Subject: [Python-buildbots] OpenSSL versions on builders
In-Reply-To: <ed8c07e7-6735-eaef-7da6-99c45cf7aeff@python.org>
References: <CAKJDb-OzfWcs9pU51YB3UMeSuYt9wz_9TKKirN2kveE39ZSH2A@mail.gmail.com>
 <CAGE7PN+k749eWO7o4UNs4evTTYAdUb4M48f7w7V1LffbP1zwrg@mail.gmail.com>
 <ed8c07e7-6735-eaef-7da6-99c45cf7aeff@python.org>
Message-ID: <CAMpsgwYE_H4BD5mAPGQydVAQ+Y_p=rd73680KNZjvrJ4Spka8w@mail.gmail.com>

2017-09-04 22:16 GMT+02:00 Christian Heimes <christian at python.org>:
> RHEL 7.4 comes with OpenSSL 1.0.2:
>
> https://access.redhat.com/errata/RHBA-2017:1929
> https://bugzilla.redhat.com/show_bug.cgi?id=1276310

"Description of problem: Node.js v4.2 depends on OpenSSL 1.0.2 which
is not present in RHEL"

Python is not alone to want to require OpenSSL 1.0.2+ :-) You can put
that in your PEP, Christian ;-)

Victor

From zachary.ware+pydev at gmail.com  Fri Sep  8 17:49:12 2017
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Fri, 8 Sep 2017 16:49:12 -0500
Subject: [Python-buildbots] Linux buildbots with the vm_socket.h kernel
 header?
In-Reply-To: <0101015d04020999-1aabee75-e6aa-402d-86c3-5b6a5ba7c979-000000@us-west-2.amazonses.com>
References: <CADiSq7fk9ZeaJqf7Fe=HbtGtQoO7WDqMBSdRObS4WG3X9PNgEg@mail.gmail.com>
 <0101015d04020999-1aabee75-e6aa-402d-86c3-5b6a5ba7c979-000000@us-west-2.amazonses.com>
Message-ID: <CAKJDb-ObX95kmTceg_X59Qq6VHTL6CpUyApcUnzYNMNUhrHJeg@mail.gmail.com>

On Sun, Jul 2, 2017 at 10:53 AM, Matt Billenstein <matt at vazor.com> wrote:
> I have an Ubuntu 16.04 box I could run a builder on.

Sorry it's taken this long for me to get back to this.  If you're
still interested in running that bot, send me a private mail and I'll
get you set up.

-- 
Zach

From rodrigc at crodrigues.org  Wed Sep 20 06:27:59 2017
From: rodrigc at crodrigues.org (Craig Rodrigues)
Date: Wed, 20 Sep 2017 03:27:59 -0700
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
Message-ID: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>

In the past two years, I've put a lot of effort into porting the following
to Python 3:

Twisted:
https://www.slideshare.net/CraigRodrigues1/the-onward-
journey-porting-twisted-to-python-3

Buildbot:
https://github.com/buildbot/buildbot/pulls?utf8=%E2%9C%93&
q=is%3Aclosed%20is%3Apr%20author%3Arodrigc%20


I would like to help Python.org move to Buildbot 9 + Python 3 for the
following
reasons:

(1)  Python.org currently uses Buildbot 0.8.14.  This branch of buildbot is
not under active development and is not being ported to Python 3.

(2)  The current version of Buildbot (0.9.11) is on the main branch which
is actively maintained.  This branch has been ported to Python 3.

(3)  Buildbot 0.8.14 is not actively being maintained.

(4)  Buildbot 0.9.x is actively receiving updates for new features such as
Docker support, and support for Hyper.sh.


A while back, I worked with Zachary Ware, who set up a proof of concept of
running Buildbot 9 on Python.org.

Zachary set this up:

https://buildbot.python.org/test/

That is an instance running Buildbot 9, using Python 3.4.3 and Twisted
17.5.0.

This is only a proof of concept, and is not actively being used or worked
on.


What are the next steps to completely convert Python.org to use Buildbot 9?
Who can help with this?

--
Craig

From rdmurray at bitdance.com  Wed Sep 20 11:02:49 2017
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Sep 2017 11:02:49 -0400
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
Message-ID: <20170920150252.CB3E91B10028@webabinitio.net>

The question most relevant to this list is, what are the consequences on
the slave maintainers?  Assuming we convert the master, will the slaves
be required to run python3/0.9?  (That would certainly be desirable,
but would it be required?)

Are there advantages for the slave maintainers in the new version?

Getting python3 on some of the platforms may be trickier than on others.

In response to your posting on core-mentorship Victor mentioned there
were issues with the proof-of-concept, so figuring out those issues is
presumably the first step.

--David

From dje.gcc at gmail.com  Wed Sep 20 11:16:40 2017
From: dje.gcc at gmail.com (David Edelsohn)
Date: Wed, 20 Sep 2017 11:16:40 -0400
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
Message-ID: <CAGWvny=Q4-66njoekvjDqOqRowxzFuY3sz+9psU0AiZ7o_HweQ@mail.gmail.com>

When another project tried to convert from Buildbot 0.8 to 0.9 earlier
this year, it was a total disaster and had to be reverted.  I don't
know if 0.9 has improved substantially since then.

My experience is that 0.9 invested a lot of effort to make the UI
pretty, but it was much less usable. It also placed an order of
magnitude higher computational burden on the master. An under-powered
master could not be effectively queried and builds started to fall
behind.

Thanks, David

On Wed, Sep 20, 2017 at 6:27 AM, Craig Rodrigues <rodrigc at crodrigues.org> wrote:
> In the past two years, I've put a lot of effort into porting the following
> to Python 3:
>
> Twisted:
> https://www.slideshare.net/CraigRodrigues1/the-onward-
> journey-porting-twisted-to-python-3
>
> Buildbot:
> https://github.com/buildbot/buildbot/pulls?utf8=%E2%9C%93&
> q=is%3Aclosed%20is%3Apr%20author%3Arodrigc%20
>
>
> I would like to help Python.org move to Buildbot 9 + Python 3 for the
> following
> reasons:
>
> (1)  Python.org currently uses Buildbot 0.8.14.  This branch of buildbot is
> not under active development and is not being ported to Python 3.
>
> (2)  The current version of Buildbot (0.9.11) is on the main branch which
> is actively maintained.  This branch has been ported to Python 3.
>
> (3)  Buildbot 0.8.14 is not actively being maintained.
>
> (4)  Buildbot 0.9.x is actively receiving updates for new features such as
> Docker support, and support for Hyper.sh.
>
>
> A while back, I worked with Zachary Ware, who set up a proof of concept of
> running Buildbot 9 on Python.org.
>
> Zachary set this up:
>
> https://buildbot.python.org/test/
>
> That is an instance running Buildbot 9, using Python 3.4.3 and Twisted
> 17.5.0.
>
> This is only a proof of concept, and is not actively being used or worked
> on.
>
>
> What are the next steps to completely convert Python.org to use Buildbot 9?
> Who can help with this?
>
> --
> Craig
> _______________________________________________
> Python-Buildbots mailing list
> Python-Buildbots at python.org
> https://mail.python.org/mailman/listinfo/python-buildbots

From rodrigc at crodrigues.org  Wed Sep 20 13:49:06 2017
From: rodrigc at crodrigues.org (Craig Rodrigues)
Date: Wed, 20 Sep 2017 10:49:06 -0700
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAGWvny=Q4-66njoekvjDqOqRowxzFuY3sz+9psU0AiZ7o_HweQ@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <CAGWvny=Q4-66njoekvjDqOqRowxzFuY3sz+9psU0AiZ7o_HweQ@mail.gmail.com>
Message-ID: <CAG=rPVf6=LJoGLsYskTEV83n3+hc=OMTymJMXoVZL6MmqT_dcg@mail.gmail.com>

I got involved with the Buildbot project only this year (focusing on Python
3 porting).

I am aware of previous reports of performance problems with buildbot nine,
but that is a bit before my time.

I believe that the earlier performance problems have been addressed.
On the buildbot-users mailing list, there are some people in companies
who have set up buildbot nine and have a master handling hundreds of builds.

The references implementation of buildbot is being actively used to build
buildbot
itself, and highlights integration with GitHub Pull Requests:

https://nine.buildbot.net/#/console
--
Craig


On Wed, Sep 20, 2017 at 8:16 AM, David Edelsohn <dje.gcc at gmail.com> wrote:

> When another project tried to convert from Buildbot 0.8 to 0.9 earlier
> this year, it was a total disaster and had to be reverted.  I don't
> know if 0.9 has improved substantially since then.
>
> My experience is that 0.9 invested a lot of effort to make the UI
> pretty, but it was much less usable. It also placed an order of
> magnitude higher computational burden on the master. An under-powered
> master could not be effectively queried and builds started to fall
> behind.
>
> Thanks, David
>
> On Wed, Sep 20, 2017 at 6:27 AM, Craig Rodrigues <rodrigc at crodrigues.org>
> wrote:
> > In the past two years, I've put a lot of effort into porting the
> following
> > to Python 3:
> >
> > Twisted:
> > https://www.slideshare.net/CraigRodrigues1/the-onward-
> > journey-porting-twisted-to-python-3
> >
> > Buildbot:
> > https://github.com/buildbot/buildbot/pulls?utf8=%E2%9C%93&
> > q=is%3Aclosed%20is%3Apr%20author%3Arodrigc%20
> >
> >
> > I would like to help Python.org move to Buildbot 9 + Python 3 for the
> > following
> > reasons:
> >
> > (1)  Python.org currently uses Buildbot 0.8.14.  This branch of buildbot
> is
> > not under active development and is not being ported to Python 3.
> >
> > (2)  The current version of Buildbot (0.9.11) is on the main branch which
> > is actively maintained.  This branch has been ported to Python 3.
> >
> > (3)  Buildbot 0.8.14 is not actively being maintained.
> >
> > (4)  Buildbot 0.9.x is actively receiving updates for new features such
> as
> > Docker support, and support for Hyper.sh.
> >
> >
> > A while back, I worked with Zachary Ware, who set up a proof of concept
> of
> > running Buildbot 9 on Python.org.
> >
> > Zachary set this up:
> >
> > https://buildbot.python.org/test/
> >
> > That is an instance running Buildbot 9, using Python 3.4.3 and Twisted
> > 17.5.0.
> >
> > This is only a proof of concept, and is not actively being used or worked
> > on.
> >
> >
> > What are the next steps to completely convert Python.org to use Buildbot
> 9?
> > Who can help with this?
> >
> > --
> > Craig
> > _______________________________________________
> > Python-Buildbots mailing list
> > Python-Buildbots at python.org
> > https://mail.python.org/mailman/listinfo/python-buildbots
>

From ncoghlan at gmail.com  Wed Sep 20 20:20:25 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Sep 2017 10:20:25 +1000
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAG=rPVf6=LJoGLsYskTEV83n3+hc=OMTymJMXoVZL6MmqT_dcg@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <CAGWvny=Q4-66njoekvjDqOqRowxzFuY3sz+9psU0AiZ7o_HweQ@mail.gmail.com>
 <CAG=rPVf6=LJoGLsYskTEV83n3+hc=OMTymJMXoVZL6MmqT_dcg@mail.gmail.com>
Message-ID: <CADiSq7e5a59AhUNp903DhwSKL2Qg5WdxVpqMcFvUakdjf1Oo=Q@mail.gmail.com>

On 21 September 2017 at 03:49, Craig Rodrigues <rodrigc at crodrigues.org> wrote:
> The references implementation of buildbot is being actively used to build
> buildbot
> itself, and highlights integration with GitHub Pull Requests:

Just noting that while the PR support in Buildbot 0.9 is definitely a
nice feature, we can't use it directly in its current form, as we
promise folks contributing workers to the BuildBot fleet that we'll
only run approved code on their systems. Since PRs contain
*un*approved code, we'd need the Buildbot trigger to be a "test this
on the Buildbot fleet" comment from a reviewer with merge privileges,
rather than something that happens automatically for every PR.

The other reason an approach like that would be better for us is that
for the majority of changes, the risk of cross-platform compatibility
issues is genuinely low, so it makes more sense to go with pre-merge
CI on Linux and Windows, and only run post-merge testing across the
entire fleet)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rodrigc at crodrigues.org  Thu Sep 21 13:41:12 2017
From: rodrigc at crodrigues.org (Craig Rodrigues)
Date: Thu, 21 Sep 2017 10:41:12 -0700
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <20170920150252.CB3E91B10028@webabinitio.net>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <20170920150252.CB3E91B10028@webabinitio.net>
Message-ID: <CAG=rPVfn=uKmsfnASUT8i7sB9hv3FxwrSEcZGDDcYPzkBg9bpQ@mail.gmail.com>

On Wed, Sep 20, 2017 at 8:02 AM, R. David Murray <rdmurray at bitdance.com>
wrote:

>
> In response to your posting on core-mentorship Victor mentioned there
> were issues with the proof-of-concept, so figuring out those issues is
> presumably the first step.
>
>
Zach reported his progress of setting up http://buildbot.python.org/test/ in
this thread:

https://github.com/python/buildmaster-config/pull/12

In June, he reported several problems.  These problems have all been fixed.
In September he updated http://buildbot.python.org/test/  and got it
working on Python 3.  He was able to do some simple builds.
He shut down the worker, because he was performing this test
on his own VM which has limited resources.

If there are additional problems, Zach will have to provide details.

I'm happy to help look into any problems, but you'll have to tell me what
those
problems are for me to investigate.
--
Craig

From zachary.ware+pydev at gmail.com  Thu Sep 21 14:12:16 2017
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Thu, 21 Sep 2017 13:12:16 -0500
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAG=rPVfn=uKmsfnASUT8i7sB9hv3FxwrSEcZGDDcYPzkBg9bpQ@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <20170920150252.CB3E91B10028@webabinitio.net>
 <CAG=rPVfn=uKmsfnASUT8i7sB9hv3FxwrSEcZGDDcYPzkBg9bpQ@mail.gmail.com>
Message-ID: <CAKJDb-OypQFYLiNu-owP-w5yF1ZBHQwy_1PE6yRbBiKZT-hQzA@mail.gmail.com>

Hi Craig,

Sorry I've been largely unresponsive; time has been hard to come by lately.

On Thu, Sep 21, 2017 at 12:41 PM, Craig Rodrigues
<rodrigc at crodrigues.org> wrote:
> I'm happy to help look into any problems, but you'll have to tell me what
> those
> problems are for me to investigate.

For starters, there is still a worker connected to the test instance,
which seems to have been failing the daily refleak builds with an
exception since I connected it.  However, there's no way to see what
that exception is from the web UI; it looks like it just doesn't load.
I don't think there's anything particularly exotic in the config to
trip it up, so it looks to me like a bug in buildbot (possibly related
to running on Python 3).

That seems to be a theme with this test instance; pretty much
everything I click on looks incomplete whether logged in or not, and
no errors appear in the logs.  Is there something missing/broken in
the configuration?

-- 
Zach

From rodrigc at crodrigues.org  Thu Sep 21 14:30:37 2017
From: rodrigc at crodrigues.org (Craig Rodrigues)
Date: Thu, 21 Sep 2017 11:30:37 -0700
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAKJDb-OypQFYLiNu-owP-w5yF1ZBHQwy_1PE6yRbBiKZT-hQzA@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <20170920150252.CB3E91B10028@webabinitio.net>
 <CAG=rPVfn=uKmsfnASUT8i7sB9hv3FxwrSEcZGDDcYPzkBg9bpQ@mail.gmail.com>
 <CAKJDb-OypQFYLiNu-owP-w5yF1ZBHQwy_1PE6yRbBiKZT-hQzA@mail.gmail.com>
Message-ID: <CAG=rPVdTr9SWu-hfnaaUysut5HREp6UMiiebpTBTzz4i5z2mUg@mail.gmail.com>

On Thu, Sep 21, 2017 at 11:12 AM, Zachary Ware <zachary.ware+pydev at gmail.com
> wrote:

>
> That seems to be a theme with this test instance; pretty much
> everything I click on looks incomplete whether logged in or not, and
> no errors appear in the logs.  Is there something missing/broken in
> the configuration?
>

I'm not sure.
If you can make the twistd.log files on the worker and master available,
there might be some error messages in there that will help shed light on
the problem.

--
Craig

From koobs at FreeBSD.org  Fri Sep 22 08:01:27 2017
From: koobs at FreeBSD.org (Kubilay Kocak)
Date: Fri, 22 Sep 2017 22:01:27 +1000
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <CAKJDb-OypQFYLiNu-owP-w5yF1ZBHQwy_1PE6yRbBiKZT-hQzA@mail.gmail.com>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <20170920150252.CB3E91B10028@webabinitio.net>
 <CAG=rPVfn=uKmsfnASUT8i7sB9hv3FxwrSEcZGDDcYPzkBg9bpQ@mail.gmail.com>
 <CAKJDb-OypQFYLiNu-owP-w5yF1ZBHQwy_1PE6yRbBiKZT-hQzA@mail.gmail.com>
Message-ID: <85b5808e-150e-1bab-acda-3bc57d15490e@FreeBSD.org>

On 9/22/17 4:12 AM, Zachary Ware wrote:
> Hi Craig,
> 
> Sorry I've been largely unresponsive; time has been hard to come by lately.
> 
> On Thu, Sep 21, 2017 at 12:41 PM, Craig Rodrigues
> <rodrigc at crodrigues.org> wrote:
>> I'm happy to help look into any problems, but you'll have to tell me what
>> those
>> problems are for me to investigate.
> 
> For starters, there is still a worker connected to the test instance,
> which seems to have been failing the daily refleak builds with an
> exception since I connected it.  However, there's no way to see what
> that exception is from the web UI; it looks like it just doesn't load.
> I don't think there's anything particularly exotic in the config to
> trip it up, so it looks to me like a bug in buildbot (possibly related
> to running on Python 3).
> 
> That seems to be a theme with this test instance; pretty much
> everything I click on looks incomplete whether logged in or not, and
> no errors appear in the logs.  Is there something missing/broken in
> the configuration?
> 

I can provision a Python 3 buildbot 0.9 worker upon request if that
would help ease/verify/test the migration.

The FreeBSD buildbot port/package was updated to 0.9 a while ago, and
given the lack of backward compatibility (with a 0.8 master), I've had
to hold off on upgrades. Getting the 0.9 migration done sooner would for
me result in less management and support overhead than keeping it as is,
effectively an unsupported state downstream.

I would also be happy to switch the existing 2 'stable' FreeBSD (0.8)
workers over to 0.9 and a 0.9 server at any time, but would want to
ensure that doing so wouldn't:

- reduce branch/build-type coverage the 0.8 server config provides
- reduce visibility of their statuses to developers. (see them go and
stay red)

--
Regards,

./koobs

From rodrigc at crodrigues.org  Fri Sep 22 10:48:16 2017
From: rodrigc at crodrigues.org (Craig Rodrigues)
Date: Fri, 22 Sep 2017 07:48:16 -0700
Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3
In-Reply-To: <20170920150252.CB3E91B10028@webabinitio.net>
References: <CAG=rPVejnnDtWY5sQwpZHzo-q7sinOO0f-qfwpwRvW3N2z9LzQ@mail.gmail.com>
 <20170920150252.CB3E91B10028@webabinitio.net>
Message-ID: <CAG=rPVci+Ks2D3NqTk7HnWeqMMhjy9uOGOMS98qb5_7=0Ae4Vw@mail.gmail.com>

On Wed, Sep 20, 2017 at 8:02 AM, R. David Murray <rdmurray at bitdance.com>
wrote:

> The question most relevant to this list is, what are the consequences on
> the slave maintainers?  Assuming we convert the master, will the slaves
> be required to run python3/0.9?  (That would certainly be desirable,
> but would it be required?)


> Are there advantages for the slave maintainers in the new version?
>


In Buildbot 9, the terminology has changed from "slave" to "worker", so I
will use that term. :)

I would like to propose the following strategy for Python.org's buildbot
upgrade strategy.

(1)  The master will be upgraded to Buildbot Nine, running under Python 3.

(2)  The existing buildbot workers will remain unchanged running at their
current 0.8.x versions under Python 2.

(3)  Over time, the buildbot workers can be upgraded to 0.9.x under Python
2 or 3, as is convenient for
      the maintainers of those workers.


There is no rush on this, and I am willing to put the effort to push fixes
into Buildbot and Twisted
in order to get this to work.

--
Craig