From Peixing.Xin at windriver.com  Thu Feb 21 03:33:49 2019
From: Peixing.Xin at windriver.com (Xin, Peixing)
Date: Thu, 21 Feb 2019 08:33:49 +0000
Subject: [Python-buildbots] The process of adding my buildbot worker to the
 buildbot fleet
Message-ID: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>

Hi, Experts:

I want to add a new platform support for cpython, that is VxWorks RTOS. A builtbot worker has been internally setup ready to join the fleet. But any process or guidance I need to follow? Where is the master.cfg located? And who and how to modify that? I checked the source code repo but no any file called master.cfg there.

Thanks,
Peixing


From zachary.ware+pydev at gmail.com  Thu Feb 21 08:38:15 2019
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Thu, 21 Feb 2019 07:38:15 -0600
Subject: [Python-buildbots] The process of adding my buildbot worker to
 the buildbot fleet
In-Reply-To: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>
References: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>
Message-ID: <CAKJDb-O-z8m8fp7Qqmkzmw+X15CtNGw6byNcLvRAcTaCLwamuQ@mail.gmail.com>

Hi Peixing,

On Thu, Feb 21, 2019 at 7:23 AM Xin, Peixing <Peixing.Xin at windriver.com> wrote:
> I want to add a new platform support for cpython, that is VxWorks RTOS. A builtbot worker has been internally setup ready to join the fleet. But any process or guidance I need to follow? Where is the master.cfg located? And who and how to modify that? I checked the source code repo but no any file called master.cfg there.

The buildbot master configuration can be found here:
https://github.com/python/buildmaster-config/tree/master/master,
please feel free to open a pull request there adding your builder(s).
Once we're ready to merge that, I'll contact you privately with
credentials to connect.

-- 
Zach

From vstinner at redhat.com  Thu Feb 21 09:06:05 2019
From: vstinner at redhat.com (Victor Stinner)
Date: Thu, 21 Feb 2019 15:06:05 +0100
Subject: [Python-buildbots] Any advice for the good "size" of a Python
 buildbot worker?
Message-ID: <CA+3bQGEGbO=dwDQcYKWs+GyT5ZX1Fa49uGYPOPNX6d4ZV7po=A@mail.gmail.com>

Hi,

We would like to add more buildbot workers for test Python on Fedora
and RHEL, but I have no idea of the ideal "size" of a worker:

* Number of CPUs? I guess that 2 CPUs is nice but 4 or more is better?
* Memory size? At least 8 GB?
* Minimum storage? Logs are sent to the buildbot server, so there is
no much stored for long term, no? On my laptop, my python source
directory takes around 500 MiB including ".git" repository and object
files (.o, .so, python binary, etc.) with Python compiled in debug
mode (keep debug symbols) I would say that 10 GiB is nice, 20 GiB is
better?

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.

From zachary.ware+pydev at gmail.com  Thu Feb 21 10:11:43 2019
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Thu, 21 Feb 2019 09:11:43 -0600
Subject: [Python-buildbots] Any advice for the good "size" of a Python
 buildbot worker?
In-Reply-To: <CA+3bQGEGbO=dwDQcYKWs+GyT5ZX1Fa49uGYPOPNX6d4ZV7po=A@mail.gmail.com>
References: <CA+3bQGEGbO=dwDQcYKWs+GyT5ZX1Fa49uGYPOPNX6d4ZV7po=A@mail.gmail.com>
Message-ID: <CAKJDb-M8qN5F72CA7EGYgDNgiufedR_bm+7rRgerggXA95HBjA@mail.gmail.com>

On Thu, Feb 21, 2019 at 8:06 AM Victor Stinner <vstinner at redhat.com> wrote:
> We would like to add more buildbot workers for test Python on Fedora
> and RHEL, but I have no idea of the ideal "size" of a worker:

It basically comes down to how long you want to wait for results :).
The devguide lists [1] 2 cpu, 512 MB RAM and 30GB storage as the
preferred minimums, but that may or may not actually be enough
anymore, especially on RAM.

For reference, here are stats on my workers (formatting may be
destroyed by Gmail, sorry if so):

+-----------------------------------------------------------------------------------------------------+
|                     | Type                  | CPU
       | RAM   | Disk (GB)     |
|                     |                       | cores | threads | freq
(GHz)  | (GB)  | Total  | Free |
+-----------------------------------------------------------------------------------------------------+
| ware-gentoo-x86     | Bare metal            | 2     | 2       | 2.6
       | 2     | 52     | 37   |
| ware-alpine         | Docker on physical    | 4     | 8       |
3.4-3.9     | 16    | 108    | 66   |
| ware-win81-release  | Azure VM              | 2     | 2       | 2.39
       | 14    | 100    | 91   |
+-----------------------------------------------------------------------------------------------------+

[1] https://devguide.python.org/buildworker/#required-resources

-- 
Zach

From Peixing.Xin at windriver.com  Mon Feb 25 22:48:05 2019
From: Peixing.Xin at windriver.com (Xin, Peixing)
Date: Tue, 26 Feb 2019 03:48:05 +0000
Subject: [Python-buildbots] The process of adding my buildbot worker to
 the buildbot fleet
In-Reply-To: <CAKJDb-O-z8m8fp7Qqmkzmw+X15CtNGw6byNcLvRAcTaCLwamuQ@mail.gmail.com>
References: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>
 <CAKJDb-O-z8m8fp7Qqmkzmw+X15CtNGw6byNcLvRAcTaCLwamuQ@mail.gmail.com>
Message-ID: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B83099@ALA-MBD.corp.ad.wrs.com>

Hi, Ware:

Is public network IP required for the buildbot worker? Currently I am putting it inside my company's local network. Do I need to move it outside to expose it on the Internet directly?

Thanks,
Peixing


-----Original Message-----
From: Python-Buildbots [mailto:python-buildbots-bounces+peixing.xin=windriver.com at python.org] On Behalf Of Zachary Ware
Sent: Thursday, February 21, 2019 9:38 PM
To: python-buildbots at python.org
Subject: Re: [Python-buildbots] The process of adding my buildbot worker to the buildbot fleet

Hi Peixing,

On Thu, Feb 21, 2019 at 7:23 AM Xin, Peixing <Peixing.Xin at windriver.com> wrote:
> I want to add a new platform support for cpython, that is VxWorks RTOS. A builtbot worker has been internally setup ready to join the fleet. But any process or guidance I need to follow? Where is the master.cfg located? And who and how to modify that? I checked the source code repo but no any file called master.cfg there.

The buildbot master configuration can be found here:
https://github.com/python/buildmaster-config/tree/master/master,
please feel free to open a pull request there adding your builder(s).
Once we're ready to merge that, I'll contact you privately with
credentials to connect.

-- 
Zach
_______________________________________________
Python-Buildbots mailing list
Python-Buildbots at python.org
https://mail.python.org/mailman/listinfo/python-buildbots

From rosuav at gmail.com  Mon Feb 25 23:30:52 2019
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 26 Feb 2019 15:30:52 +1100
Subject: [Python-buildbots] The process of adding my buildbot worker to
 the buildbot fleet
In-Reply-To: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B83099@ALA-MBD.corp.ad.wrs.com>
References: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>
 <CAKJDb-O-z8m8fp7Qqmkzmw+X15CtNGw6byNcLvRAcTaCLwamuQ@mail.gmail.com>
 <8488FBC4EAAC5941BA4B85DD1ECCF1870133B83099@ALA-MBD.corp.ad.wrs.com>
Message-ID: <CAPTjJmocaZogDNyOrDtAdt2qvqf+3HgLissGsE4wC2ik97yF8g@mail.gmail.com>

On Tue, Feb 26, 2019 at 3:00 PM Xin, Peixing <Peixing.Xin at windriver.com> wrote:
>
> Hi, Ware:
>
> Is public network IP required for the buildbot worker? Currently I am putting it inside my company's local network. Do I need to move it outside to expose it on the Internet directly?
>

Shouldn't be. Mine is a VM that's natted behind VirtualBox. It will
need to be able to connect out to the internet, but shouldn't need any
incoming connections.

ChrisA

From zachary.ware+pydev at gmail.com  Mon Feb 25 23:34:19 2019
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 25 Feb 2019 22:34:19 -0600
Subject: [Python-buildbots] The process of adding my buildbot worker to
 the buildbot fleet
In-Reply-To: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B83099@ALA-MBD.corp.ad.wrs.com>
References: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>
 <CAKJDb-O-z8m8fp7Qqmkzmw+X15CtNGw6byNcLvRAcTaCLwamuQ@mail.gmail.com>
 <8488FBC4EAAC5941BA4B85DD1ECCF1870133B83099@ALA-MBD.corp.ad.wrs.com>
Message-ID: <CAKJDb-Oc9FD2z3qh6-czgUJNakjPxQTT6HeCvrFxcf+-99mpsQ@mail.gmail.com>

On Mon, Feb 25, 2019 at 9:48 PM Xin, Peixing <Peixing.Xin at windriver.com> wrote:
> Is public network IP required for the buildbot worker? Currently I am putting it inside my company's local network. Do I need to move it outside to expose it on the Internet directly?

No public address necessary; all connections are initiated from the
worker.  See https://devguide.python.org/buildworker/#required-ports
for more (slightly but not substantially out-of-date) information
about network configuration (in short, if you can reach the internet,
don't worry about it :)).

Regards,
Zach

From ncoghlan at gmail.com  Tue Feb 26 04:00:04 2019
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 26 Feb 2019 19:00:04 +1000
Subject: [Python-buildbots] The process of adding my buildbot worker to
 the buildbot fleet
In-Reply-To: <CAKJDb-Oc9FD2z3qh6-czgUJNakjPxQTT6HeCvrFxcf+-99mpsQ@mail.gmail.com>
References: <8488FBC4EAAC5941BA4B85DD1ECCF1870133B7F566@ALA-MBD.corp.ad.wrs.com>
 <CAKJDb-O-z8m8fp7Qqmkzmw+X15CtNGw6byNcLvRAcTaCLwamuQ@mail.gmail.com>
 <8488FBC4EAAC5941BA4B85DD1ECCF1870133B83099@ALA-MBD.corp.ad.wrs.com>
 <CAKJDb-Oc9FD2z3qh6-czgUJNakjPxQTT6HeCvrFxcf+-99mpsQ@mail.gmail.com>
Message-ID: <CADiSq7eYS9t_MB1Ex8e4JwVPgbDgn9hNmgwHS7b7aTEgALSYow@mail.gmail.com>

On Tue, 26 Feb 2019 at 14:34, Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
>
> On Mon, Feb 25, 2019 at 9:48 PM Xin, Peixing <Peixing.Xin at windriver.com> wrote:
> > Is public network IP required for the buildbot worker? Currently I am putting it inside my company's local network. Do I need to move it outside to expose it on the Internet directly?
>
> No public address necessary; all connections are initiated from the
> worker.  See https://devguide.python.org/buildworker/#required-ports
> for more (slightly but not substantially out-of-date) information
> about network configuration (in short, if you can reach the internet,
> don't worry about it :)).

Note though that it would still be good security practice to firewall
the buildbot away from the rest of your network such that it can
*only* reach the internet, and not the rest of your private intranet.

While the buildbots only run on commits that have been reviewed and
accepted by a core developer, there's always a chance of someone's
credentials getting compromised and problems being caused as a result.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia