[python-committers] Visual Studio Team Services checks on pullrequests

[Steve Dower]

Okay, now that it's morning and I have coffee, here's a full update on
what I've been doing (those at the language summit have heard some of
this already).

Visual Studio Team Services is Microsoft's integrated code/build/release
infrastructure service. The official marketing page is
https://www.visualstudio.com/team-services/ but you can think of it as
github code+github issues+Travis but engineered for huge teams (i.e.
Microsoft keeps all of Windows in git, all the millions of issues they
have, and all the builds they do, are in VSTS and it scales to thousands
of developers just fine).

I've been working with the team to improve their Python support and
generally raise awareness of the new service, and one of the things they
agreed to is to provide a free set of build machines for CPython. These
allow us to run up to 20 simultaneous Windows, macOS and Linux builds
with no other limits.

My main project at the PyCon US sprints has been setting these builds
up. As of about 16 hours ago the PR builds are now hooked up to github
(for an example, see https://github.com/python/cpython/pull/6933). They
are not required to merge, so don't feel the need to block PRs if
everything else passes, but we'd like to fix up the issues they are
hitting. As far as I can tell, most of the problems are transient test
failures that ought to be fixed anyway.

Brief interruption for some links:
* https://python.visualstudio.com/cpython/_build/index?_a=queued (queued
and recently completed builds)
* https://github.com/python/cpython/tree/master/.vsts (build definitions)
* https://github.com/python/cpython/pull/6933 (sample PR with build status)

Authentication for python.visualstudio.com is currently manually
managed, but everything relevant under
https://python.visualstudio.com/cpython is visible without
authentication (this is a new VSTS feature that we got enabled early).
Right now I don't want to add lots of people manually, and we'll be
looking at how to make this simpler in the future. There isn't much you
can do while logged in anyway :)

There are some missing features. I'm still in contact with the team and
I'll be passing along requests, so feel free to let me know if you need
anything. Currently I'm aware of:
* no way to requeue a PR build (whether logged in or not)
* no link from a build to the github PR page
* templated build steps aren't enabled yet (see linux-deps.yml if you're
interested)

We don't yet have a specific timeline for making VSTS builds required
and reducing Travis/AppVeyor usage. I also haven't set up VSTS for
Python 2.7, and honestly I'm fine with keeping the existing systems
there. As a slight aside, we're also working with some other projects to
provide similar setups (specifically Twisted and some PyPA-associated
projects), so if you think you'd benefit from this on one of your other
projects let me know and we can see what's available.

Also, if you start evaluating/using VSTS for other projects because of
this, *please* let me know. New users is what the VSTS team is trying to
achieve, so every time I can send back positive reports it helps
convince them to keep donating build time.

Feel free to email me with any questions or feedback, either on or off-list.

Cheers,
Steve