From rodrigc at crodrigues.org Fri Oct 6 15:35:44 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 6 Oct 2017 12:35:44 -0700 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: <20170920150252.CB3E91B10028@webabinitio.net> References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: On Wed, Sep 20, 2017 at 8:02 AM, R. David Murray wrote: > > In response to your posting on core-mentorship Victor mentioned there > were issues with the proof-of-concept, so figuring out those issues is > presumably the first step. > > I've been working with Zach quite a lot, and I believe we have solved the issues that Victor previously saw. If you look at the proof of concept at: http://buildbot.python.org/test/ This has been building things from https://github.com/zware/cpython which is a test fork of cpython. -- Craig From victor.stinner at gmail.com Fri Oct 6 16:20:40 2017 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 6 Oct 2017 22:20:40 +0200 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: 2017-10-06 21:35 GMT+02:00 Craig Rodrigues : > http://buildbot.python.org/test/ The new UI LGTM. It seems like email notifications are also working. So everything seems to be ready from my point of view. I propose right now to start moving (slowly) buildbot slaves from the old buildbot 0.8/Python 2 instance to the new buildbot 0.9/Python 3 instance. We can stop the migration if things go bad, or even rollback in the worst case. I'm fully in favor of moving to buildbot 0.9 since previously we used a fork of buildbot, and I hate to maintain a fork. We have limited resources working on buildbots: mostly Zachary Ware, R. David Murray and me. I'm taking any change which gives us less work to do for the long-term :-) We now have a major buildbot contributor, Craig Rodrigues, who offers his strong support for the migration. So if something goes wrong, we will harass him until he fixes buildbot :-D By the way, it like the idea of eating our own dog food: migrate from Python 2 to Python 3 :-) Victor From rodrigc at crodrigues.org Fri Oct 6 16:33:50 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 6 Oct 2017 13:33:50 -0700 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: On Fri, Oct 6, 2017 at 1:20 PM, Victor Stinner wrote: > > By the way, it like the idea of eating our own dog food: migrate from > Python 2 to Python 3 :-) > = I performed some interop testing between Python 2 buildbot worker and Python 3 buildbot master: https://lists.buildbot.net/pipermail/devel/2017-October/012433.html Based on my experiments, I think it is safe to perform the migration as follows: 1. Update the buildbot master to buildbot 9 + Python 3. Leave the buildbot workers/slaves alone. They should be able to successfully connect to the new buildbot master and work. 2. For the buildbot worker/slave maintainers, give them the following options: a. Upgrade the buildbot worker to buildbot 9 + Python 3 (MOST PREFERRED) b. Upgrade the buildbot worker to buildbot 9 + Python 2 (LESS PREFERRED, BUT WILL WORK) c. Leave the buildbot slave alone at buildbot 8 + Python 2 (LEAST PREFERRED, BUT WILL WORK) I did all my experiments on Linux + OS X, so we might want to double check with a Windows buildbot worker. I don't have Windows environments readily available for this type of interop testing. -- Craig From rodrigc at crodrigues.org Fri Oct 6 16:36:24 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 6 Oct 2017 13:36:24 -0700 Subject: [Python-buildbots] Buildbot 9 + Postgres backend Message-ID: I'd like to suggest that when we upgrade the buildbot master, we use a Postgres backend instead of sqlite: http://docs.buildbot.net/latest/manual/cfg-global.html#postgres This is for better performance. Since postgres is involved, this would require coordination with the Python infra team. -- Craig From zachary.ware+pydev at gmail.com Fri Oct 6 16:42:52 2017 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Fri, 6 Oct 2017 15:42:52 -0500 Subject: [Python-buildbots] Buildbot 9 + Postgres backend In-Reply-To: References: Message-ID: On Fri, Oct 6, 2017 at 3:36 PM, Craig Rodrigues wrote: > I'd like to suggest that when we upgrade the buildbot master, > we use a Postgres backend instead of sqlite: > > http://docs.buildbot.net/latest/manual/cfg-global.html#postgres Since it's such a simple change, we can just factor `db_uri` out to `local.py`, defaulting to sqlite, and we can switch to postgres at our leisure later. I'm not concerned about resetting the database to do so. -- Zach From db3l.net at gmail.com Fri Oct 6 16:46:52 2017 From: db3l.net at gmail.com (David Bolen) Date: Fri, 6 Oct 2017 16:46:52 -0400 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: On Fri, Oct 6, 2017 at 4:33 PM, Craig Rodrigues wrote: > Based on my experiments, I think it is safe to perform the migration as > follows: > > 1. Update the buildbot master to buildbot 9 + Python 3. Leave the > buildbot workers/slaves alone. They should be > able to successfully connect to the new buildbot master and work. > > 2. For the buildbot worker/slave maintainers, give them the following > options: > > a. Upgrade the buildbot worker to buildbot 9 + Python 3 (MOST > PREFERRED) > b. Upgrade the buildbot worker to buildbot 9 + Python 2 (LESS > PREFERRED, BUT WILL WORK) > c. Leave the buildbot slave alone at buildbot 8 + Python 2 (LEAST > PREFERRED, BUT WILL WORK) Yes, my own preference would be to do workers last. It sounds to me like most of the benefits of this change accrue on the master side (eliminating our custom version) and at least for my part, I'd prefer not to be critical path to allowing that upgrade to proceed. Over a longer period, I'm curious about the preferences - are there concrete benefits in our environment of 0.9 over 0.8 on the worker side? E.g., what does upgrading bring on the worker side? In some cases an upgrade is trivial, but I've still got a few local tweaks on the Windows side that, while probably not too difficult to port, incur cost only worth doing if there is a concrete requirement or benefit. Oh, and I can offer any of my Windows workers for interoperability testing if you'd like me to temporarily configure them to an additional master. At least one of them (XP) is probably not worth the effort of getting to (a) vs. (b). -- David From victor.stinner at gmail.com Fri Oct 6 16:53:12 2017 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 6 Oct 2017 22:53:12 +0200 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: I talked with Zach on IRC. He wants to move everything to buildbot nine right now. Well, if he takes the responsability, I'm totally fine with that. Let me elaborate. If a few workers are broken, we can skip them to give time to the buildbot owner to fix them, or give us time to fix their config. It's not a blocker issue. A blocker issue would be to break *everything*. This option seems unlikely since I'm seeing a working buildbot nine instance. Moreover, as Zach wrote me on IRC, we *can* revert to buildbot 0.8. So ok, let's move "everything" (the server) to buildbot nine! Yahoo! 2017-10-06 22:33 GMT+02:00 Craig Rodrigues : > Based on my experiments, I think it is safe to perform the migration as > follows: > > 1. Update the buildbot master to buildbot 9 + Python 3. Leave the > buildbot workers/slaves alone. They should be > able to successfully connect to the new buildbot master and work. Oh great, you confirmed that we can upgrade the server right now, and only upgrade workers later. Nice. Victor From rodrigc at crodrigues.org Fri Oct 6 17:12:46 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 6 Oct 2017 14:12:46 -0700 Subject: [Python-buildbots] Buildbot 9 + Postgres backend In-Reply-To: References: Message-ID: On Fri, Oct 6, 2017 at 1:42 PM, Zachary Ware wrote: > On Fri, Oct 6, 2017 at 3:36 PM, Craig Rodrigues > wrote: > > I'd like to suggest that when we upgrade the buildbot master, > > we use a Postgres backend instead of sqlite: > > > > http://docs.buildbot.net/latest/manual/cfg-global.html#postgres > > Since it's such a simple change, we can just factor `db_uri` out to > `local.py`, defaulting to sqlite, and we can switch to postgres at our > leisure later. I'm not concerned about resetting the database to do > so. > Sounds good to me. I just wanted to raise the issue early on in the process, so as to keep the python infra team in the loop and not leave them blindsided. -- Craig From rodrigc at crodrigues.org Fri Oct 6 17:40:30 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 6 Oct 2017 14:40:30 -0700 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: On Fri, Oct 6, 2017 at 1:46 PM, David Bolen wrote: > > Over a longer period, I'm curious about the preferences - are there > concrete benefits in our environment of 0.9 over 0.8 on the worker > side? E.g., what does upgrading bring on the worker side? > On the worker side, the are no performance improvements migrating a buildbot-worker from 8 to 9. The only things that are there are: -> buildbot 8 worker will not run under Python 3, and never will -> I performed some minor cleanups in the buildbot-worker code regarding bytes vs. unicode issues. Not a major deal with a setup where mostly ASCII is involved, but becomes an issue when dealing with non-ASCII output from commands, and when the locale is set to something which is not UTF-8. For example, I'm working with someone in China who is trying to set up buildbot 9 + Python 3 + cp936 locale. I haven't fully fixed those issues in the buildbot-worker because I don't have access to Windows set up with that locale. :) But I'm slowly working through the issues and fixing things. For python.org, I don't think these issues are really a concern right now, because the python.org jobs and builders are doing mostly ASCII and UTF-8 compatible stuff, and not using any "exotic" locales on the worker. buildbot was written quite a while ago, and the assumptions about bytes/unicode were done in the Python 2 days. Obviously this has to be fixed properly for Python 3 :) > Oh, and I can offer any of my Windows workers for interoperability > testing if you'd like me to temporarily configure them to an > additional master. At least one of them (XP) is probably not worth > the effort of getting to (a) vs. (b). > Thank you for the offer. I will contact you off the list to coordinate setting up some test environment. I'm a bit busy in the next few days, but maybe next week. -- Craig From zachary.ware+pydev at gmail.com Sat Oct 7 22:55:59 2017 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Sat, 7 Oct 2017 21:55:59 -0500 Subject: [Python-buildbots] Upgrading buildbot master Message-ID: Just a heads-up, I'm going to try to get the build master upgraded tonight. If things aren't running at something approximating normal by 00:00 CDT, I'll be reverting back to the current setup. -- Zach From rosuav at gmail.com Mon Oct 9 03:18:06 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 18:18:06 +1100 Subject: [Python-buildbots] Looking up status of build bots? Message-ID: The direct links to partial waterfalls (eg http://buildbot.python.org/stable/ for the "Stable" set) are no longer active. Is there a way to look up a specific subset of builders to check status? Specific use-case: I just reset my buildbot host, and after doing so, I always like to check that it's come online and reconnected with the master. For that, I have a bookmark that would show me just the four builders on my machine (for four different branches), and it would show either a grey "Offline" or a green "Idle" depending on whether it's all come good. What I have now, I'm not sure about. There's this: http://buildbot.python.org/all/#/builders/27 but it doesn't tell me whether the builder is connected or not. What would be the best way to check on the node's connection? ChrisA From victor.stinner at gmail.com Mon Oct 9 04:14:54 2017 From: victor.stinner at gmail.com (Victor Stinner) Date: Mon, 9 Oct 2017 10:14:54 +0200 Subject: [Python-buildbots] Looking up status of build bots? In-Reply-To: References: Message-ID: Hi, Zachary Ware migrated the buildbot server to 0.9 last weekend. It changes a *lot* of things. It will need a few days (or weeks?) to adapt our workflow and habits to the new version. Our buildbot configuratioin can maybe be also enhanced to better use buildbot features. -- The generic link now redirects to https://www.python.org/dev/buildbot/ https://www.python.org/dev/buildbot/ contains links which are now broken: * http://buildbot.python.org/stable/ : HTTP Error 404 But http://buildbot.python.org/all/#/ works Victor 2017-10-09 9:18 GMT+02:00 Chris Angelico : > The direct links to partial waterfalls (eg > http://buildbot.python.org/stable/ for the "Stable" set) are no longer > active. Is there a way to look up a specific subset of builders to > check status? > > Specific use-case: I just reset my buildbot host, and after doing so, > I always like to check that it's come online and reconnected with the > master. For that, I have a bookmark that would show me just the four > builders on my machine (for four different branches), and it would > show either a grey "Offline" or a green "Idle" depending on whether > it's all come good. What I have now, I'm not sure about. There's this: > > http://buildbot.python.org/all/#/builders/27 > > but it doesn't tell me whether the builder is connected or not. What > would be the best way to check on the node's connection? > > ChrisA > _______________________________________________ > Python-Buildbots mailing list > Python-Buildbots at python.org > https://mail.python.org/mailman/listinfo/python-buildbots From rosuav at gmail.com Mon Oct 9 05:21:08 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 20:21:08 +1100 Subject: [Python-buildbots] Looking up status of build bots? In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 7:14 PM, Victor Stinner wrote: > Hi, > > Zachary Ware migrated the buildbot server to 0.9 last weekend. It > changes a *lot* of things. It will need a few days (or weeks?) to > adapt our workflow and habits to the new version. Our buildbot > configuratioin can maybe be also enhanced to better use buildbot > features. Yeah, I'm aware that there are changes, and I'm fine with that (the expected benefits are definitely worth a bit of change-over hassle). I'm wondering if anyone knows of the new way to look up the info I'm after. If not, that's fine, but I know there are people here who've already used the new buildbot software and might just know. ChrisA From zachary.ware+pydev at gmail.com Mon Oct 9 15:29:38 2017 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Mon, 9 Oct 2017 14:29:38 -0500 Subject: [Python-buildbots] Looking up status of build bots? In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 4:21 AM, Chris Angelico wrote: > On Mon, Oct 9, 2017 at 7:14 PM, Victor Stinner wrote: >> Hi, >> >> Zachary Ware migrated the buildbot server to 0.9 last weekend. It >> changes a *lot* of things. It will need a few days (or weeks?) to >> adapt our workflow and habits to the new version. Our buildbot >> configuratioin can maybe be also enhanced to better use buildbot >> features. > > Yeah, I'm aware that there are changes, and I'm fine with that (the > expected benefits are definitely worth a bit of change-over hassle). > I'm wondering if anyone knows of the new way to look up the info I'm > after. If not, that's fine, but I know there are people here who've > already used the new buildbot software and might just know. The closest you can get that I've found so far is http://buildbot.python.org/all/#/workers/16. If the "Masters" column shows a green 1, you're connected; if it's a red X (see http://buildbot.python.org/all/#/workers/12 at the moment), you're not connected. That page will also show the history of builds that have been done on your worker. Craig, is there anything more comparable to /waterfall?show=Builder1&show=Builder2 in Buildbot 0.9.x? -- Zach From rosuav at gmail.com Mon Oct 9 15:32:31 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Oct 2017 06:32:31 +1100 Subject: [Python-buildbots] Looking up status of build bots? In-Reply-To: References: Message-ID: On Tue, Oct 10, 2017 at 6:29 AM, Zachary Ware wrote: > On Mon, Oct 9, 2017 at 4:21 AM, Chris Angelico wrote: >> On Mon, Oct 9, 2017 at 7:14 PM, Victor Stinner wrote: >>> Hi, >>> >>> Zachary Ware migrated the buildbot server to 0.9 last weekend. It >>> changes a *lot* of things. It will need a few days (or weeks?) to >>> adapt our workflow and habits to the new version. Our buildbot >>> configuratioin can maybe be also enhanced to better use buildbot >>> features. >> >> Yeah, I'm aware that there are changes, and I'm fine with that (the >> expected benefits are definitely worth a bit of change-over hassle). >> I'm wondering if anyone knows of the new way to look up the info I'm >> after. If not, that's fine, but I know there are people here who've >> already used the new buildbot software and might just know. > > The closest you can get that I've found so far is > http://buildbot.python.org/all/#/workers/16. If the "Masters" column > shows a green 1, you're connected; if it's a red X (see > http://buildbot.python.org/all/#/workers/12 at the moment), you're not > connected. That page will also show the history of builds that have > been done on your worker. Thanks! That's about what I want, I think! ChrisA From rodrigc at crodrigues.org Wed Oct 11 00:01:02 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Tue, 10 Oct 2017 21:01:02 -0700 Subject: [Python-buildbots] Looking up status of build bots? In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 12:29 PM, Zachary Ware wrote: > > Craig, is there anything more comparable to > /waterfall?show=Builder1&show=Builder2 in Buildbot 0.9.x? > There is not a 1:1 mapping of that buildbot 0.8 feature to buildbot 0.9, unfortunately. It's not the same, but one thing you can do is go to: http://buildbot.python.org/all/#/grid and in the tags column, click on one of the tags that are of interest. That will then cause the display to only display that selected tag. -- Craig From rodrigc at crodrigues.org Fri Oct 13 15:55:16 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 13 Oct 2017 12:55:16 -0700 Subject: [Python-buildbots] Outdated wiki information Message-ID: This page: https://www.python.org/dev/buildbot/ should probably be changed to from "buildbot slave" to "buildbot worker" These two wiki pages related to buildbots have really oudated information: https://wiki.python.org/moin/BuildBot * references to "buildbot slave" should be changed to "buildbot worker" * references to "python-dev at python.org" should be changed to " python-buildbots at python.org * reference to bbreport should be removed https://wiki.python.org/moin/BuildbotOnWindows * this page is so outdated to be almost useless, and should just be deleted -- Craig From rodrigc at crodrigues.org Fri Oct 13 15:59:16 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 13 Oct 2017 12:59:16 -0700 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: On Fri, Oct 6, 2017 at 1:46 PM, David Bolen wrote: > > Oh, and I can offer any of my Windows workers for interoperability > testing if you'd like me to temporarily configure them to an > additional master. At least one of them (XP) is probably not worth > the effort of getting to (a) vs. (b). > I don't think I need this right now. If I look at: http://buildbot.python.org/all/#/workers Your windows buildbot workers seem to be working fine with the new buildbot master. This is one buildbot worker kloth-win64 which is not working. I don't know what the problem with that one is. -- Craig From zachary.ware+pydev at gmail.com Fri Oct 13 16:00:28 2017 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Fri, 13 Oct 2017 15:00:28 -0500 Subject: [Python-buildbots] Outdated wiki information In-Reply-To: References: Message-ID: On Fri, Oct 13, 2017 at 2:55 PM, Craig Rodrigues wrote: > This page: > https://www.python.org/dev/buildbot/ > > should probably be changed to from "buildbot slave" to "buildbot worker" > > These two wiki pages related to buildbots have really > oudated information: > > https://wiki.python.org/moin/BuildBot > > * references to "buildbot slave" should be changed to "buildbot worker" > * references to "python-dev at python.org" should be changed to " > python-buildbots at python.org > * reference to bbreport should be removed > > > https://wiki.python.org/moin/BuildbotOnWindows > > * this page is so outdated to be almost useless, and should just be > deleted Agreed with all of the above. Additionally, the `buildslave` devguide page needs an update, and w.p.o/dev/buildbot could point to it instead of the wiki. -- Zach From db3l.net at gmail.com Fri Oct 13 16:07:41 2017 From: db3l.net at gmail.com (David Bolen) Date: Fri, 13 Oct 2017 16:07:41 -0400 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: Yes they look to be ok now. Although for future reference, both Windows XP and 7 workers did initially break - apparently due to an incompatibility when using Python < 2.7 on the worker (I have 2.6 on those machines). The master is sending remote methods down using unicode for method names. So 2.7+ appears to be a worker requirement in this mixed-mode setup (though I doubt any new worker would try to use 2.6). For now, I've locally modified my version of twisted on each buildbot to deal with that, pending time to have them upgraded to 2.7 (or possibly in the Win7 case, just go to 3.x). I'm not sure if that might be the same issue with kloth-win64. -- David On Fri, Oct 13, 2017 at 3:59 PM, Craig Rodrigues wrote: > On Fri, Oct 6, 2017 at 1:46 PM, David Bolen wrote: > >> >> Oh, and I can offer any of my Windows workers for interoperability >> testing if you'd like me to temporarily configure them to an >> additional master. At least one of them (XP) is probably not worth >> the effort of getting to (a) vs. (b). >> > > > I don't think I need this right now. > If I look at: > http://buildbot.python.org/all/#/workers > > Your windows buildbot workers seem to be working fine with the new > buildbot master. > > This is one buildbot worker kloth-win64 which is not working. > I don't know what the problem with that one is. > > -- > Craig > From rodrigc at crodrigues.org Fri Oct 13 16:17:08 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 13 Oct 2017 13:17:08 -0700 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: Did you try leaving things on Python 2.6, but upgrading the worker to buidlbot-worker 0.9.12 ? buildbot-worker is still tested on Python 2.6. -- Craig On Fri, Oct 13, 2017 at 1:07 PM, David Bolen wrote: > Yes they look to be ok now. Although for future reference, both Windows > XP and 7 workers did initially break - apparently due to an incompatibility > when using Python < 2.7 on the worker (I have 2.6 on those machines). The > master is sending remote methods down using unicode for method names. So > 2.7+ appears to be a worker requirement in this mixed-mode setup (though I > doubt any new worker would try to use 2.6). > > For now, I've locally modified my version of twisted on each buildbot to > deal with that, pending time to have them upgraded to 2.7 (or possibly in > the Win7 case, just go to 3.x). > > I'm not sure if that might be the same issue with kloth-win64. > > -- David > > > On Fri, Oct 13, 2017 at 3:59 PM, Craig Rodrigues > wrote: > >> On Fri, Oct 6, 2017 at 1:46 PM, David Bolen wrote: >> >>> >>> Oh, and I can offer any of my Windows workers for interoperability >>> testing if you'd like me to temporarily configure them to an >>> additional master. At least one of them (XP) is probably not worth >>> the effort of getting to (a) vs. (b). >>> >> >> >> I don't think I need this right now. >> If I look at: >> http://buildbot.python.org/all/#/workers >> >> Your windows buildbot workers seem to be working fine with the new >> buildbot master. >> >> This is one buildbot worker kloth-win64 which is not working. >> I don't know what the problem with that one is. >> >> -- >> Craig >> > > From db3l.net at gmail.com Fri Oct 13 16:36:13 2017 From: db3l.net at gmail.com (David Bolen) Date: Fri, 13 Oct 2017 16:36:13 -0400 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: Does testing include a mixture with a master under 3.x? I only ask since I'd guess that a master under 2.x might be sending parameters down as strings rather than unicode. Anyway, no, I hadn't tried a new worker, because both the same and an earlier buildbot version was working fine on my Win8/10 workers, so I focused more on other environmental differences, and since time was tight, once I identified a workaround, I stopped for the time being. I did misspeak in the last note - the issue was a unicode name for a keyword parameter remote function call, not the function name itself. The actual failure occurs inside twisted while dispatching the remote call from the master. It takes a PB-transmitted dictionary for the arguments, using either apply() or **kwargs against them directly. That fails with a "TypeError: keywords must be strings" under 2.6 but 2.7 is fine. Since the dispatch failure occurs during the dispatch to buildbot-worker, but before it gets any chance to run, I'm not sure that it could change things. Unless the master changes something in how it transmits request arguments that needed a newer worker on the client to identify. Oh, I did also peek at the latest twisted source but didn't see any differences in that path at a quick glance (aside from switching from apply() to **kw) that would seem to be able to resolve this, but that's something else I could test. Or I could just move off of 2.6 which I last updated to in 2009 :-) -- David On Fri, Oct 13, 2017 at 4:17 PM, Craig Rodrigues wrote: > Did you try leaving things on Python 2.6, but upgrading the worker to > buidlbot-worker 0.9.12 ? > buildbot-worker is still tested on Python 2.6. > > -- > Craig > > On Fri, Oct 13, 2017 at 1:07 PM, David Bolen wrote: > >> Yes they look to be ok now. Although for future reference, both Windows >> XP and 7 workers did initially break - apparently due to an incompatibility >> when using Python < 2.7 on the worker (I have 2.6 on those machines). The >> master is sending remote methods down using unicode for method names. So >> 2.7+ appears to be a worker requirement in this mixed-mode setup (though I >> doubt any new worker would try to use 2.6). >> >> For now, I've locally modified my version of twisted on each buildbot to >> deal with that, pending time to have them upgraded to 2.7 (or possibly in >> the Win7 case, just go to 3.x). >> >> I'm not sure if that might be the same issue with kloth-win64. >> >> -- David >> >> >> On Fri, Oct 13, 2017 at 3:59 PM, Craig Rodrigues >> wrote: >> >>> On Fri, Oct 6, 2017 at 1:46 PM, David Bolen wrote: >>> >>>> >>>> Oh, and I can offer any of my Windows workers for interoperability >>>> testing if you'd like me to temporarily configure them to an >>>> additional master. At least one of them (XP) is probably not worth >>>> the effort of getting to (a) vs. (b). >>>> >>> >>> >>> I don't think I need this right now. >>> If I look at: >>> http://buildbot.python.org/all/#/workers >>> >>> Your windows buildbot workers seem to be working fine with the new >>> buildbot master. >>> >>> This is one buildbot worker kloth-win64 which is not working. >>> I don't know what the problem with that one is. >>> >>> -- >>> Craig >>> >> >> > From rodrigc at crodrigues.org Fri Oct 13 16:44:21 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 13 Oct 2017 13:44:21 -0700 Subject: [Python-buildbots] Moving Python.org to Buildbot 9 + Python 3 In-Reply-To: References: <20170920150252.CB3E91B10028@webabinitio.net> Message-ID: On Fri, Oct 13, 2017 at 1:36 PM, David Bolen wrote: > Does testing include a mixture with a master under 3.x? I only ask since > I'd guess that a master under 2.x might be sending parameters down as > strings rather than unicode. > I've tested the following combinations of master 3.x <==> worker 2.x, master 2.x <==> worker 3.x: https://lists.buildbot.net/pipermail/devel/2017-October/012432.html > > Oh, I did also peek at the latest twisted source but didn't see any > differences in that path at a quick glance (aside from switching from > apply() to **kw) that would seem to be able to resolve this, but that's > something else I could test. > Yes, that was something in Twisted that was fixed in the past year. > > Or I could just move off of 2.6 which I last updated to in 2009 :-) > It looks like you have done more than your fair share of investigation to get things working. Believe me, testing all the permutations and combinations of things is a major pain in the neck. You have identified the problem, applied a workaround to get things working now, and have identified a path to success moving forward. If you are OK with that, then that is probably good enough. -- Craig From victor.stinner at gmail.com Fri Oct 13 17:18:07 2017 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 13 Oct 2017 23:18:07 +0200 Subject: [Python-buildbots] Outdated wiki information In-Reply-To: References: Message-ID: FYI python.org is now managed on http://github.com/python/pythondotorg/ Victor 2017-10-13 21:55 GMT+02:00 Craig Rodrigues : > This page: > https://www.python.org/dev/buildbot/ > > should probably be changed to from "buildbot slave" to "buildbot worker" > > These two wiki pages related to buildbots have really > oudated information: > > https://wiki.python.org/moin/BuildBot > > * references to "buildbot slave" should be changed to "buildbot worker" > * references to "python-dev at python.org" should be changed to " > python-buildbots at python.org > * reference to bbreport should be removed > > > https://wiki.python.org/moin/BuildbotOnWindows > > * this page is so outdated to be almost useless, and should just be > deleted > > > -- > Craig > _______________________________________________ > Python-Buildbots mailing list > Python-Buildbots at python.org > https://mail.python.org/mailman/listinfo/python-buildbots From berker.peksag at gmail.com Fri Oct 13 22:57:13 2017 From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=) Date: Sat, 14 Oct 2017 05:57:13 +0300 Subject: [Python-buildbots] Outdated wiki information In-Reply-To: References: Message-ID: On Fri, Oct 13, 2017 at 10:55 PM, Craig Rodrigues wrote: > This page: > https://www.python.org/dev/buildbot/ > > should probably be changed to from "buildbot slave" to "buildbot worker" I've now changed "buildbot slave" to "buildbot worker" at https://www.python.org/dev/buildbot/. > These two wiki pages related to buildbots have really > oudated information: > > https://wiki.python.org/moin/BuildBot > > * references to "buildbot slave" should be changed to "buildbot worker" > * references to "python-dev at python.org" should be changed to " > python-buildbots at python.org > * reference to bbreport should be removed Can't we refer people to https://devguide.python.org/buildslave/ and update all outdated information there? It looks like the buildbot pages on the wiki mostly contain outdated and duplicate information. --Berker From rodrigc at crodrigues.org Sat Oct 14 02:20:41 2017 From: rodrigc at crodrigues.org (Craig Rodrigues) Date: Fri, 13 Oct 2017 23:20:41 -0700 Subject: [Python-buildbots] Outdated wiki information In-Reply-To: References: Message-ID: On Fri, Oct 13, 2017 at 7:57 PM, Berker Peksa? wrote: > > > Can't we refer people to https://devguide.python.org/buildslave/ and > update all outdated information there? It looks like the buildbot > pages on the wiki mostly contain outdated and duplicate information. > That sounds like a perfectly reasonable thing to do. It would be good to rename that link, probably to https://devguide.python.org/buildbot . The devguide is at: https://github.com/python/devguide/ if you want to submit pull requests, I can review if there is no available reviewer. -- Craig From rdmurray at bitdance.com Tue Oct 17 21:54:35 2017 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 17 Oct 2017 21:54:35 -0400 Subject: [Python-buildbots] Outdated wiki information In-Reply-To: References: Message-ID: <20171018015436.E836F1B10002@webabinitio.net> On Fri, 13 Oct 2017 23:20:41 -0700, Craig Rodrigues wrote: > On Fri, Oct 13, 2017 at 7:57 PM, Berker Peksa?? > wrote: > > Can't we refer people to https://devguide.python.org/buildslave/ and > > update all outdated information there? It looks like the buildbot > > pages on the wiki mostly contain outdated and duplicate information. The devguide pages started with a copy of what was on the wiki, and then I updated it based on actually doing installs and some additional research. I was planning to delete the wiki pages (well, replace them with a link to the devguide), but I forgot to do it. --David