[Python-Dev] Python "2migr8"

[Nick Coghlan]

On 14 Apr 2014 17:17, "Donald Stufft" <donald at stufft.io> wrote:
>
> Now I will admit I personally have probably had a harder time than some
others because of the nature of the things I was trying to work on, and
lately it’s gotten better (although I think that’s partially because I’m
more known now, and I think in general the experience of contributing to
CPython changes depending on who you are, so the more integrated into the
culture you are, the less likely you are to see the issues and
unfortunately those people are also the ones with the most power to change
it). I do however think that just in general it might be getting better too?

>From my perspective, the main issue is that new contributors typically
don't acknowledge the large number of competing constraints we're balancing
in core development, and hence grow frustrated when we reject their
"obvious" idea on the basis of a concern that seems completely irrelevant
to them. (Due to my background, I'll occasionally phrase this along the
lines of "Supporting submarines and secure enclaves matters to me, but most
open source devs don't care").

Core development for a programming language is a genuinely hard problem,
and one that often involves thinking on timescales of years and decades
rather than the weeks and months involved in more typical development
environments. There's a huge impedance mismatch in expectations that can
lead to major communication problems if people don't take the time to lurk
for a while and get a feel for the kinds of novel concerns that arise when
sitting at the centre of a massive ecosystem like Python's.

> Specific details are hard because it’s nothing major and obvious like
having Linus go off on rants and tearing things apart, it’s death by a
thousand cuts so it’s hard to point a finger at one behavior (or a few
behaviors) and look at them in isolation and “see” it. That being said I’m
more than happy to *try* and explain it, but right this moment I don’t have
a lot of time as I’m getting ready to step out the door, but I didn’t want
to leave this email hanging without a reply.

>From my perspective, the heart of the issue is personal time management on
the part of the core development team. We're highly cognizant of the limits
of what can be sustained through volunteer development, and one of the big
issues is that loading volunteers up with too many activities that they
feel obliged to do but don't find inherently enjoyable is a recipe for
burnout. So we "default to no" not just because the number of ways we can
make Python worse is unbounded, but also because the time we have available
to do anything at all is incredibly limited.

Now, consider that we're operating in an environment where multi-billion
dollar companies are relying on our software while making only relatively
small contributions to its ongoing support and evolution, and where we have
multiple prominent community members wishing vocally (and encouraging
others to advocate) for the core development team to devote our volunteer
efforts to improving a legacy language rather than the new one we shifted
en masse to working on instead. (Note that the latter actually makes about
as much sense to me as telling the Rust and Go developers they should spend
their free time working on C compilers instead because the latter would be
more immediately useful to commercial users)

On top of this, various outreach efforts to encourage new contributors are
working, and working well enough that they are actually *exceeding* the
existing team's ability to effectively *absorb* those new contributors.

So, tremendously high stress levels for the core development team, on top
of whatever stress we have to deal with in our personal and professional
lives.

This stress then manifests as irritability, impatience and outright anger -
my own stress levels reached sufficiently high levels earlier this year
that I almost decided to walk away from my job. Red Hat management
intervened to keep that from actually happening, but I think it's important
to make that incident more public to help people understand how utterly
unsustainable the status quo is when it comes to CPython - we can't keep
relying on almost entirely volunteer effort to maintain 2.7 LTS, 3.x, all
the python.org infrastructure *and* the PSF without also anticipating
complete and total burnout of some highly invested contributors.

PEP 462 describes some ideas to make more effective use of core developer
time, and to potentially distribute particular tasks to better suited
groups of people (such as the tutorial and HOWTO guides), but in itself
involves a substantial amount of up front work.

That's where Guido's suggestion of corporations offering more "50%" jobs
for core developers comes in. That 50% time wouldn't be about working on
things we would have done anyway - it would be about working on things that
are difficult to get jumpstarted with purely volunteer effort. More work on
2.7 maintenance, more work on CPython infrastructure, more work on
encouraging and supporting new contributors, more work on improving
communications.

As things stand, the centre cannot hold (not indefinitely, anyway).
However, I think Guido's proposed approach to resolving the situation is a
good one, and similar in many respects to the way the Linux and OpenStack
communities already work. I think we also have the necessary contacts at
various organisations to make it a reality if we make it clear not only
that we're open to the idea, but also emphasise the strategic risk
corporate users are currently taking by treating CPython as magic
programming pixie dust spontaneously generated on the internet - a few
relatively flexible roles for core developers at various organisations
would drastically reduce that risk without ceding any one organisation
undue levels of influence.

Regards,
Nick.

>
> [1] See Also
http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140415/0bbf518c/attachment.html>