[Python-ideas] A tuple of various Python suggestions

[Nick Coghlan]

On 10 April 2016 at 10:28, Keith Curtis <keithcu at gmail.com> wrote:
> I agree that more companies paying for Python support would be a great
> thing, but most often companies buy support at a different level of
> granularity: for example, RHEL where they get it for an entire OS, or
> hiring Django people to build and maintain a website.

For context, I work for Red Hat's developer experience team and
occasionally put the graph at
http://bugs.python.org/issue?@template=stats in front of our Python
maintenance team and ask "This is a trend that worries me, what are we
currently doing about it?".

I'd encourage anyone using Python in a commercial or large
organisational context that are getting their Python runtimes via a
commercially supported Linux distro, Platform-as-a-Service provider,
or one of the cross-platform CPython redistributors to take that
graph, show it to their vendor and say: "These upstream CPython
metrics worry us, what are you doing about them on our behalf?". (If
you don't know how to ask your vendor that kind of question since
another team handles the supplier relationship, then find that team
and ask *them* what they're doing to address your concerns)

For folks in those kinds of contexts running directly off the binaries
or source code published by the Python Software Foundation, then my
recommendation is a bit different:

* option 1 is to advocate for switching to a commercial redistributor,
and asking prospective vendors how they're supporting ongoing CPython
maintenance as part of the vendor evaluation
* option 2 is to advocate for becoming a PSF Sponsor Member and then
advocate for a technical fellowship program aimed at general
maintenance and project health monitoring for CPython (keeping in mind
that "How do we manage it fairly and sustainably?" would actually be a
bigger challenge for such a fellowship than funding it)

Whether advocating for option 1 or option 2 makes more sense will vary
by organisation, as it depends greatly on whether or not the
organisation would prefer to work with a traditional commercial
supplier, or engage directly with a charitable foundation that
operates in the wider public interest.

There's also option 3, which is to hire existing core developers and
give them time to work on general upstream CPython maintenance, but
there are actually relatively few organisations for which that
strategy makes commercial sense (and they're largely going to be
operating system companies, public cloud infrastructure companies, or
commercial Python redistributors).

> I wouldn't count
> on commercial redistributors of Python as ever being a big source of
> resources to fix the thousands of random bugs. The vast majority of
> people use the standard runtime.

If that were an accurate assessment, I wouldn't need to write articles
like http://www.curiousefficiency.org/posts/2015/04/stop-supporting-python26.html
and the manylinux cross-distro Linux ABI wouldn't be based on the 9
year old CentOS 5 userspace. (We also wouldn't be needing to pursue
redistributor-friendly proposals like PEP 493 to help propagate the
network security enhancements in the 2.7.x series)

Direct downloads from python.org are certainly a major factor on
Windows (especially in educational contexts), but even there, many of
the largest commercial adopters are using ActiveState's distribution
(see http://www.activestate.com/customers ) or one of the analytical
distributions from Enthought or Continuum Analytics.

> I realize that volunteers work on their own time and choice of tasks,
> but you've got quite a large community, and by periodically reporting
> on the bug count and having a goal to resolve old ones eventually, you
> will even find problem areas no one is talking about. Sometimes the
> issue isn't resources, but grit, focus, pride, etc.

So, you're suggesting we deliberately set out to make our volunteers
feel bad in an effort to guilt them into donating more free work to
corporations and other large organisations? When I phrase it in that
deliberately negative way, I hope it becomes obvious why we *don't* do
this.

We do send out the weekly tracker metrics emails, and offers the
statistics graphs on the tracker itself if people want to use them as
parts of their own business cases for increased investment (whether
direct or indirect) in upstream sustaining engineering for CPython.

> If you never weighed yourself, at looked at yourself in the mirror,
> you'd get heavier than you realized. You are mostly volunteers, but
> you can try to produce software as good as what paid professionals can
> do.

You seem to be operating under some significant misapprehensions here.
As far as I am aware, most of the core developers *are* paid
professionals (even the folks that didn't start out that way) - we're
just typically not paid specifically to work on CPython. Instead, we
work on CPython to make sure it's fit for *our* purposes, or because
we find it to be an enjoyable way to spend our free time.

Engaging effectively with that environment means pursuing a
"co-contributor" experience - we each make CPython better for our own
purposes, and by doing so, end up making it better for everyone.

The experience you seem to be seeking in this thread is that of a pure
software consumer, and that's the role downstream redistributors
fulfil in an open source ecosystem: they provide (usually for a fee) a
traditional customer experience, with the redistributor taking care of
the community engagement side of things.

> Perhaps the bug list should be broken up into two categories: future
> optional or incompatible features, and code that people can easily
> agree are worth doing now. That way important holes don't get ignored
> for years. People should be able to trust the standard runtime.

Important holes don't get ignored for years - they get addressed,
since people are motivated to invest the time to ensure they're fixed.
That said, we do experience the traditional open source "community
gap" where problems that are only important to folks that aren't yet
personally adept at effective community engagement can languish
indefinitely (hence my emphasis above on organisations considering the
idea of outsourcing their community engagement efforts, rather than
assuming that it's easy, or that someone else will take care of it for
them without any particular personal motivation).

> It didn't seem like WebAssembly would enable access to Numpy. It seems
> doubtful it will be much adopted if it can't access the rich Python
> runtime that might be installed.

Project Jupyter already addressed that aspect by separating out the
platform independent UI service for code entry from the language
kernel backend for code evaluation. There's certainly a lot that could
be done in terms of making Jupyter language kernels installable as web
browser add-ons running native code, but that would be a question for
the Project Jupyter folks, rather than here.

> Python is much cleaner than some of the alternatives, so you should
> consider adoption as a humanitarian effort!

As tempting as it is to believe that "things that matter to software
developers" dictate where the world moves, it's important to keep in
mind that it's estimated we make up barely 0.25% of the world's
population [1] - this means that for everyone that would consider
themselves a "software developer", there's around 399 people who
don't.

Accordingly, if your motivation is "How do we help empower people to
control the technology in their lives, rather than having it control
them?" (which is a genuinely humanitarian effort), then I'd recommend
getting involved with education-focused organisations like the
Raspberry Pi Foundation, Software Carpentry and Django Girls, or
otherwise helping to build bridges between the software development
community and professional educators (see [2] for an example of the
latter).

When it comes to technology choices *within* that 0.25%, it's worth
remembering that one of the key design principles of Python is
*trusting developers to make their own decisions* (hence things like
allowing monkeypatching as being incredibly useful in cases like
testing, even while actively advising against it as a regular
programming practice). That trust extends to trusting them to choose
languages that are appropriate for them and their use cases. If that
leads to them choosing Python, cool. If it leads to them choosing
something else, that's cool, too - there are so many decent open
source programming languages and runtimes out there these days that
software developers are truly spoiled for choice, so providing a solid
foundation for exploring those alternatives is at least as important a
goal as reducing the incentives for people that already favour Python
to need to explore them.

Cheers,
Nick.

[1] http://www.infoq.com/news/2014/01/IDC-software-developers
[2] https://2016.pycon-au.org/programme/python_in_education_seminar

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia