[Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 12:55, Paul Moore <p.f.moore at gmail.com> wrote:
> On 18 April 2014 16:58, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> As part of thrashing out the respective distribution ecosystem roles
>> of pip and conda (still a work in progress), we're at least converging
>> on the notion that there are actually now *two* main ways of consuming
>> Python: as a "software integrator" (the way most of us have
>> traditionally consumed it, and the way that dominates most project
>> documentation outside the scientific Python community) and as an "end
>> user" (the way Linux system administrators have long consumed it, and
>> the way scientists, financial analysts and folks just learning Python
>> are likely best off consuming it).
>>
>> Making these different personas explicit is a process that has barely
>> begun (this email is mostly based on some conversations I had in
>> person at PyCon and via email during the sprints), but here's the gist
>> (based on listing examples):
>
> Interesting perspective. However, I'm not convinced it's complete.

Personas are never complete - however, they represent sufficiently
useful archetypes that help guide design decisions. Where alarm bells
need to start ringing is when two personas are far enough apart in
interests and skill levels that it is highly unlikely that one tool
will be able to satisfy both groups of users.

I think that's the situation we find ourselves in with respect to
folks that are already professional programmers and folks that just
need to do some programming as part of another activity.

> Specifically, there's one group of people who I encounter relatively
> often, who don't seem to me to fit well into either category you're
> proposing. That is, (Windows in my experience, but maybe Linux as
> well) users who want to write "simple scripts" and for whom batch
> files or similar are not sufficient. Such people typically don't have
> the sort of "single application area" focus that your "end user"
> category seems to imply, but on the other hand they don't really fit
> the "software integrators" role in the sense of necessarily being
> comfortable setting up their own development environment.

The "single application area" in that case is "Windows scripting", and
it *would* make sense to have a sumo distribution that provided
PyWin32 and other components. That distribution may include other
things as well, though.

> I worry that your classification risks ignoring that group (maybe
> because Unix users are well served with other alternatives than Python
> for this type of task, or because on Unix "use the system Python" is
> the right answer).

I think the conda "end user" approach handles that case pretty well
(while it certainly started with the scientific Python stack, conda
certainly isn't limited to that). So the entry can be "system
administrators" in general, rather than being specific to Linux.

The "lack of integration" issues on Linux and Mac OS X show up because
there *is* a system Python for conda to be separate from. We don't
have that problem on Windows (although integrating with the Python
launcher may be an issue).

> Your list of "end user" targeted distributions seem to be limited to:
>
>   - Linux distribution vendors

As noted above, this category is broader than just Linux - it really
covers any system scripting context.

>   - Vendors focused on the essentially scientific community (in the
> broadest sense)

I'd say "analytical", rather than specifically "scientific". From a
user experience perspective though, the important point is that many
of the folks being targeted *aren't* programmers in the traditional
sense - they just need to do some programming as part of their actual
job. It turns out that's also a useful usability proxy for people just
starting out with programming :)

>   - Embedded Python
>
> That's very far from being complete coverage of all the people *I'd*
> like to be able to recommend Python to. Specifically, unless we're not
> interested in "generic" Windows users,

I think conda covers that "generic Windows user" case as well - that's
why I emphasised the fully open source nature of it. We've also been
having some interesting conversations with Travis about potentially
improving the level of collaboration between the PyPA and conda devs.

> I think we need to offer *some*
> form of equivalent of the OS-packaged Python on Linux for Windows
> users. That's what the python.org builds, plus pip, wheels and PyPI,
> give for Windows users now. Hmm, if we assume that supporting that
> remains a priority, is what you're really saying that we *don't* try
> to extend that to work for Linux/OSX, as doing so competes with the OS
> vendors - but rather we see python.org binaries and binary
> infrastructure like wheels as being focused on the Windows user
> experience?

I'd still like to improve the wheel story for Linux (there are various
benefits of that which accrue to system integrators as well), but I
think the key benefit of promoting both pip *and* conda is that it
lets us hit *two* different points on the flexibility vs usability
curve, rather than being limited to one.

pip needs to preserve a lot of flexibility, because it's already the
upstream for a *lot* of different use cases. However, that flexibility
often comes at a cost in usability, especially for new users.

Since conda *doesn't* play that upstream role the way pip does, the
flexibility can be dialled down a bit in favour of increased "out of
the box" usability, making it easier to offer a compelling new user
experience.

Ideally we'd like to make the two tools share a common back end to
reduce duplicated effort (and distlib may end up filling that role),
but I think such a split makes a lot of sense from a front end
perspective.

> (I wish I'd been at PyCon, this would have been a very interesting
> discussion to have face to face. Email isn't ideal for this...)

We didn't even get far with it at PyCon - there's certainly no
consensus around these characterisations at this point. The discussion
was mostly focused on pip vs conda, since the requirements for the two
tools are very, very similar. My contention is that it *doesn't* make
sense to try to make conda a replacement for pip (or vice-versa)
because the two tools are focused on different *audiences*, even
though an awful lot of the backend engineering requirements are the
same.

At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)

Regards,
Nick.

>
> Paul.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia