[Distutils] Current Python packaging status (from my point of view)

[Chris Barker]

On Wed, Nov 2, 2016 at 9:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> No, as the post was about the fundamental and irreconcilable
> differences in capabilities, not the incidental ones that can be
> solved if folks choose (or are paid) to put in the necessary design
> and development time.


It's your post, but the external dependency is hardly an incidental issue.


> The post isn't written for beginners deciding which tool to use, it's
> written for intermediate folks that have already chosen one or the
> other for their own needs, and are wondering why the other still
> exists (or, worse, are telling people that have chosen the other tool
> for good reasons that they're wrong, and should switch).


again -- those reasons are very much about external dependency management!

- you need a system for specifying environmental *constraints* (like
> dynamically linked C libraries and command line applications you
> invoke)
> - you need a system for asking the host environment if it can satisfy
> those constraints
>

and it it can't -- you're then done -- that's actually the easy part (and
happens already and build or run time, yes?):

I try to build libgdal, it'll fail if I don't have that huge pile of
dependencies installed.

I try to run a wheel someone else built -- it'll also fail.

It'd be better if this could be hashed out a compilation or linking error,
sure, but that's not goign to do a whole lot except make the error messages
nicer :-)


> dnf/yum, apt, brew, conda, et al all *work around* the current lack of
> such a system by asking humans (aka "downstream package maintainers")
> to supply the additional information by hand in a platform specific
> format.


if conda is a "platform", then yes. but in that sense pip is a platform,
too.

I'll beat this drum again -- if you want to extend pip to solve all (most)
of the problems conda solves, then you are re-inventing conda. If someone
doesn't like the conda design, and has better ideas, great, but only
re-invent the wheel if you really are going to make a better wheel.

However, I haven't thought about it carefully -- maybe it would be possible
to have a system than managed everything except python itself. But that
would be difficult, and I don't see the point, except to satisfy brain dead
IT security folks :-)


> > But the different audiences aren't "data science folks" vs "web
> developers"
> > -- the different audiences are determined by deployment needs, not
> domain.
>
> Deployment needs are strongly correlated with domain though, and
> there's a world of difference between the way folks do exploratory
> data analysis and the way production apps are managed in
> Heroku/OpenShift/Cloud Foundry/Lambda/etc.
>

sigh. not everyone that uses the complex scipy stack is doing "exploratory
data analysis" -- a lot of us are building production software, much of it
behind web services...

and that's what I mean by deployment needs.

 However, if you're specifically interested in web service
> development, then swapping in your own Python runtime rather than just
> using a PaaS provided one is really much lower level than most
> beginners are going to want to be worrying about these days - getting
> opinionated about that kind of thing comes later (if it happens at
> all).
>

it's not a matter of opinion, but needs -- if this "beginner" is doing
stuff only with pure-python packages, then great -- there are many easy
options. But if they need some other stuff - maybe this beginner needs to
work with scientific data sets.. then they're dead in the water with a
Platform that doesn't support what they need.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20161102/5e484386/attachment-0001.html>