[Distutils] command line versus python API for build system abstraction (was Re: build system abstraction PEP)

[Nathaniel Smith]

On Wed, Nov 11, 2015 at 11:34 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 11 November 2015 at 18:38, Nathaniel Smith <njs at pobox.com> wrote:
>> Have you tried current dev versions of pip recently?
>
> No, but I did see your work on this, and I appreciate and approve of it.
>
>> The default now is to
>> suppress the actual output but for progress reporting to show a spinner that
>> rotates each time a line of text would have been printed. It's low tech but
>> IMHO very effective. (And obviously you can also flip a switch to either see
>> all or nothing of the output as well, or if that isn't there now if books
>> really be added.) So I kinda feel like these are solved problems.
>
> And this relies on build tools outputting to stdout, not stderr, and
> not buffering their output.

FWIW the spinner patch actually looks at both stdout and stderr, and
it also takes care to force the child process's sys.stdout/sys.stderr
into line-buffered mode, but of course this buffering tweak only helps
for output printed by python code running in the immediate child. So
yeah, it wouldn't hurt to add a few non-normative words about
buffering to my original one-sentence specification :-).

> That's an interface spec. Not everything has to be massively
> complicated, and I wasn't implying it needed to be. Just that we need
> conventions. One constant annoyance for pip is that distutils doesn't
> properly separate stdout and stderr, so we can't suppress unnecessary
> status reports without losing important error messages. Users report
> this as a bug in pip, not in distutils, and I don't imagine that would
> change if a project was using <name your build tool here>.

Sorry for misunderstanding!

I guess the other thing we could do is to try to convince build
systems to do a better job of separating stdout and stderr, but I'm
dubious about how much this would help, because I think the problem is
more fundamental than that. For outright errors, there isn't really a
problem IMO, because when the build fails that gives you a clear
signal that you should probably show the user the output :-). The case
that's trickier, and could potentially benefit, is warnings that don't
cause the build to fail. If gcc outputs a warning, should we show that
to the user? Yes if this is the developer building their own code...
but probably not if this is pip building from an automatically
downloaded sdist for an end-user -- there are lots and lots of
harmless warnings in the output of popular packages, and dumping those
scary and inscrutable messages on end-users is going to create all the
problems we were trying to avoid by hiding the output in the first
place.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org