[Numpy-discussion] Remove bento from numpy

[Matthew Brett]

On Sat, Jul 5, 2014 at 3:21 PM, David Cournapeau <cournape at gmail.com> wrote:
>
>
>
> On Sat, Jul 5, 2014 at 11:17 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Sat, Jul 5, 2014 at 2:32 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> >
>> > On Sat, Jul 5, 2014 at 1:54 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>
>> >> On 5 Jul 2014 09:23, "Ralf Gommers" <ralf.gommers at gmail.com> wrote:
>> >> >
>> >> > On Sat, Jul 5, 2014 at 10:13 AM, David Cournapeau
>> >> > <cournape at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On Sat, Jul 5, 2014 at 11:25 AM, Charles R Harris
>> >> >> <charlesr.harris at gmail.com> wrote:
>> >> >>>
>> >> >>> Ralf likes the speed of bento, but it is not currently maintained
>> >> >>
>> >> >>
>> >> >> What exactly is not maintained ?
>> >> >
>> >> >
>> >> > The issue is that Julian made some slightly nontrivial changes to
>> >> > core/setup.py and didn't want to update core/bscript. No one else has
>> >> > taken
>> >> > the time either to make those changes. That didn't bother me enough
>> >> > yet to
>> >> > go fix it, because they're all optional features and using Bento
>> >> > builds
>> >> > works just fine at the moment (and is part of the Travis CI test
>> >> > runs, so
>> >> > it'll keep working).
>> >>
>> >> Perhaps a compromise would be to declare it officially unsupported and
>> >> remove it from Travis CI, while leaving the files in place to be used
>> >> on an
>> >> at-your-own-risk basis? As long as it's in Travis, the default is that
>> >> anyone who breaks it has to fix it. If it's not in Travis, then the
>> >> default
>> >> is that the people (person?) who use bento are responsible for keeping
>> >> it
>> >> working for their needs.
>> >
>> > -1 that just means that simple changes like adding a new extension will
>> > not
>> > get made before PRs get merged, and bento support will be in a broken
>> > state
>> > much more often.
>>
>> Yes, and then the handful of people who care about this would fix it
>> or not. Your -1 is attempting to veto other people's *not* paying
>> attention to this build system. I... don't think -1's work that way
>> :-(
>>
>> >> > I don't think the above is a good reason to remove Bento support. The
>> >> > much faster builds alone are a good reason to keep it. And the
>> >> > assertion
>> >> > that all numpy devs understand numpy.distutils is more than a little
>> >> > questionable:)
>> >>
>> >> They surely don't. But thousands of people use setup.py, and one or two
>> >> use bento.
>> >
>> > I'm getting a little tired of these assertions. It's clear that David
>> > and I
>> > use it. A cursory search on Github reveals that Stefan, Fabian, Jonas
>> > and
>> > @aksarkar do (or did) as well:
>> >    https://github.com/scipy/scipy/commit/74d823b3
>> >    https://github.com/numpy/numpy/issues/2993
>> >    https://github.com/numpy/numpy/pull/3606
>> >    https://github.com/numpy/numpy/issues/3889
>> > For every user you can measure there's usually a number of users that
>> > you
>> > don't hear about.
>>
>> I apologize for forgetting before that you do use Bento, but these
>> patches you're finding don't really change the overall picture. Let's
>> assume that there are 100 people using Bento, who would be slightly
>> inconvenienced if they had to use setup.py instead, or got stuck
>> patching the bento build themselves to keep it working. 100 is
>> probably an order of magnitude too high, but whatever. OTOH numpy has
>> almost 7 million downloads on PyPI+sf.net, of which approximately
>> every one used setup.py one way or another, plus all the people get it
>> from alternative channels like distros, which also AFAIK universally
>> use setup.py. Software development is all about trade-offs. Time that
>> numpy developers spend messing about with bento to benefit those
>> hundred users is time that could instead be spent on improvements that
>> benefit many orders of magnitudes more users. Why do you want us to
>> spend our time producing x units of value when we could instead be
>> producing 100*x units of value for the same effort?
>>
>> >> Yet supporting both requires twice as much energy and attention as
>> >> supporting just one.
>> >
>> > That's of course not true. For most changes the differences in where and
>> > how
>> > to update the build systems are small. Only for unusual changes like
>> > Julian
>> > patches to make use of optional GCC features, Bento and distutils may
>> > require very different changes.
>> >>
>> >> We've probably spent more person-hours talking about this, documenting
>> >> the
>> >> missing bscript bits, etc. than you've saved on those fast builds.
>> >
>> > Then maybe stop talking about it:)
>> >
>> > Besides the fast builds, which is only one example of why I like Bento
>> > better, there's also the fundamental question of what we do with build
>> > tools
>> > in the long term. It's clear that distutils is a dead end. All the PEPs
>> > related to packaging move in the direction of supporting tools like
>> > Bento
>> > better. If in the future we need significant new features in our build
>> > tool,
>> > Bento is a much better base to build on than numpy.distutils. It's
>> > unfortunate that at the moment there's no one that works on improving
>> > our
>> > build situation, but that is what it is. Removing Bento support is a
>> > step in
>> > the wrong direction imho.
>>
>> "We must do something! This is something!"
>>
>> Bento is pre-alpha software whose last upstream commit was in July
>> 2013. It's own CI tests have been failing since Feb. 2013, almost a
>> year and a half ago. Bento build support was added to numpy in early
>> 2011, and 3.5 years later it still hasn't convinced most of the core
>> team that it provides any value at all, yet it continues to take up
>> time and attention.
>>
>> Maybe bento will revive and take over the new python packaging world!
>> Maybe not. Maybe something else will. I don't see how our support for
>> it will really affect these outcomes in any way. And I especially
>> don't see why it's important to spend time *now* on keeping bento
>> working, just in case it becomes useful *later*.
>
>
> But it is working right now, so that argument is moot.

Why don't we wait until there is a significant problem with getting
the Bento builds to work, and revisit then.

Cheers,

Matthew