[SciPy-Dev] Bundling Boost?

[David Warde-Farley]

It looks like Andreas' message didn't make it to the list, so here it
is forwarded.

Looks like there are a few things to learn about bundling boost, but
nothing show-stopping. Of course, some of it (compiler hell) is out of
our control, but "get a better compiler" is a reasonable response on
most platforms (... maybe not Windows).

---------- Forwarded message ----------
From: Andreas Kloeckner <lists at informa.tiker.net>
Date: Wed, Oct 10, 2012 at 7:43 PM
Subject: Re: [SciPy-Dev] Bundling Boost?
To: David Warde-Farley <wardefar at iro.umontreal.ca>, SciPy Developers
List <scipy-dev at scipy.org>


David Warde-Farley <wardefar at iro.umontreal.ca> writes:

> On Sun, Oct 7, 2012 at 11:40 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> Hi,
>>
>> I'd like to consider replacing some of the function implementations in
>> scipy.special with versions from the C++ Boost library, cf.
>> http://www.boost.org/doc/libs/1_51_0/libs/math/doc/sf_and_dist/html/index.html
>>
>> The reason is that these implementations appear to be made with more
>> care than what we use currently, and behave more correctly in corner
>> cases (cf. e.g. ticket #1740).
>>
>> To minimize work, it would be useful just to use the Boost functions
>> direcly, rather than doing manual C transcriptions. The drawback here is
>> that the portion of Boost library required weights about 8 MB of source
>> code, and we would most likely like to bundle it, as it is not really a
>> standard part of many installations. This does not reflect much on the
>> compiled binary size, however.
>>
>> I'm not 100 % certain about the compiler support. Perhaps C++ is already
>> mature enough to work across the platforms we care about.
>>
>> I'm not aware of many good BSD-compatible floating-point special
>> function libraries, so if you know others, or would be opposed to
>> bundling Boost, please chime up!
>
> At SciPy2011 a few of us managed to convince Andreas Klöckner to
> bundle the necessary bits of boost with PyCUDA so as to remove a
> dependency and grow the user-base. I recall this causing trouble for
> some segment of his users that had boost installed on the system, but
> I don't know the details.
>
> Andreas (CCed): Do you remember what problems those were? Fred B. told
> me about it over lunch one day but I can't recall what exactly went
> wrong.

Two main failure modes:

- User has boost headers in /usr/include/boost. If stuff goes wrong,
  you'll build the binary parts of boost that you ship (whether you have
  those depends on which of the libraries you use) against the
  non-matching headers on the user's system. This usually results in
  missing symbols, but could conceivably lead to crashes.

  This doesn't seem to happen too much if you make sure that you have a
  -I for your shipped headers (i.e. only old/broken compilers).

- Two modules using their individual shipped versions of boost::python
  (if you plan on using that) get loaded. Really weird stuff starts
  happening. The fix here is to -Dboost=mypackageboost. Strangely, this
  works without a hitch. That way, the two live in different C++
  namespaces. They won't know about each other, but they also won't step
  on each other's feet.

I've found both of those to be pretty benign.

See here:
https://github.com/inducer/aksetup/blob/master/aksetup_helper.py#L534
and here:
https://github.com/inducer/bpl-subset

for my work on this. ("bpl" = "boost python library"). do-extract here
uses bcp, which refers to this:

http://www.boost.org/doc/libs/1_51_0/tools/bcp/doc/html/index.html

HTH,
Andreas