[Wheel-builders] manylinux wheel troubles using C++ and streaming exceptions

[Vitaly Kruglikov]

>>>>>>>>

From: Wheel-builders <wheel-builders-bounces+vitaly.krugl.numenta=gmail.com at python.org<mailto:wheel-builders-bounces+vitaly.krugl.numenta=gmail.com at python.org>> on behalf of Vitaly Kruglikov <vkruglikov at numenta.com<mailto:vkruglikov at numenta.com>>
Date: Saturday, August 6, 2016 at 2:43 AM
To: "wheel-builders at python.org<mailto:wheel-builders at python.org>" <wheel-builders at python.org<mailto:wheel-builders at python.org>>
Subject: [Wheel-builders] manylinux wheel troubles using C++ and streaming exceptions

I added the ability to create a manylinux wheel for the nupic.core project (github.com/numenta/nupic.core). However, in testing, I found that some exception-handling logic that used to work is now failing when running the wheel on Ubuntu 16.04, which uses the gcc/g++ 5.4.0 toolchain. One example in particular is the failure to catch the exception std::ios::failure.

This makes it impractical to create manylinux wheels from legacy C++ code that also, as in the case of nupic.core, includes globs of additional 3rd party code. It's not practical because the incompatibilities may not be limited to just this one exception class and the actual failures are unreasonably difficult to simulate in testing, making it difficult, if not impossible, to find all occurrences in one's own code, not to mention 3rd party code that it links with.

It turns out that, among other things, "the C++11 standard mandated an ABI change for std::ios_base::failure, by giving it a std::system_error base class that wasn't present in C++03" (see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66145). The 4.8.2 compiler toolchain on the manylinux docker image supports C++11 in spirit, but doesn't implement the new ABI (its std::ios_base::failure is derived from std::exception); however, the 5.4.0 toolchain on Ubuntu 16.04 implements the new ABI (derives from std::system_error, etc.), and its stdc++ library is compiled using the new toolchain. So, the signature of std::ios_base::failure compiled into the manylinux wheel doesn't match the signature of std::ios_base::failure that's raised by the stdc++ library on Ubuntu 16.04.

The following simple app demonstrates the issue:

```
#include <iostream>
#include <fstream>

int main() {

  try {
    std::ifstream input("notafile");
    input.exceptions(std::ifstream::failbit | std::ifstream::badbit);
    input.get();
  } catch(const std::ios::failure &e) {
    std::cout << "caught a std::ios::failure: what=" << e.what()
      << '\n';
  }
}
```

First, we build and run it on the manylinux docker image:
$ docker run -it -v /home/vitaly/localbuilds/ios-base-failure:/ios-base-failure quay.io/pypa/manylinux1_x86_64 bash
[root at 39eecd65e630 ios-base-failure]# g++ -std=c++11 test.cpp

[root at 39eecd65e630 ios-base-failure]# ./a.out
caught a std::ios::failure: what=basic_ios::clear

As you can see above, running it on the manylinux container works correctly: it catches std::ios::failure and prints the expected message to stdout.

However, when we run the same executable on Ubuntu 16.04, we get a very different result below. We see that the exception wasn't caught.

g++ (Ubuntu 5.4.0-6ubuntu1~16.04.1) 5.4.0 20160609
('Ubuntu', '16.04', 'denial')

vitaly at ubuntuvm:~/localbuilds/ios-base-failure$ ./a.out
terminate called after throwing an instance of 'std::ios_base::failure[abi:cxx11]'
  what():  basic_ios::clear: iostream error
Aborted (core dumped)

Vitaly
<<<<<<<<<<<<<



It turns out that this problem was due to C++ ABI incompatibility. In case anyone runs into a similar issue, the solution is described here: https://discourse.numenta.org/t/segmentation-fault-while-running-basic-swarm/877/24?u=vkruglikov

Best,
Vitaly


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/wheel-builders/attachments/20160817/598c1011/attachment.html>