[Wheel-builders] [Matplotlib-devel] Manylinux external library test case - matplotlib, tk

[Matthew Brett]

On Tue, May 17, 2016 at 4:23 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Wed, May 11, 2016 at 2:26 PM, Leonardo Rochael Almeida
> <leorochael at gmail.com> wrote:
>> On 11 May 2016 at 10:58, Matthew Brett <matthew.brett at gmail.com> wrote:
>>>
>>> On Tue, May 10, 2016 at 6:38 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>> > On Tue, May 10, 2016 at 3:09 PM, Leonardo Rochael Almeida
>>> > <leorochael at gmail.com> wrote:
>>> >> On 10 May 2016 at 18:52, Matthew Brett <matthew.brett at gmail.com> wrote:
>>> >>>
>>> >>> On Tue, May 10, 2016 at 5:26 PM, Matthew Brett
>>> >>> <matthew.brett at gmail.com>
>>> >>> wrote:
>>> >>> >> [...]
>>> >>> >>
>>> >>> >> This would probably look like:
>>> >>> >>
>>> >>> >> _tkagg.so has a DT_NEEDED entry naming tkinter.so (or whatever
>>> >>> >> Python
>>> >>> >> calls
>>> >>> >> this module)
>>> >>> >>
>>> >>> >> Before loading _tkagg.so, we use Python level introspection figure
>>> >>> >> out
>>> >>> >> where
>>> >>> >> tkinter.so lives
>>> >>> >>
>>> >>> >> We add its directory to LD_LIBRARY_PATH
>>> >>> >>
>>> >>> >> we import _tkagg.so
>>> >>> >>
>>> >>> >> We take its directory back off of LD_LIBRARY_PATH
>>> >>> >>
>>> >>> >> Very weird, but I can't see why it wouldn't work, and probably more
>>> >>> >> reliable
>>> >>> >> than anything where we try to reimplement the dynamic loader's
>>> >>> >> search
>>> >>> >> logic
>>> >>> >> ourselves.
>>> >>> >
>>> >>> > Nice - yes - it does work in a first-pass test - I'll look into
>>> >>> > automating that.
>>> >>>
>>> >>> Ah - except I do not believe it is possible to change the linker path
>>> >>> for Python modules, within the Python process:
>>> >>>
>>> >>> http://www.gossamer-threads.com/lists/python/python/393770#393770
>>> >
>>> > Doh :-(
>>>
>>> Any other suggestions?
>>
>>
>> Calling `patchelf` at runtime with the result of introspecting the
>> `_tkinter.so` module? :-)
>>
>> On a more serious note, even though it would be dangerous to call
>> `sys.setdlopenflags()` with `RTLD_GLOBAL`, we could perhaps manually call:
>>
>> ctypes.CDLL("/path/to/introspected/libtk.so", DLFCN.RTLD_GLOBAL)
>> ctypes.CDLL("/path/to/introspected/libtcl.so", DLFCN.RTLD_GLOBAL)
>>
>> before importing a `_tkagg.so` with `libtk` and `libtcl` patchelf-removed
>> (but not re-added).
>>
>> We would be dumping only libtcl/tk's symbols in the namespace of other
>> python extension modules.
>
> I have implemented dlopen dynamic loading of the TCL / Tk functions
> from the Python tkinter extension module here :
> https://github.com/matplotlib/matplotlib/pull/6442
>
> It appears to work.   I'd love any feedback y'all can offer.

I patched matplotlib 1.5.1 with a version of the PR at [1]  to build
the wheels using the script at [2] and travis-ci repo build at [3].

The built wheels are up at
http://ccdd0ebb5a931e58c7c5-aae005c4999d7244ac63632f8b80e089.r77.cf2.rackcdn.com/
for testing.

They work for me, for example doing tkagg animations.

Cheers,

Matthew

[1] https://github.com/matplotlib/matplotlib/pull/6442
[2] https://github.com/matthew-brett/manylinux-builds/blob/master/build_matplotlibs.sh
[3] https://travis-ci.org/matthew-brett/manylinux-builds/builds/132304156