[Wheel-builders] [Matplotlib-devel] Manylinux external library test case - matplotlib, tk

[Matthew Brett]

On Mon, May 23, 2016 at 3:37 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> On Mon, May 23, 2016 at 12:53 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>> On Tue, May 17, 2016 at 4:23 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>>> Hi,
>>>
>>> On Wed, May 11, 2016 at 2:26 PM, Leonardo Rochael Almeida
>>> <leorochael at gmail.com> wrote:
>>>> On 11 May 2016 at 10:58, Matthew Brett <matthew.brett at gmail.com> wrote:
>>>>>
>>>>> On Tue, May 10, 2016 at 6:38 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>>>> > On Tue, May 10, 2016 at 3:09 PM, Leonardo Rochael Almeida
>>>>> > <leorochael at gmail.com> wrote:
>>>>> >> On 10 May 2016 at 18:52, Matthew Brett <matthew.brett at gmail.com> wrote:
>>>>> >>>
>>>>> >>> On Tue, May 10, 2016 at 5:26 PM, Matthew Brett
>>>>> >>> <matthew.brett at gmail.com>
>>>>> >>> wrote:
>>>>> >>> >> [...]
>>>>> >>> >>
>>>>> >>> >> This would probably look like:
>>>>> >>> >>
>>>>> >>> >> _tkagg.so has a DT_NEEDED entry naming tkinter.so (or whatever
>>>>> >>> >> Python
>>>>> >>> >> calls
>>>>> >>> >> this module)
>>>>> >>> >>
>>>>> >>> >> Before loading _tkagg.so, we use Python level introspection figure
>>>>> >>> >> out
>>>>> >>> >> where
>>>>> >>> >> tkinter.so lives
>>>>> >>> >>
>>>>> >>> >> We add its directory to LD_LIBRARY_PATH
>>>>> >>> >>
>>>>> >>> >> we import _tkagg.so
>>>>> >>> >>
>>>>> >>> >> We take its directory back off of LD_LIBRARY_PATH
>>>>> >>> >>
>>>>> >>> >> Very weird, but I can't see why it wouldn't work, and probably more
>>>>> >>> >> reliable
>>>>> >>> >> than anything where we try to reimplement the dynamic loader's
>>>>> >>> >> search
>>>>> >>> >> logic
>>>>> >>> >> ourselves.
>>>>> >>> >
>>>>> >>> > Nice - yes - it does work in a first-pass test - I'll look into
>>>>> >>> > automating that.
>>>>> >>>
>>>>> >>> Ah - except I do not believe it is possible to change the linker path
>>>>> >>> for Python modules, within the Python process:
>>>>> >>>
>>>>> >>> http://www.gossamer-threads.com/lists/python/python/393770#393770
>>>>> >
>>>>> > Doh :-(
>>>>>
>>>>> Any other suggestions?
>>>>
>>>>
>>>> Calling `patchelf` at runtime with the result of introspecting the
>>>> `_tkinter.so` module? :-)
>>>>
>>>> On a more serious note, even though it would be dangerous to call
>>>> `sys.setdlopenflags()` with `RTLD_GLOBAL`, we could perhaps manually call:
>>>>
>>>> ctypes.CDLL("/path/to/introspected/libtk.so", DLFCN.RTLD_GLOBAL)
>>>> ctypes.CDLL("/path/to/introspected/libtcl.so", DLFCN.RTLD_GLOBAL)
>>>>
>>>> before importing a `_tkagg.so` with `libtk` and `libtcl` patchelf-removed
>>>> (but not re-added).
>>>>
>>>> We would be dumping only libtcl/tk's symbols in the namespace of other
>>>> python extension modules.
>>>
>>> I have implemented dlopen dynamic loading of the TCL / Tk functions
>>> from the Python tkinter extension module here :
>>> https://github.com/matplotlib/matplotlib/pull/6442
>>>
>>> It appears to work.   I'd love any feedback y'all can offer.
>>
>> I patched matplotlib 1.5.1 with a version of the PR at [1]  to build
>> the wheels using the script at [2] and travis-ci repo build at [3].
>>
>> The built wheels are up at
>> http://ccdd0ebb5a931e58c7c5-aae005c4999d7244ac63632f8b80e089.r77.cf2.rackcdn.com/
>> for testing.
>>
>> They work for me, for example doing tkagg animations.
>
> Travis testing of these wheels, and some errors, reported at
> https://github.com/matplotlib/matplotlib/issues/6469

Thanks Olivier for testing.

Thomas Caswell gave me the pointers I needed to fix the test failures,
tests now passing at :
https://travis-ci.org/matthew-brett/manylinux-testing/builds/132577898

I think these wheels are now ready to ship - MPL issue at :
https://github.com/matplotlib/matplotlib/issues/6473

Cheers,

Matthew