non-standard glibc location

[Sandi Cimerman]

On Friday, September 8, 2017 at 3:16:12 AM UTC+2, Fetchinson . wrote:
> On 9/7/17, Thomas Jollans <tj... at tjol.eu> wrote: 
> > On 2017-09-06 16:14, Fetchinson . via Python-list wrote: 
> >> Hi folks, 
> >> 
> >> I'm trying to install a binary package (tensorflow) which contains 
> >> some binary C extensions. Now my system glibc is 2.15 but the binaries 
> >> in the C extensions were created (apparently) with glibc 2.17. So I 
> >> thought no problemo I installed glibc 2.17 to a custom location, built 
> >> python2.7 from source (hopefully using my custom glibc) and installed 
> >> pip and everything else using this custom built python. But still when 
> >> I try to import tensorflow I get: 
> >> 
> >> ImportError: /lib64/libc.so.6: version `GLIBC_2.17' not found 
> >> (required by 
> >> /home/nogradi/fetch/custom/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so) 
> >> 
> >> So apparently it's trying to use my system glibc, not the custom one. 
> >> 
> >> How do I tell this extension to use the custom glibc? Is it even 
> >> possible? 
> > 
> > It's going to use the same libc as python, so first of all check which 
> > libc your python interpreter is actually linked to. Maybe your custom 
> > build didn't do quite what you wanted. 
> > 
> > ldd `which python` # or something like that 
> > 
> > Once you've convinced yourself that python has the correct libc, you 
> > could try building tensorflow from source rather than installing the 
> > binaries. 
> > 
> > Maybe something in here helps: 
> > https://github.com/tensorflow/tensorflow/issues/53
> Thanks a lot for all the comments, my problem was indeed that the 
> compiled python was still using the system glibc. The solution was to 
> set the environment variables 
> 
> p=/path/to/custom/glibc 
> export CFLAGS=-I${p}/include 
> export LDFLAGS="-Wl,--rpath=${p}/lib 
> -Wl,--dynamic-linker=${p}/lib/ld-linux-x86-64.so.2" 
> 
> And then the compiled python was using the new glibc. 
> 
> Side note 1 on tensorflow: the compiled tensorflow binary uses unicode 
> ucs4, so for python I had to ./configure --enable-unicode=ucs4 because 
> the default is ucs2 
> 
> Side note 2 on tensorflow: it also depends on libstdc++ and my version 
> was too old for that as well. Instead of compiling gcc from source 
> (which includes libstdc++) I copied a binary libstdc++ from a newer 
> linux distro and it was working fine. 
> 
> And the reason, if anyone cares, I had to go through the above is that 
> I couldn't compile tensorflow from source. 
> 
> Thanks again, 
> Daniel
> > 
> >> 
> >> But maybe I have an even more basic issue: how do I link python not 
> >> with the system glibc but with my custom glibc? 
> >> 
> >> Cheers, 
> >> Daniel 
> >> 
> >> 
> >> 
> > 
> > 
> > -- 
> > Thomas Jollans
> > -- 
> > https://mail.python.org/mailman/listinfo/python-list
> > 
> 
> 
> -- 
> Psss, psss, put it down! - http://www.cafepress.com/putitdown

First of all, thank you a lot Fetchinson, it helped on my system (Centos 6) to today recompile the newest Python (3.10.7)!
Of course I had foss2018 available on the system, so I "just" had to compile glibc, zlib, and openssl 1.1.1q ... but hey it was worth it! I have my environment up and running!

FYI:
1. to me it helped that since I had nonstandard openssl location to edit Modules/Setup.dist as suggested here https://stackoverflow.com/questions/5937337/building-python-with-ssl-support-in-non-standard-location
2. to me it helped to configure with
./configure --with-libc=/tmp/glibc/lib/libc.so --prefix=/tmp/python --with-openssl=/tmp/openssl

Hope this helps someone else who will try to do the same.

Cheers