[Wheel-builders] manylinux compile/optimization options

[Vlad K.]

Hello list,

I need to understand if there are any specific compilation options set 
for the manylinux precompiled binaries, by the packaging system or by 
individual python module developers.

We have an image processing application that does certain color 
correction, resizing and watermarking of images for a webapp. The 
sources are quite large (some as large as 100 megapixels) and the 
execution time is very noticeable, some images taking up to a minute to 
complete (yes we have post-processing cache, but that initial hit is 
crucial).

I have noticed a difference in execution time of our test and production 
cases between Linux (Gentoo) and FreeBSD. Takes the same code on FreeBSD 
3 times more to process. Investigating further I realized on Linux that 
pip is pulling-in already compiled .so libraries, whereas on FreeBSD it 
compiles locally (testing with virtualenv and installation with pip).

So when I reinstalled on Linux but ignoring the precompiled .so libs 
(with --no-binary :all: given to pip), the execution time shot up 3x 
(longer) and equals that of FreeBSD. But, when I recompiled again, but 
forcing CFLAGS="-O3" (just tried as a test, and that worked right off 
teh bat), the execution time came down to equal the one when installed 
from PyPi wheel cache. Same thing happened when I did so on FreeBSD.

So adding -O3 to CFLAGS when building Pillow and ignoring precompiled 
libs from wheel cache, improved execution time (of our use case) 3 
times!

Now, since -O3 is not a wise thing to do unless you know what you're 
doing (and I don't, in this case), I need to find out if that's what was 
added to the PyPi wheel cache, or something else, and where was that 
defined as I could find no relevant CFLAGS definitions in the Pillow 
source code (or maybe it's there but I just can't see it), or in fact 
any compilation flags other than -O0 for the test cases.

I also need to understand this so I can force the same compilation 
environment for the FreeBSD port of Python Pillow module 
(graphics/py-pillow).

Thanks.


-- 
Vlad K.