[Pythonmac-SIG] building universal binaries

[Ronald Oussoren]

On 27-jan-2006, at 8:46, Kevin Ollivier wrote:

> Hi Bob,
>
> On Jan 26, 2006, at 10:55 PM, Bob Ippolito wrote:
>
> [snip]
>
>>> Sorry, I wasn't really thinking about extensions. By Panther I  
>>> did mean building against the 10.3.9 SDK, which would give the  
>>> desired results for the Python binary itself, but as you said  
>>> extending that to correctly build extensions that did the same  
>>> thing would be a royal PITA.
>>
>> You still need to build against two different SDKs, which can't be  
>> done with just compiler flags.  You have to do two separate  
>> compiles, and then lipo it together.  At that point you might as  
>> well use different compilers and target older versions of OS X.
>
> Yes, this was what I meant in my previous email about letting the  
> SDK get specified at configure time. I was thinking to have bash  
> script that automates the various steps. I guess you could put it  
> into the configure script, but that does sound to me to be pretty  
> painful.

A bash script? Heresy  :-) Why not a python script. We can be pretty  
sure that python is available at build time.

>
>> The other thing is that Python's build probably detects a few  
>> things at runtime, and at runtime it's seeing 10.4, so it might  
>> not actually work on the target version of OS X.
>
> Yeah, if this is the case it'd be a real mess. ;-/

That's more problematic on Panther :-). Python's configure has  
hardcoded that OSX 10.2 and 10.4 must not have _POSIX_SOURCE and  
friends defined. Annoyingly this means that if you build Python on  
Panther you'll have to patch the build if you want to build  
extensions on Tiger. I'm sitting on yet another patch that adds OSX  
10.3 to the list.

>
>> Ideally all that stuff would be pushed to runtime checks, so that  
>> it could choose a different code path based upon which APIs are  
>> available... but first you have to isolate the instances where  
>> that happens.
>>
>>>> I think the only approach that doesn't require hacking a  
>>>> significant amount of Python's build process and distutils is to  
>>>> have two Python installations, one x86 only and one PPC only.   
>>>> For distribution, you would cook up some way to lipo it all  
>>>> together.
>>>
>>> I can see how the issues with the strategy I outlined would be  
>>> significant, but I don't see how Universal support on Tiger would  
>>> be similarly difficult. Apple's Python 2.3.5 already is  
>>> Universal, and I'm unaware of any issues with it (except that it  
>>> doesn't build Universal extensions ;). The changes they made to  
>>> the build process seemed to be fairly minimal.
>>
>> Yeah, a 10.4+ universal build wouldn't be too big of a deal  
>> because you can do that with one invocation of GCC and one set of  
>> flags.
>>
>>> As far as distutils/working with extensions, I admit I'm not very  
>>> familiar with that. I did notice when compiling wxPython though  
>>> that there weren't any distutils flags passed to gcc that would  
>>> conflict with the building of Universal binaries. The main  
>>> question in my mind would be whether or not the built Universal  
>>> extension libraries could be loaded by Python's dynamic library  
>>> loader. If they can be loaded, and we could add the flags to  
>>> distutils' default compiler/linker flags, what other major issues  
>>> would need to be overcome?
>>
>> Some extensions aren't going to build cleanly universal, and most  
>> users probably aren't going to have all the SDKs installed so if  
>> it shipped with universal flags then it wouldn't be able to build  
>> extensions.
>
> Unless we had distutils check for the existence of the Universal  
> SDK before setting the flags, or probably more accurately, remove  
> them if the SDK isn't available. Then perhaps add a distutils  
> 'flag' to manually turn off Universal building for extensions that  
> can't be fixed. This might be a bit of a hack, but I don't imagine  
> it would be too painful. The question is how many extensions would  
> have troubles... But I guess the only way we'll know is to test the  
> waters. In any case, even if it wouldn't be reasonable to make a  
> Universal build the "official" build, it would be very useful to  
> people packaging apps, even if it were Tiger+ only.

I know of one popular extension (cElementTree) that won't build with  
'-arch i386 -arch ppc' because it calcutes a #define based on the  
current byte-order. There might be others, I haven't done a thorough  
search yet.

>
> As far as the SDK dependency issue, IMHO, not everyone would need  
> to build Universal - the key would be for distributors/packages to  
> make Universal binaries, and I think it would be reasonable for us  
> to tell them "have the 10.4u SDK and DevTools installed". (And  
> maybe the Panther/Jaguar SDK assuming we could sort the issues you  
> mentioned above out...) Again, assuming a Universal build does make  
> a sensible default.

Note that this means "install Tiger" which not everyone might want to.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/pythonmac-sig/attachments/20060127/6dd235ab/attachment.bin