[Pythonmac-SIG] Building extension modules with Xcode 5

[Ned Deily]

On Oct 28, 2013, at 11:46 , Andrew Barnert <abarnert at yahoo.com> wrote:

> Sorry, I've been trying to reproduce this, and I haven't identified exactly what the problem is, except that you have to upgrade Xcode from 4.x to 5.0.1 after upgrading to 10.9 and without upgrading/reinstalling Python. If I do that, _sometimes_ it doesn't work, with failures looking for gcc-4.2. If I do any other sequence, it always succeeds. I'll keep trying to figure out the repro case.


Thanks, Andrew, for your efforts!  I *really* want to get to the bottom of this issue.

I assume you know this but, for the benefit of others new to Xcode 5 and Mavericks: Apple has changed some things about the Command Line Tools component.  The new way to install the CLT in 10.9 is:
 
   xcode-select --install

TL;DR Just run xcode-select --install.

For some time now (long before Xcode 5), OS X has not shipped with a full set of the usual header files, development libraries, and build tools installed, e.g. not in /usr/include, /System/Library/Frameworks/.., /usr/bin etc.  They are provided in SDKs, one for each OS X release.  If you are using Xcode.app to build a project, Xcode takes care of supplying the correct locations of header files in SDKs and various build tools.  One advantage of not installing files and depending on a fixed path to them is that it simplifies building projects for different target platforms: different versions of OS X or, more importantly, iOS vs OS X.  Because of that, Apple has seen fit to change the locations of things between Xcode releases.  However, many open source projects, like Python, expect things to be in their "traditional" Unix file system locations, like /usr/bin/cc or /usr/include/stdio.h.  To accommodate such usage, Xcode has also long provided an option to install these files from an SDK into their tradition location under the system root, /.  This component has had various names within Xcode and in the past was selected by an installer option then more recently via an Xcode.app -> Preferences option.  Also, in Xcode 4, bowing to popular demand for a smaller download, Apple made available a standalone version of the Command Line Tools that could be downloaded and installed without the need for downloading all of Xcode.  Python, like most similar open source projects, requires the Command Line Tools to be installed, either via Xcode or via the standalone installer.  More accurately, Python requires the CLT for two purposes: 1. building C or C++ extension modules with Distutils; 2. building Python itself.  If you are using a binary-installed Python and don't need to install any third-party extension modules from source, you don't actually need the CLT.

For Mavericks, a big initial source of confusion has been how the CLT get installed.  10.9 comes with a set of stub files for the CLT build tools, like /usr/bin/cc, /usr/bin/gcc, /usr/bin/clang, /usr/bin/make.  The first time you try to run one of these tools, a pop-up window appears and asks if you want to install the Command Line Tools or install all of Xcode.  If you pick the CLT option, an installer package will be automatically downloaded and installed via the system installer and, in the future, should be updated via the standard Software Update mechanism like other parts of OS X.  When it works, it's very slick.  If for some reason, it doesn't work, you may be stumped as to what to do.  In the Xcode 5 app, there no longer appears to be a Preferences option to install the CLT component.  This has led to a lot of confusion when people look for things in /usr/include and find nothing there:  "Apple has broken everything.  No /usr/include".  (This was exacerbated by a misstep by Apple during the Mavericks development cycle that was changed before the release.)  The solution to this problem and many other problems is to force the install of the CLT and the new way to do it in 10.9 Mavericks is: xcode-select --install.

--
  Ned Deily
  nad at acm.org -- []