Pyrex and Distuils: an enhanced build_ext command

[Graham Fawcett]

"Thomas Heller" <theller at python.net> wrote in message news:<agh6k6$ltutn$1 at ID-59885.news.dfncis.de>...
> "Graham Fawcett" <gmfawcett at operamail.com> wrote in message news:3d799735.0207100216.4eb8a9cd at posting.google.com...
> >                     # compiling with mingw32 gets an "initializer not
> > a constant" error
> >                     #
> > http://www.python.org/cgi-bin/faqw.py?req=show&file=faq03.024.htp
> >                     # doesn't appear to happen with MSVC
> >                     # so if we are compiling with mingw32, massage the
> > Pyrex-generated
> >                     # C files to compile properly
> >
>
> Isn't this problem (initializer not a constant) solved by compiling
> the file in C++ mode (with .cpp extension)?
>
> Thomas

Yes -- at first I couldn't get c++ compilation to work, so I tried the
other approach suggested in the FAQ.

I just spent some time trying again, and I think I've got the c++
approach working. I still need to massage the Pyrex-generated C files,
but the code is much cleaner.

I also saw your suggestions re: adding command classes using the
canoncial approach, and the extension point at swig_sources(), and
have simplified my code accordingly.

Here are the revised files:

#######################################
#Pyrex.Distutils.__init__.py

from build_ext import build_ext

#######################################
#Pyrex.Distutils.build_ext.py
#
# Subclasses disutils.command.build_ext,
# replacing it with a Pyrex version that compiles pyx->c
# before calling the original build_ext command.
# July 2002, Graham Fawcett
# Pyrex is (c) Greg Ewing.

import distutils.command.build_ext
import Pyrex.Compiler.Main
from Pyrex.Compiler.Errors import PyrexError
from distutils.dep_util import newer
import os
import sys

class build_ext (distutils.command.build_ext.build_ext):

    description = "compile Pyrex scripts, then build C/C++ extensions
(compile/link to build directory)"

    def finalize_options (self):
        distutils.command.build_ext.build_ext.finalize_options(self)

        # compiling with mingw32 gets an "initializer not a constant"
error
        # doesn't appear to happen with MSVC!
        # so if we are compiling with mingw32,
        # switch to C++ mode, to avoid the problem
        if self.compiler == 'mingw32':
            self.swig_cpp = 1

    def swig_sources (self, sources):
        if not self.extensions:
            return

        # collect the names of the source (.pyx) files
        pyx_sources = []
        pyx_sources = [source for source in sources if
source.endswith('.pyx')]

        extension = self.swig_cpp and '.cpp' or '.c'
        for pyx in pyx_sources:
            # should I raise an exception if it doesn't exist?
            if os.path.exists(pyx):
                source = pyx
                target = source.replace('.pyx', extension)
                if newer(source, target) or self.force:
                    self.pyrex_compile(source)

                    if self.swig_cpp:
                        # rename .c to .cpp (Pyrex always builds .c
...)
                        if os.path.exists(target):
                            os.unlink(target)
                        os.rename(source.replace('.pyx', '.c'),
target)
                        # massage the cpp file
                        self.c_to_cpp(target)

        return [src.replace('.pyx', extension) for src in pyx_sources]

    def pyrex_compile(self, source):
        try:
            Pyrex.Compiler.Main.compile(source, c_only=1,
use_listing_file=0)
        except PyrexError, e:
            print e
            sys.exit(1)

    def c_to_cpp(self, filename):
        """touch up the Pyrex generated c/cpp files to meet
mingw32/distutils requirements."""
        f = open(filename, 'r')
        lines = [line for line in f.readlines() if not
line.startswith('staticforward PyTypeObject __pyx_type_')]
        f.close()
        f = open(filename, 'w')
        lines.insert(1, 'extern "C" {\n')
        lines.append('}\n')
        f.write(''.join(lines))
        f.close()

#######################################
# a sample setup.py
from distutils.core import setup
from distutils.extension import Extension
from Pyrex.Distutils import build_ext

setup(name='foo',
    ext_modules=[ Extension("foo", ["foo.pyx"]) ],
    cmdclass = {'build_ext': build_ext}
)

#######################################

Tested on Win32, msvc and mingw32.

I'm happy so far -- the revision is shorter, cleaner and more
explicit.

Comments are welcome!

Cheers,

-- Graham