[Cython] Sources list handling in Build/Dependencies.py:create_extension_list

[Oleksandr Kreshchenko]

Hello!

Following the "CEP 201 - Distutils Preprocessing" (http://wiki.cython.org/enhancements/distutils_preprocessing)
there are two possibilities to build a module with cythonize function in setup.py.

Second one uses list of distutils.extension.Extension class which accept all necessary compiler and
linker options along with list of non-pyx sources to compile and link with for a particular extension module.
As this way appears more natural for me I do::

     ext_modules = cythonize([Extension(
         'CyCont', ['CyCont.pyx', 'Cont.c', 'ContCet.c', 'Clash.c', 'ContInit.c', 'MemMan.c', '../Log/Log.c'],
         define_macros = [('CONT_API', '')], # None value expands to 1, whereas '' to empty
         include_dirs = dirs, #libraries = ['libgcov'],
         extra_compile_args = sys_comp_opts[sys.platform],
         extra_link_args = sys_link_opts[sys.platform])],
       show_version = 1, language_level = 3, verbose = 1, include_path = dirs)

But, only CyCont.c is created from CyCont.pyx then compiled and linked into the module CyCont.so,
but the rest of sources to be wrapped are completely ignored in the build process.
So, doing import CyCont I've got undefined symbol error.

However when I move 'CyCont.pyx' to the end of the sources list i.e.::

     ext_modules = cythonize([Extension(
         'CyCont', ['Cont.c', 'ContCet.c', 'Clash.c', 'ContInit.c', 'MemMan.c', '../Log/Log.c', 'CyCont.pyx'],

     ...

the module is built properly.

Execution flow goes though Cython/Build/Dependencies.py (latest revision)::

393 def create_extension_list(patterns, exclude=[], ctx=None, aliases=None)
...
404     for pattern in patterns:
405         if isinstance(pattern, str):
406             filepattern = pattern
407             template = None
408             name = '*'
409             base = None
410             exn_type = Extension
411         elif isinstance(pattern, Extension):
412             filepattern = pattern.sources[0]
413             if os.path.splitext(filepattern)[1] not in ('.py', '.pyx'):
414                 # ignore non-cython modules
415                 module_list.append(pattern)
416                 continue
417             template = pattern
418             name = template.name
419             base = DistutilsInfo(exn=template)
420             exn_type = template.__class__
...
423         for file in glob(filepattern):
...
437                 module_list.append(exn_type(
438                         name=module_name,
439                         sources=[file],
440                         **kwds))
...
443     return module_list

If the first list member (sources[0]) is *.pyx or *.py then the only list member is CyCont.pyx (see line 439).

Otherwise if CyCont.pyx is placed at the end of the list or, generally, the first list member is
neither *.pyx nor *.py than the loop over patterns/extensions (on line 404) breaks at the line 416
so sources list is left untouched and cythonize function (line 446) does the job.

This difference is a bug. Isn't it?

Have "sources[0]" on line 412 and "sources=[file]" on line 439 to be revised?


BTW, I catch the idea of twiking an extension in setup.py like this::

for ext in ext_modules:
     ext.define_macros = [('CONT_API', '')]
     ext.extra_compile_args = sys_comp_opts[sys.platform]
     ext.extra_link_args = sys_link_opts[sys.platform]
     ext.sources.extend(['Cont.c', 'ContCet.c', 'Clash.c', 'ContInit.c', 'MemMan.c', '../Log/Log.c'])

but it seams **better** to handle this stuff in the first place within Extension(...),
not to mention using module-level distutils directives.


Best regards,
Alex