[Cython] Multiple modules in one compilation unit

[Stefan Behnel]

Dag Sverre Seljebotn, 02.03.2011 11:54:
> On 03/02/2011 11:48 AM, Stefan Behnel wrote:
>> Dag Sverre Seljebotn, 02.03.2011 11:20:
>>> c) Somehow provide more than one module in the same compilation unit.
>>> Again, this requires the build to work correctly, but seems less dangerous,
>>> and also has the advantage of *allowing* static linking of the Fortran
>>> library, if one wants to.
>>>
>>> But is something like this possible? Could one implement
>>>
>>> cython -o assembly.c file1.pyx file2.pyx file3.pyx
>>>
>>> ...where assembly.so would contain three Python modules? (initfile1,
>>> initfile2, and so on...)
>>
>> Can't currently work because the three modules would define the same
>> static C names. This could be fixed as part of the PEP 3121 implementation:
>>
>> http://trac.cython.org/cython_trac/ticket/173
>>
>> Or it could be worked around by overriding the prefixes in Naming.py
>> (which sounds ugly).
>>
>> Generally speaking, I'm -1 on this idea. I don't see a real use case, and
>> you're saying yourself that this isn't required to make your Fortran use
>> case work either.
>
> But assuming work is spent on Cython, it *could* work? I.e. there's not a
> problem with import mechanisms

You can have as many modules in a .so as you like, see the embedding 
support, for example. You just have to find a way to import them 
explicitly, which usually requires some call into the .so (as CPython does 
when calling the initmodule() function) to let it register the modules.

Potentially, you could have one module in there that has the name of the 
.so file, and let that register the other modules automatically in its 
initmodule() function. Similar to a package's __init__.py that can import 
other modules from the package when it gets imported itself. You could 
think of your feature as a "package in a module".

Actually, if Cython had direct support for this, it could simply join the 
static definitions in the separate modules into one, including merged 
string constants etc.


>>> - If you build the Fortran code as a static library (rather common...),
>>> then each pyx file will have their own copy. This will link successfully
>>> but likely have a rather poor effect.
>>
>> So? lxml has two main modules, and if you build it statically against
>> libxml2/libxslt, you end up with two static versions of that library in
>> the two modules. Takes up some additional space, but otherwise works
>> beautifully.
>
> Problem is that Fortran code often has...interesting...programming
> practices. Global variables abound, and are often initialised between
> modules. Imagine:
>
> settings_mod.set_alpha(0.34)
> print compute_mod.get_alpha_squared()
>
> This behaves quite differently with two static versions rather than one...

Then I'd suggest always linking dynamically.

Stefan