[Cython] Multiple modules in one compilation unit

[Lisandro Dalcin]

On 2 March 2011 13:01, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Dag Sverre Seljebotn, 02.03.2011 16:37:
>>
>> On 03/02/2011 04:11 PM, Lisandro Dalcin wrote:
>>>
>>> On 2 March 2011 08:35, Stefan Behnel<stefan_ml at behnel.de> wrote:
>>>>
>>>> Dag Sverre Seljebotn, 02.03.2011 11:54:
>>>>>
>>>>> Problem is that Fortran code often has...interesting...programming
>>>>> practices. Global variables abound, and are often initialised between
>>>>> modules. Imagine:
>>>>>
>>>>> settings_mod.set_alpha(0.34)
>>>>> print compute_mod.get_alpha_squared()
>>>>>
>>>>> This behaves quite differently with two static versions rather than
>>>>> one...
>>>>
>>>> Then I'd suggest always linking dynamically.
>>>>
>>> And where are you going to put your fortran shared libraries? Dynamic
>>> linking details varies wildly across platforms... I'm very much
>>> understand Dag's use case and concerns, and I do think that some
>>> research in all this is worth it.
>>
>> I'm not sure if there's much more to research at the moment -- Stefan says
>> it is possible, and that's what I wanted to know at this stage. If I want
>> it, I obviously need to implement it myself. (And if such a patch
>> implements PEP 3121 and there's a demonstrated need for it with some
>> users,
>> I really can't see it getting rejected just out of it being in "poor
>> taste").
>>
>> I.e., I'm going to make Fwrap spit out multiple pyx files and worry about
>> this later. If multiple .pyx in one .so was fundamentally impossible, I
>> might have gone another route with Fwrap. That was all.
>
> The feature I could imagine becoming part of Cython is "compiling packages".
> I.e. you'd call "cython" on a package and it would output a directory with a
> single __init__.so that contains the modules compiled from all .pyx/.py
> files in that package. Importing the package would then trigger an import of
> that __init__.so, which in turn will execute code in its init__init__()
> function to register the other modules.
>

$ cd /tmp/
$ mkdir pkg
$ touch pkg/__init__.so
$ python -c 'import pkg'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named pkg


So that's not going to work... The import machinery ignores
__init__.so files. Instead, you need a "package.so" file, the
initpackage() init function should in turn setup the package structure
and initialize submodules. I've sent privately to Dag an example of
how this should be done.




-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169