[C++-sig] Multiple modules in a single pyd

[Niall Douglas]

On 15 Nov 2011 at 10:00, Olivier Voyer wrote:

> What if I have this big C++ project that I cannot split in multiple smaller
> projects? I have no choice but to create a big Python module exposing all
> the functions/classes? What is the common practice?

You are aware, I assume, that the python wrappings can live in a 
separate DLL/SO than the thing you are wrapping?

So, you can have a monolithic big C++ project DLL/SO several dozen 
megabytes in size, but with multiple wrapper DLL/SO's wrapping just a 
portion of the APIs provided by the monolithic DLL/SO. Each can be 
loaded, as needed, by the python runtime.

Generally speaking, one wants to try and keep symbol counts low when 
possible. Dynamic linkers have become much better in recent years at 
becoming O(N) with symbol count, but there still a few O(N^2) 
behaviours in there. A very large DLL/SO therefore typically will be 
much slower to load in than many smaller DLL/SOs even if they 
represent the same amount of code. And on POSIX, don't forget to make 
judicious use of "-fvisibility=hidden" and 
"-fvisibility-inlines-hidden" as they can very dramatically reduce 
symbol count for the linker.

HTH,
Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.