[Python-Dev] buildin vs. shared modules

["Martin v. Löwis"]

Guido van Rossum wrote:

>>I don't see why it matters, though. Adding modules to pythonxy.dll does 
>>not increase the memory consumption if the modules are not used.
> 
> 
> Can you explain why not?  Doesn't the whole DLL get loaded into
> memory?

No. In modern operating systems (including all Win32 implementations,
i.e. W9x and NT+), the code segment is *mapped* instead of being loaded
(in Win32 terminology, by means of MapViewOfFileEx). This causes
demand-paging of the code, meaning that code is only in memory if it is
actually executed.

There are some pitfalls, e.g. that paging operates only on 4k (on x86)
granularity, and that relocations may cause the code to get loaded at
startup time instead of at run-time (in the latter case, it also stops
being shared across processes).

The code still consumes *address space*, but of this, any process has
plenty (2GB on Win32, unless you use the /3GB boot.ini option of W2k+).

The same is true for executables and shared libraries on Unix, meaning
that making extension modules shared libraries does not reduce memory
consumption. It may increase it, as code segments are 4k-aligned, so if
you have many small segments, you may experience rounding overhead.

Regards,
Martin