[New-bugs-announce] [issue25027] Python 3.5.0rc3 on Windows can not load more than 127 C extension modules

[Christoph Gohlke]

New submission from Christoph Gohlke:

This issue was first mentioned at <http://bugs.python.org/issue24872#msg249591>.

The attached script (test_dll_load_failed.py) builds up to 256 C extension modules and imports them. On Python 3.4.3 the script passes while on Python 3.5.0rc3 the script fails with:

```
Traceback (most recent call last):
File "test_dll_load_failed.py", line 42, in <module>
import_module(name)
File "X:\Python35\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
File "<frozen importlib._bootstrap>", line 577, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 903, in create_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
```

Tested on Windows 7 and 10, 64 bit, with Python 3.5.0rc3, 32 and 64 bit.

Due to this issue the "scientific stack" is practically unusable. For example, Pandas unit tests error or abort. In a Jupyter notebook, the following simple imports fail (using current builds from <http://www.lfd.uci.edu/~gohlke/pythonlibs/>):

```
In [1]:

import matplotlib.pyplot
import pandas
import statsmodels.api
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-69fcce4de550> in <module>()
      1 import matplotlib.pyplot
      2 import pandas
----> 3 import statsmodels.api

<snip>

X:\Python35\lib\site-packages\scipy\signal\__init__.py in <module>()
    276 # The spline module (a C extension) provides:
    277 #     cspline2d, qspline2d, sepfir2d, symiirord1, symiirord2
--> 278 from .spline import *
    279 
    280 from .bsplines import *

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
```

The cause of this issue is that as of Python 3.5.0rc1 C extension modules are linked statically to the multi-threaded runtime library (/MT) instead of the multi-threaded DLL runtime library (/MD). A process can not load more than 127 statically-linked CRT DLLs using LoadLibrary due to a limit of fiber-local storage (FLS) as mentioned in the following links:

<http://stackoverflow.com/questions/1437422>
<https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/3546c3c4-1b36-4552-85c5-1b3ba860ee84>

To put the 127 limit in perspective: the pywin32 package contains 51 C extension modules, pygame 36, scipy 65, and scikit-image 41. 

In addition to C extension modules, the 127 limit also applies to statically-linked CRT DLLs that are dynamically loaded via Ctypes or LoadLibrary.

----------
components: Extension Modules, Windows
messages: 250169
nosy: cgohlke, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python 3.5.0rc3 on Windows can not load more than 127 C extension modules
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25027>
_______________________________________