[Python-Dev] runtime dlls on Windows

[Steve Dower]

On 26May2016 0942, Chris Barker wrote:
>     An alternative approach would be to stick MSVCP140.DLL into a tiny
>     shim wheel and upload that to PyPI, and then wxPython and matplotlib's
>     windows wheels could declare a dependency on this msvcp410 wheel.
>
>     Basically this is the idea of my pynativelib proposal,
>
>
> Yes -- if pynativelib becomes standard practice, then yes, that would be
> a good way to handle this issue -- jsut ike any other lib multiple
> packages might need.
>
>     (I really need to get back to that... if anyone wants to help sprint
>     on this at PyCon let me know :-))
>
>
> I might -- haven't decided what to sprint on yet -- let's talk. (let's
> talk anyway)

This seems like a good solution. I won't be around for sprints, but we 
can chat about it before then.

>
>         MSVCP140.DLL
>
>
>     There are two different versions of this DLL for each architecture
>     (same name).
>
>
> got to love Windows! Does the OS do the right thing if they are both
> installed? ouch!

If they're installed properly into System32/SysWOW64 (using the official 
installer), then yes. If you simply drop them into your Python install 
directory, then no, unless you drop the right one - it has to match the 
python.exe architecture.

>
>         Unfortunately, it won't "just work". More often than not, it
>         will break in weird and very difficult to diagnose ways, as the
>         version of msvcp140.dll changes on a regular basis (there are at
>         least four different version of it "in the wild" since VS 2015
>         released, not counting the preview releases before mid-last year).
>
>
> oh God. Well, that will  pretty much kill it. So essentially every
> single app or lib written in C++ has to ship this dll? And IIUC, folks
> that have the compiler installed have SOME version on the system paths,
> and who knows what version other arbitrary stuff is linked to? How can
> this possibly be considered "Enterprise" software? Wow!

To make it worse, you can only load it once per process. So the first 
extension to load their private copy wins.

The DLL is designed to be installed using the official installer, 
statically linked, or embedded as part of an entire application. It's 
not meant to be injected into someone else's application (which is 
essentially what Python extension modules are doing).

>     There's also a slippery slope argument that would legitimately
>     justify shipping all sorts of "common" dependencies with the core
>     product. I chose to draw the line at "needed by the core Python
>     product", rather than "created by Microsoft" or "used by many
>     packages". (Hence vcruntime140.dll is included, despite not having
>     forwards-compatibility, and I tried real hard to avoid including
>     that one too.)
>
>
> yeah, a line has to be drawn, and "created by Microsoft" wouldn't make
> any sense either. but "standard runtime expected by the compiler" would
> -- but, as you say above, you have may have to ship a new version with
> your extension anyway -- so it seems adding the feature for distutils to
> do that for C++ extensions would both solve the problem and not ship
> anything extra with the core install.

It would be possible. It might be simpler to update the default compiler 
settings to statically link the C++ runtime while still dynamically 
linking the C runtime (and any project is already capable of doing this 
on their own - I'm 99% sure it works).

>     Finally, unless we demand users are administrators for every
>     install, we will quickly have many Python installs using versions of
>     msvcp140.dll (or any other dependency) with security
>     vulnerabilities, with no way to update them other than a full Python
>     release. Installing the regular runtime (which is patched
>     automatically) or as part of a package (which can be independently
>     updated) is far more feasible. I really don't want to be doing
>     security updates for 15 years just to keep an unused DLL secure.
>
>
> Though we have to do that for msvcp140.dll anyway, yes? god, what a
> pain! (and why MS should ship the damn things themselves!)

Maybe, but it's also the same reason why MS doesn't want to ship it 
themselves either. In fact, all the reasons why we don't want to include 
it in Python equally apply to Windows - until it's a direct dependency 
of the product itself, it has a really high bar to be included (SQLite 
is included in Windows because it is a dependency, not just as a 
convenience to developers).

> Anyway, proposal withdrawn.
>
> Replaced with possible proposal to have distutils include the C++
> runtime dll like it does now for the C one.

The big difference here is that msvcp140.dll contains significant 
functionality while vcruntime140.dll is a very small amount of shared 
state and initialization. Changes/additions/fixes to msvcp140.dll are 
far more likely as a result.

I don't recall whether pip extensions are a thing yet, but if they are 
this would be a great candidate for that (trigger the runtime installer 
if necessary). Otherwise, having a package that only includes this DLL 
would probably be fine too, though it really only defers the problem to 
the next DLL you depend on.

Cheers,
Steve