[Python-Dev] Python 3.5 on VC14 - update

[Steve Dower]

For anyone who is interested in more details on the CRT changes, there's a blog post from my colleague who worked on most of them at http://blogs.msdn.com/b/vcblog/archive/2014/06/10/the-great-crt-refactoring.aspx

I wanted to call out one section and add some details:

    In order to unify these different CRTs [desktop, phone, etc], we have split the CRT into three pieces:
    1. VCRuntime (vcruntime140.dll): This DLL contains all of the runtime
       functionality required for things like process startup and exception handling,
       and functionality that is coupled to the compiler for one reason or another. We
       may need to make breaking changes to this library in the future.
    2. AppCRT (appcrt140.dll): This DLL contains all of the functionality that is
       usable on all platforms. This includes the heap, the math library, the stdio and
       locale libraries, most of the string manipulation functions, the time library,
       and a handful of other functions. We will maintain backwards compatibility for
       this part of the CRT.
    3. DesktopCRT (desktopcrt140.dll): This DLL contains all of the functionality
       that is usable only by desktop apps. Notably, this includes the functions for
       working with multibyte strings, the exec and spawn process management functions,
       and the direct-to-console I/O functions. We will maintain backwards
       compatibility for this part of the CRT.

The builds of Python I've already made are indeed linked against these three DLLs, though it happens transparently. Most of the APIs are from the AppCRT, which is a good sign as it will simplify portability to other Windows-based platforms (though the direct references to the Win32 API will arise again to complicate this).

Very few functions are imported from VCRuntime, which is the only part that *may* have breaking changes in the future (that's the current promise, and I'd expect it to be strengthened one way or the other by releas). Apart from the standard memcpy/strcpy type functions (which may be moved in later builds), these other imports are compiler helpers:

* void terminate(void) (currently exported as a decorated C++ function, but that's going to be fixed)
* __vcrt_TerminateProcess
* __vcrt_UnhandledException
* __vcrt_cleanup_type_info_names
* _except_handler4_common
* _local_unwind4

I've checked with our CRT dev and he says that these don't keep any state (and won't cause problems like we've seen in the past with FILE*), and are only there to deal with potential C++ exceptions - they are included at a point where it is impossible to tell whether C++ is involved, and so can't be removed.

My builds pass almost all of regrtest.py and the only issues are with Tcl/tk and OpenSSL, which need to update their compiler version detection. I've built them with changes, though as usual Tcl/tk is a real pain.

I ran a quick test with profile-guided optimization (PGO, pronounced "pogo"), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?)

Cheers,
Steve