Python loading library containing embedded python...

["Martin v. Löwis"]

Brennus wrote:
> I have solved my problem, or at least part of it by downgrading
> to Visual Studio 2003. For whatever reason, the same steps applied
> to Visual Studio 2005 result in massive problems.
> 
> Even the python.exe compiled by VS 2005 throws A/Vs on startup.
> 
> Does Python have to use Visual Studio?

Is this the debug build, by any chance? Microsoft broke support
for ISO C in VS2005. The Python trunk has a work-around for that
breakage.

> I have, of course, seen
> the MinGW Python patches and tutorials, but I am told I should
> not use anything but what Python was compiled with.

More likely, you managed to mix msvcrt versions somehow. You
need to inspect your resulting DLL in depends.exe; please report
what it depends on.

Also notice that you cannot use extension DLLs in your embedded
Python: all extension DLLs depend on python2x.dll, so importing
one of them leads to multiple copies of the Python code in your
interpreter.

> Are there plans to move away from VS? 

Well, VS is the the "platform compiler", so no. Of course, if
you wanted that to happen, you could write a PEP, and explain
what alternative compiler (plus alternative build procedure)
should be used.

> It's horrible imho. I only
> upgraded from VS 6 at the recommendation I should use the same
> compiler used to compile the latest Python. I asked why, citing my
> own knowledge of struct alignment concerns between compilers, but
> was vaguely told it was very bad, and that FILE structs can differ.

The problem really is with multiple copies of msvcrt. The struct
alignment in FILE structs is the same across all releases of
VC, yet having two copies of msvcrt in memory still might lead
to crashes.

The problem is that each msvcrt has its own FILE table, so
each one has its own stdout, for example. Now, for ages,
Microsoft also supports multi-threading in msvcrt, so there
was a need to extend struct FILE. They did so by introducing
struct FILEX, with an additional lock field. For
compatibility, they couldn't change the layout of the existing
array of FILE, so for the first 20 FILE values, the
lock is stored elsewhere (not in the __iob array, but in a
separate _locktable).

Now, when a file gets locked, msvcrt checks whether its pointer
is between __iob and __iob+20; if it is, it assumes it is
its own FILE, and looks for the lock in _locktable, else it
assumes it is FILEX, and looks for the lock in the struct itself.
If the FILE* really points into __iob of a different msvcrt,
that crashes.

Similar issues exist for malloc/free: you must not free a
pointer allocated by the malloc of a different msvcrt. Each
mscvrt has its own heap, and releasing memory to the wrong
heap might cause memory leaks.

Regards,
Martin