Problem with msvcrt60 vs. msvcr71 vs. strdup/free

["Martin v. Löwis"]

abkhd at earth.co.jp wrote:
> Believe it or not I do not get a crash here.

I can believe this. strdup does malloc, i.e. gets some
memory from the heap, and free puts it back into the heap.
Since this is a different CRT, it puts it back into
a different heap. This is a leak, but it should not cause
a crash.

> I have a gut feeling that "DLLs" here does not mean inter-dll passing
> of resources, but intra-dll passing of resources. In other words that
> if the self-same DLL uses both run-times then you are in trouble; but I
> don't think that this relates to the situation where one DLL is using
> msvcrt and another DLL is using msvcr71. Please bear with me.

I think you are wrong. What DLL invokes the CRT function does not matter
- that the CRT functions come from different DLLs matters. Each CRT
is loaded exactly once into an address space, regardless of how many
DLLs refer to it. However, different CRTs have different sets of global
variables, and this is what hurts.

E.g. malloc use a pointer to the free list in the heap, and so does
free. Each CRT has its own heap, and each has its own pointer to the
free list. Then, doing malloc in one CRT and free in the other does
not cause crashes (perhaps unless you use the debug CRT, which tries
to determine whether it owns the pointer being freed). The CRT will
put the memory into its own free list, and allocate it again if its
own malloc asks for it. However, if you repeatedly do malloc() in one
CRT and free() in another, you get a memory leak, since malloc() will
not find the memory that has been freed.

> In this light, I truly don't know why would we try, or recommend
> others, to link with libmsvcr71.a, if some mix-up is still possible. I
> have yet to see a crash occur when Python24.dll or an extension of the
> same uses the old way of  linking with libmsvcrt.a. That way we are
> assured that no references are made to msvcr71 when msvcrt is the one
> used when linking.

This is very easy to create. Just do fopen() in an extension linked
with msvcrt.dll, then do PyRun_AnyFile. This will crash, because fread()
on a file opened in a different CRT will always crash.

> I could be mistaken, but I think people are confusing the behavior of
> the MS compilers with MinGW's. The old MS compiler (V. 6) uses
> msvcrt.lib by default, and even when the newer MSVS uses the same name
> for the its default runtime, that msvcrt link implicitly means the
> newer msvcr71 [2].

You are mistaken. The name of the import library is, and always was,
msvcrt.lib. However, different versions of msvcrt.lib refer to different
DLLs: msvcrt40.dll, msvcr70.dll, and msvcr71.dll.

 > This behind the scenes behavior is what seems to
> have people mixed up. You see if one has older projects compiled using
> V. 6, then linking these object files to other object files compiled
> with the newer MSVS could effectively mean that the resulting DLL will
> have references to both run-times, something by the way is also
> mentioned in the MSDN resources [3].

I think you are misreading [3]: It explicitly says

"If you have a .lib or .obj file that needs to link to msvcrt.lib, then
you should *not* have to recompile it to work with the new msvcrt.lib in
Visual C++ .NET." (emphasis mine)

The only case where you can link a single DLL with different CRTs using
MS tools is when you combine different versions of the CRTs, e.g. debug
and non-debug, or DLL and non-DLL.

> But as far as I know this is not the case with regards to MinGW. MinGW
> explicitly uses msvcrt, and so old and newer projects should in theory
> have no problems if they stick to msvcrt, as opposed to going the MS
> compilers' path-- shooting ourselves in the foot-- and risking linking
> to two run-times at the same time.

Python extensions should never ever link with msvcrt.dll. As your
reference says:

'The msvcrt.dll is now a "known DLL," meaning that it is a system 
component owned and built by Windows. It is intended for future use only 
by system-level components. An application should use and redistribute 
msvcr71.dll'

Now, a Python extension is released from the need to distribute
msvcr71.dll, since Python already does that. It should still link
with that DLL.

> I have tried changing the specs, and changing the location of msvcr71
> and the results are never 100% re-assuring. And so I have resorted to
> linking with the good old msvcrt and I have not had a crash due to this
> to date. Of course this is my humble experience and I could be mistaken
> big time.

You are.

> For what it is worth, however, my extension DLLs are clean in that they
> reference msvcrt only and not msvcr71, and hence all the malloc, and
> free and what have you are never shared with any other runtime. 

How do you know? What if your extension returns a pointer malloc'ed by
msvcrt.dll, and returns it to python24.dll, and then python24.dll
frees the pointer, using free() from msvcr71.dll?

> My
> point being that lacking empirical evidence to the contrary, I think
> this is how making extensions to Python 2.4 with MinGW should be: using
> the default runtime libmsvcrt.a. It would be fantastic if the good
> people at MinGW, or others, can get that hard-wiring issue of the
> msvcrt resolved. But until that happens I think it is safer to extend
> Python 2.4 the old way. It seems to work fine.

Indeed, it *will* work most of the time, as you rarely pass resources
across CRTs in a typical Python extension. However, it is very hard
to tell from the source code of the extension if such resource passing
could ever happen (e.g. what about setlocale(), atexit(), malloc(),
...), so if you need something better than "seems to work", you really
need to make sure that you understand all the issues, and have a build
process that resolves them all.

Regards,
Martin