[python-win32] Missing memory de/allocation in com server causes APPCRASH?

[Roger Upole]

I'm guessing you have a 32/64 bit problem, as most XP installs are 32-bit.
I'd take a closer look at hServer, since handles are pointer-sized (4 bytes on
32 bit and 8 bytes on x64).

    Roger

"Wedel, Jan" <jan.wedel at ettex.de> wrote in message 
news:47EE7F193EDC7E468E74BB0ACFB17929011692C5FE at EX10MBOX1B.hosting.inetserver.de...
Ok,

being pretty desperate, I tried a few ways of returning the data until I got it partially working.

1. I returned null pointers:
                return POINTER(OpcDa.tagOPCITEMRESULT)(), POINTER(HRESULT)()

Obviously, it did not return any sensible data, but it did not crash! The client actually receives the null pointer which is better 
that crashing without any information.

2. Using the above knowledge, I tried to refactor the existing code as follows:
- creating the arrays in advance:
    add_results = (OpcDa.tagOPCITEMRESULT*count)()
    errors = (HRESULT*count)()
- looping over all elements of the array and filling in the data:
   (...)
   add_results[index].hServer = server_item_handle
  (...)
   Errors[index] = HRESULT(S_OK)
- returning a new pointer to the arrays
   return POINTER(OpcDa.tagOPCITEMRESULT)(add_results), POINTER(HRESULT)(errors)
- Do not store the arrays in python's "self"

Using this procedure, it does not crash the client and it does return data. Now, I am wondering why I had to do this on Win Server 
2008 and on Xp the old code worked.

However, this led me to next problem:
hServer is of type c_ulong which means 4 bytes and is the first element of the tagOPCITEMRESULT structure.  Reproducibly, the first 
two bytes of the first element of the array gets corrupted *somewhere* in the same way, more specifically it gets overwritten to 
0x4A00.  The value 0x4A doesn't ring any bells that could reveal the cause for that. All following elements of the array are not 
corrupted.

The question is, how could that happen?
What could possible overwrite memory?
Do I have to prevent freeing memory used by these arrays somehow before returning them via COM?

Thanks,

//Jan

Von: python-win32 [mailto:python-win32-bounces+jan.wedel=ettex.de at python.org] Im Auftrag von Wedel, Jan
Gesendet: Mittwoch, 11. Dezember 2013 10:38
An: python-win32 at python.org
Betreff: [python-win32] Missing memory de/allocation in com server causes APPCRASH?

Hi,

I already posted a related question on the comtypes mailing list but unfortunately Thomas Heller told me that he cannot actively 
provide support anymore.
So I'm hoping to get some help here because it might also relate more to ctypes.

At first, sorry for the long mail but it's a rather complex issue and I want to provide as much information and context as required.

Heres the problem:

We wrote a COM server using comtypes which worked well on Win XP. However, on Windows 2008 and Win 7, the same code crashes after 
calling a specific method that our server provides.
By "crashes" I mean the COM client (Tried it with a C++ and python client) receives a COMError:
_ctypes.COMError: (-2147417851, 'Ausnahmefehler des Servers.', (None, None, None, 0, None))

The Server does *not* throw any exceptions  as far as I can see and keeps on running. I added some printf debugging in comtypes but 
everything seems fine and ends up calling some win32api functions after message dispatching is done.

Every time this happens, in the windows application event log, there is an entry saying "python.exe" was the offending application, 
"ntdll.dll" was the offending module. I also set up windows debugging modules with shows the following stack trace (just first few 
lines at the top):

ntdll!RtlpLowFragHeapFree+0x31 (FPO: [Non-Fpo])
01 0021ed04 76f49c46 00390000 00000000 029e76e0 ntdll!RtlFreeHeap+0x101
02 0021ed18 7718afbd 00390000 00000000 029e76e8 kernel32!HeapFree+0x14
03 0021ed2c 7718afd9 7725f6f8 029e76e8 0021ed54
ole32!CRetailMalloc_Free+0x1c (FPO: [Non-Fpo])
04 0021ed3c 75be6efc 029e76e8 01f685fe 0021edcc ole32!CoTaskMemFree+0x13

Since the problem arises reproducibly when calling one method the has a pointer to an array as input and output, I believe this is 
some kind of memory issue (which also the stack trace suggests) that was probably just not recognized/caught by WinXP before but now 
shows up in more recent windows versions.
My assumption is, that either I need to actively allocate or deallocate memory for in/out parameters.

This is the method signature of the method that our server provides and that crashes:
IOPCItemMgt::ValidateItems
HRESULT ValidateItems(
[in] DWORD dwCount,
[in, size_is(dwCount)] OPCITEMDEF * pItemArray,
[in] BOOL bBlobUpdate,
[out, size_is(,dwCount)] OPCITEMRESULT ** ppValidationResults,
[out, size_is(,dwCount)] HRESULT ** ppErrors
);

Assume that this is the signature in python comtypes server:
      def ValidateItems(self, count, p_item_array, update_blob):

and this is the ctypes structure which should be returned in " ppValidationResults ":
      OpcDa.tagOPCITEMRESULT

How would I create these structures (especially ppValidationResults), its contents and the pointers so that I would not create any 
memory issues, null pointers or whatsoever?

My implementation creates a few items by calling
                validation_result = OpcDa.tagOPCITEMRESULT()
filling in the contents with e.g.
                validation_result.hServer = server_item_handle
adding it to a list
                validation_results.append(validation_result)
then converting it to a pointer to an array with a helper function:
      def p_list(items):
                c_items = (type(items[0])*len(items))(*items)
                p_items = cast(c_items, POINTER(type(items[0])))

                return p_items

      (...)

      p_validation_results = p_list(validation_results)

doing the same with "errors" and then returning it:
     return p_validation_results, p_errors

Is that correct?
Do I have to keep references to the list, the pointer or the item strcuture objects to avoid them beeing garbage collected?

Thanks a lot!





--------------------------------------------------------------------------------


> _______________________________________________
> python-win32 mailing list
> python-win32 at python.org
> https://mail.python.org/mailman/listinfo/python-win32
>