Need help calling a proprietary C DLL from Python
Craig
craigm3604 at gmail.com
Mon Mar 24 18:21:11 EDT 2008
On Mar 24, 3:45 pm, Craig <craigm3... at gmail.com> wrote:
> On Mar 24, 12:27 pm, Craig <craigm3... at gmail.com> wrote:
>
>
>
> > On Mar 23, 7:59 pm, Dennis Lee Bieber <wlfr... at ix.netcom.com> wrote:
>
> > > On Sun, 23 Mar 2008 14:24:52 -0700 (PDT), Craig <craigm3... at gmail.com>
> > > declaimed the following in comp.lang.python:
>
> > > > This dll was designed to be used from either C or Visual Basic 6.
>
> > > > I have the declare statements for VB6, if that helps.
>
> > > Probably not that much -- I'd bet it's full of variant records <G>
>
> > > > Based on the results I have so far (and I have tried MANY permutations
> > > > like trying to use the LPSTR for SecKey and PriKey which are returned
> > > > as is TypeDef), it looks like SecKey and PriKey are being used as data
> > > > instead of pointers.
>
> > > > For this, I try:
> > > > LPSTR = c_char_p
> > > > VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPBSTR,
> > > > LPSTR]
> > > > SrchKey =
> > > > windll.oleaut32.SysAllocStringByteLen("MSD19DN
> > > > \x00", 41)
> > > > SecKey = create_string_buffer(41)
> > > > SecKey.raw = "1234567890123456789012345678901234567890"
> > > > PriKey =
> > > > windll.oleaut32.SysAllocStringByteLen("ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234\x00",
> > > > 41)
> > > > TypeDef = create_string_buffer(128)
> > > > TypeDef.raw = "X".center(128, "X")
> > > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > > byref(c_void_p(SrchKey)), SecKey, byref(c_void_p(PriKey)), TypeDef )
> > > > and I get:
> > > > Traceback (most recent call last):
> > > > File "C:\temp\vbisam_test_2.py", line 158, in <module>
> > > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > > byref(c_void_p(S
> > > > rchKey)), SecKey, byref(c_void_p(PriKey)), TypeDef )
> > > > WindowsError: exception: access violation reading 0x3433322D
>
> > > > I notice that SecKey.raw starts with "1234" and the exception address
> > > > is 0x3433322D, which is "432-".
>
> > > 0x2D is 4 less than the expected 0x31...
>
> > > > And, changing to:
> > > > VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPBSTR, LPSTR,
> > > > LPSTR]
> > > > PriKey = create_string_buffer(41)
> > > > PriKey.raw = "ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234"
> > > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > > byref(c_void_p(SrchKey)), byref(c_void_p(SecKey)), PriKey, TypeDef )
> > > > I then get:
> > > > Traceback (most recent call last):
> > > > File "C:\temp\vbisam_test_2.py", line 159, in <module>
> > > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > > byref(c_void_p(S
> > > > rchKey)), byref(c_void_p(SecKey)), PriKey, TypeDef )
> > > > WindowsError: exception: access violation reading 0x4443423D
>
> > > > I notice that PriKey.raw starts with "ABCD" and the exception address
> > > > is 0x4443423D, which is "DBC=".
>
> > > ... and 0x3D is 4 less than the expected 0x41
>
> > > Which leads me to suspect that BSTR are a structure, not a plain
> > > string, in which the address given is supposed to be prefaced with a
> > > length value. IOWs, something like:
>
> > > |----|--------------------------------------|
> > > ^ ^ C-string data
> > > | |Address to be passed
> > > |(address - 4) to get to a length count field
>
> > > Confirmed:http://msdn2.microsoft.com/en-us/library/ms221069(VS.85).aspx
>
> > > (the URL is from HTTP through to the end, including .aspx)
>
> > > Creating such may not be difficult -- but passing it to the DLL
> > > could be. I don't know if ctypes allows pointer arithmetic.
>
> > > ctypes OLESTR is a c_wchar_p, but that is still a c-style string with no
> > > length prefix.
>
> > > The only mention of BSTR in the win32 extensions is in text that
> > > implies that the win32 extension library takes care of converting
> > > from/to BSTR and Python strings transparently -- but that won't work for
> > > your third-party library I suspect.
>
> > > Might have to beg the author(s) of ctypes to add a BSTR type to the
> > > list of those supported... as I can find no means of tweaking the
> > > address computed by byref() to point somewhere into a structure rather
> > > than the beginning of the structure.
>
> > > >>> pk = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 12)
> > > >>> pk
> > > 1373844
> > > >>> id(pk)
> > > 18941724
> > > >>> ctypes.string_at(pk)
> > > '1234567890'
> > > >>> ctypes.string_at(pk-4, 20)
>
> > > '\x0c\x00\x00\x001234567890\x00\x01\x00\x00\x00\x00'
>
> > > Okay, the return value from SysAlloc... IS the integer representing
> > > the address of a BSTR structure (IE, the address four bytes in from the
> > > real memory start)... Note how backing up 4 bytes reveals the BSTR
> > > length field
>
> > > What happens if you just pass that item as-is, no byref(), no
> > > conversion to ctypes pointer types...
>
> > > PriKey = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 41)
> > > SecKey = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 41)
>
> > > res = VmxGet( byref(hwmcb),
> > > byref(SecIndex),
> > > byref(Option),
> > > byref(c_void_p(SrchKey)),
> > > SecKey,
> > > PriKey,
> > > TypeDef )
>
> > > >>> PriKey = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 41)
> > > >>> ctypes.string_at(PriKey, 41)
>
> > > "1234567890\x00\x01:\x00\x00\x00\xb6\x01\x00\x000\x99#\x01\xe0\x94\x1a\x1e\x0b\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00'">>> ctypes.string_at(PriKey-4, 41+4)
>
> > > ")\x00\x00\x001234567890\x00\x01:\x00\x00\x00\xb6\x01\x00\x000\x99#\x01\xe0\x94\x1a\x1e\x0b\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00'"
>
> > > >>> ord(")")
> > > 41
>
> > > You'll need to use the .string_at() or .wstring_at() functions to
> > > extract any data changed by the call. Should not be a Python
> > > immutability problem as all "you" allocated in Python is the integer
> > > holding the address. You may need to call a Sys... function to free the
> > > memory that had been allocated -- Python doesn't know about it.
> > > --
> > > Wulfraed Dennis Lee Bieber KD6MOG
> > > wlfr... at ix.netcom.com wulfr... at bestiaria.com
> > > HTTP://wlfraed.home.netcom.com/
> > > (Bestiaria Support Staff: web-a... at bestiaria.com)
> > > HTTP://www.bestiaria.com/
>
> > The VB6 Declare is:
> > Declare Function VmxGet Lib "vbis5032.DLL" (DatasetHandle&,
> > SecIndexField%, Options%, SelectorKey$, RSecondaryKey$, RPrimaryKey$,
> > RRecordVariable As Any) As Integer
>
> > The only variant is the RRecordVariable, which is actually meant to be
> > a Type structure.
>
> > Anyway, I thought maybe there was a clue in here that I am missing.
>
> > Back to Python. I ran with the following:
> > VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPSTR,
> > LPSTR]
> > SrchKey =
> > windll.oleaut32.SysAllocStringByteLen("MSD19DN
> > \x00", 41)
> > SecKey =
> > windll.oleaut32.SysAllocStringByteLen("1234567890123456789012345678901234567890\x00",
> > 41)
> > PriKey =
> > windll.oleaut32.SysAllocStringByteLen("ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234\x00",
> > 41)
> > TypeDef = create_string_buffer(128)
> > TypeDef.raw = "X".center(128, "X")
> > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > byref(c_void_p(SrchKey)), SecKey, PriKey, TypeDef )
> > And, now I get:
> > Traceback (most recent call last):
> > File "C:\temp\vbisam_test_2.py", line 156, in <module>
> > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > byref(c_void_p(S
> > rchKey)), SecKey, PriKey, TypeDef )
> > ctypes.ArgumentError: argument 5: <type 'exceptions.TypeError'>: wrong
> > type
>
> > So, now the problem is with:
> > VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPSTR,
> > LPSTR]
> > I am not sure what to put for arguments 5 and 6 now.
>
> > I realized the need to release the BSTRs myself, and already included:
> > windll.oleaut32.SysFreeString(SrchKey)
> > windll.oleaut32.SysFreeString(SecKey)
> > windll.oleaut32.SysFreeString(PriKey)
> > at the end of the program.
>
> > I have no problem using .string_at() to extract the returned values -
> > once the .argtypes let the call execute.
>
> > In case this helps, here is an excerpt from a C program that was
> > provided with the dll:
> > typedef struct tagMYREC
> > {
> > char name[51]; // 1 xref generated name key
>
> > char lastname[31]; // 2 last name
> > char firstname[21]; // 3 first name
> > char address[41]; // 4 address
> > char city[31]; // 5 xref city key
> > char state[3]; // 6 xref state
> > char zip[11]; // 7 xref zip code
> > char phone[21]; // 8 xref phone number
> > BSTR notes; // 9 notes
>
> > } MYREC;
> > typedef MYREC FAR * LPMYREC;
>
> > short rc; // return code (from VB/ISAM function calls)
> > static HANDLE nDatasetNumber;
> > static short cur_index;
> > static short GetOpt;
> > static BSTR Dummy;
> > static BSTR Throwaway;
> > static BSTR PrimaryKey;
> > static MYREC myrec;
>
> > rc = VmxGet(&nDatasetNumber, // int DatasetNumber // VIS_BUSY ?
> > &cur_index, // int SelectedIndex
> > &GetOpt, // int OptionParameter
> > &Dummy, // Don't need a Selector param with
> > these options
> > &Throwaway, // Don't need returned index entry in
> > this case
> > &PrimaryKey, // LPBSTR lpPriKey
> > (void *) &myrec); // LPVOID lpRecordStructure
>
> > For someone who knows C and Python, this may provide a clue to the
> > BSTR passing/return problem.
>
> I received two emails from Dennis, included below (for completeness):
> ++++++++++++++++++++++++ Start of email #1 ++++++++++++++++++++++++
> I'm back at work, hence the email.
>
> -=-=-=-=-=-=-
> So, now the problem is with:
> VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPSTR,
> LPSTR]
> I am not sure what to put for arguments 5 and 6 now.
>
> I realized the need to release the BSTRs...
>
> read more >>
I received an email from Dennis, which I have included below (for
completeness):
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Last email; since I'm at the end of my analytical skills
(lacking the library I can't perform tests of my own).
Since the value being passed should have been the address of
the BSTR structure (put a >> print "address = 0x%8.8X" % PriKey <<
just before the call), for it to claim it can't read 0x3433322D
implies another layer of indirection in the library... in that it must
have used the passed address to pick up the first four bytes of the
string portion of the BSTR, and is then trying to treat /that/ as the
address to what it expects to really be the BSTR (subtracting four
from that to get to the expected length field).
My last try therefor...
Try changing the argtype to a pointer-to-long (or whatever
pointer types have been defined). Then pass >> byref(c_int32(PriKey))
<< (Have we gone full circle yet?)
My hope is that, based upon the above evaluation, this final
attempt will pass an address pointing TO the address stored in the
c_int32, and that the library will use /that/ address to get to the
BSTR itself.
I'd also suggest that, just for safety, you hold onto a
reference to the original allocations... Just in case the library really
intends to allocate internally and change the pointer value being
passed (you may need to encapsulate it outside of the library call:
ci_PriKey = c_int32(PriKey)
do library call: ... , byref(ci_PriKey), ...
if ci_PriKey.value != PriKey:
# you were returned a new buffer
Don't forget to treat the other BSTR the same way...
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I would have gladly provided the dll in question if you had requested
it.
Anyway, I added the following, for completeness:
LPLONG = POINTER(c_long)
And, then:
VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPLONG, LPLONG,
LPSTR]
print "\nVmxGet test - looking for valid record..."
printf ("Before - PriKey = 0x%8.8X, SecKey = 0x%8.8X\n", PriKey,
SecKey)
printf (" ci_PriKey = 0x%8.8X, ci_SecKey = 0x%8.8X\n",
ci_PriKey.value, ci_SecKey.value)
res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
byref(c_void_p(SrchKey)), byref(ci_SecKey), byref(ci_PriKey),
TypeDef )
printf ("After - PriKey = 0x%8.8X, SecKey = 0x%8.8X\n", PriKey,
SecKey)
printf (" ci_PriKey = 0x%8.8X, ci_SecKey = 0x%8.8X\n",
ci_PriKey.value, ci_SecKey.value)
printf ("Record returned = \x22%s\x22\n", TypeDef.raw)
printf (" Key returned = \x22%s\x22\n", string_at(ci_PriKey.value,
41))
printf ("2ndKey returned = \x22%s\x22\n", string_at(ci_SecKey.value,
41))
And this is what I got:
VmxGet test - looking for valid record...
Before - PriKey = 0x0044F56C, SecKey = 0x0044F534
ci_PriKey = 0x0044F56C, ci_SecKey = 0x0044F534
After - PriKey = 0x0044F56C, SecKey = 0x0044F534
ci_PriKey = 0x0043D49C, ci_SecKey = 0x0043D484
Record returned = "MSD19DN Testing
Board Of E
ducation 4 "
Key returned = "MSD19DN"
2ndKey returned = "MSD19DN"
PERFECT!
Now, I just have to write some wrapper code and I should be able to
turn this set of calls into a VB/ISAM object, that can then be
instantiated for each file I need.
Dennis, thank you very much for all of your wonderful contributions.
More information about the Python-list
mailing list