Need help calling a proprietary C DLL from Python
Craig
craigm3604 at gmail.com
Mon Mar 24 15:45:25 EDT 2008
On Mar 24, 12:27 pm, Craig <craigm3... at gmail.com> wrote:
> On Mar 23, 7:59 pm, Dennis Lee Bieber <wlfr... at ix.netcom.com> wrote:
>
>
>
> > On Sun, 23 Mar 2008 14:24:52 -0700 (PDT), Craig <craigm3... at gmail.com>
> > declaimed the following in comp.lang.python:
>
> > > This dll was designed to be used from either C or Visual Basic 6.
>
> > > I have the declare statements for VB6, if that helps.
>
> > Probably not that much -- I'd bet it's full of variant records <G>
>
> > > Based on the results I have so far (and I have tried MANY permutations
> > > like trying to use the LPSTR for SecKey and PriKey which are returned
> > > as is TypeDef), it looks like SecKey and PriKey are being used as data
> > > instead of pointers.
>
> > > For this, I try:
> > > LPSTR = c_char_p
> > > VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPBSTR,
> > > LPSTR]
> > > SrchKey =
> > > windll.oleaut32.SysAllocStringByteLen("MSD19DN
> > > \x00", 41)
> > > SecKey = create_string_buffer(41)
> > > SecKey.raw = "1234567890123456789012345678901234567890"
> > > PriKey =
> > > windll.oleaut32.SysAllocStringByteLen("ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234\x00",
> > > 41)
> > > TypeDef = create_string_buffer(128)
> > > TypeDef.raw = "X".center(128, "X")
> > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > byref(c_void_p(SrchKey)), SecKey, byref(c_void_p(PriKey)), TypeDef )
> > > and I get:
> > > Traceback (most recent call last):
> > > File "C:\temp\vbisam_test_2.py", line 158, in <module>
> > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > byref(c_void_p(S
> > > rchKey)), SecKey, byref(c_void_p(PriKey)), TypeDef )
> > > WindowsError: exception: access violation reading 0x3433322D
>
> > > I notice that SecKey.raw starts with "1234" and the exception address
> > > is 0x3433322D, which is "432-".
>
> > 0x2D is 4 less than the expected 0x31...
>
> > > And, changing to:
> > > VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPBSTR, LPSTR,
> > > LPSTR]
> > > PriKey = create_string_buffer(41)
> > > PriKey.raw = "ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234"
> > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > byref(c_void_p(SrchKey)), byref(c_void_p(SecKey)), PriKey, TypeDef )
> > > I then get:
> > > Traceback (most recent call last):
> > > File "C:\temp\vbisam_test_2.py", line 159, in <module>
> > > res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> > > byref(c_void_p(S
> > > rchKey)), byref(c_void_p(SecKey)), PriKey, TypeDef )
> > > WindowsError: exception: access violation reading 0x4443423D
>
> > > I notice that PriKey.raw starts with "ABCD" and the exception address
> > > is 0x4443423D, which is "DBC=".
>
> > ... and 0x3D is 4 less than the expected 0x41
>
> > Which leads me to suspect that BSTR are a structure, not a plain
> > string, in which the address given is supposed to be prefaced with a
> > length value. IOWs, something like:
>
> > |----|--------------------------------------|
> > ^ ^ C-string data
> > | |Address to be passed
> > |(address - 4) to get to a length count field
>
> > Confirmed:http://msdn2.microsoft.com/en-us/library/ms221069(VS.85).aspx
>
> > (the URL is from HTTP through to the end, including .aspx)
>
> > Creating such may not be difficult -- but passing it to the DLL
> > could be. I don't know if ctypes allows pointer arithmetic.
>
> > ctypes OLESTR is a c_wchar_p, but that is still a c-style string with no
> > length prefix.
>
> > The only mention of BSTR in the win32 extensions is in text that
> > implies that the win32 extension library takes care of converting
> > from/to BSTR and Python strings transparently -- but that won't work for
> > your third-party library I suspect.
>
> > Might have to beg the author(s) of ctypes to add a BSTR type to the
> > list of those supported... as I can find no means of tweaking the
> > address computed by byref() to point somewhere into a structure rather
> > than the beginning of the structure.
>
> > >>> pk = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 12)
> > >>> pk
> > 1373844
> > >>> id(pk)
> > 18941724
> > >>> ctypes.string_at(pk)
> > '1234567890'
> > >>> ctypes.string_at(pk-4, 20)
>
> > '\x0c\x00\x00\x001234567890\x00\x01\x00\x00\x00\x00'
>
> > Okay, the return value from SysAlloc... IS the integer representing
> > the address of a BSTR structure (IE, the address four bytes in from the
> > real memory start)... Note how backing up 4 bytes reveals the BSTR
> > length field
>
> > What happens if you just pass that item as-is, no byref(), no
> > conversion to ctypes pointer types...
>
> > PriKey = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 41)
> > SecKey = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 41)
>
> > res = VmxGet( byref(hwmcb),
> > byref(SecIndex),
> > byref(Option),
> > byref(c_void_p(SrchKey)),
> > SecKey,
> > PriKey,
> > TypeDef )
>
> > >>> PriKey = ctypes.windll.oleaut32.SysAllocStringByteLen("1234567890", 41)
> > >>> ctypes.string_at(PriKey, 41)
>
> > "1234567890\x00\x01:\x00\x00\x00\xb6\x01\x00\x000\x99#\x01\xe0\x94\x1a\x1e\x0b\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00'">>> ctypes.string_at(PriKey-4, 41+4)
>
> > ")\x00\x00\x001234567890\x00\x01:\x00\x00\x00\xb6\x01\x00\x000\x99#\x01\xe0\x94\x1a\x1e\x0b\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00'"
>
> > >>> ord(")")
> > 41
>
> > You'll need to use the .string_at() or .wstring_at() functions to
> > extract any data changed by the call. Should not be a Python
> > immutability problem as all "you" allocated in Python is the integer
> > holding the address. You may need to call a Sys... function to free the
> > memory that had been allocated -- Python doesn't know about it.
> > --
> > Wulfraed Dennis Lee Bieber KD6MOG
> > wlfr... at ix.netcom.com wulfr... at bestiaria.com
> > HTTP://wlfraed.home.netcom.com/
> > (Bestiaria Support Staff: web-a... at bestiaria.com)
> > HTTP://www.bestiaria.com/
>
> The VB6 Declare is:
> Declare Function VmxGet Lib "vbis5032.DLL" (DatasetHandle&,
> SecIndexField%, Options%, SelectorKey$, RSecondaryKey$, RPrimaryKey$,
> RRecordVariable As Any) As Integer
>
> The only variant is the RRecordVariable, which is actually meant to be
> a Type structure.
>
> Anyway, I thought maybe there was a clue in here that I am missing.
>
> Back to Python. I ran with the following:
> VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPSTR,
> LPSTR]
> SrchKey =
> windll.oleaut32.SysAllocStringByteLen("MSD19DN
> \x00", 41)
> SecKey =
> windll.oleaut32.SysAllocStringByteLen("1234567890123456789012345678901234567890\x00",
> 41)
> PriKey =
> windll.oleaut32.SysAllocStringByteLen("ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234\x00",
> 41)
> TypeDef = create_string_buffer(128)
> TypeDef.raw = "X".center(128, "X")
> res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> byref(c_void_p(SrchKey)), SecKey, PriKey, TypeDef )
> And, now I get:
> Traceback (most recent call last):
> File "C:\temp\vbisam_test_2.py", line 156, in <module>
> res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
> byref(c_void_p(S
> rchKey)), SecKey, PriKey, TypeDef )
> ctypes.ArgumentError: argument 5: <type 'exceptions.TypeError'>: wrong
> type
>
> So, now the problem is with:
> VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPSTR,
> LPSTR]
> I am not sure what to put for arguments 5 and 6 now.
>
> I realized the need to release the BSTRs myself, and already included:
> windll.oleaut32.SysFreeString(SrchKey)
> windll.oleaut32.SysFreeString(SecKey)
> windll.oleaut32.SysFreeString(PriKey)
> at the end of the program.
>
> I have no problem using .string_at() to extract the returned values -
> once the .argtypes let the call execute.
>
> In case this helps, here is an excerpt from a C program that was
> provided with the dll:
> typedef struct tagMYREC
> {
> char name[51]; // 1 xref generated name key
>
> char lastname[31]; // 2 last name
> char firstname[21]; // 3 first name
> char address[41]; // 4 address
> char city[31]; // 5 xref city key
> char state[3]; // 6 xref state
> char zip[11]; // 7 xref zip code
> char phone[21]; // 8 xref phone number
> BSTR notes; // 9 notes
>
> } MYREC;
> typedef MYREC FAR * LPMYREC;
>
> short rc; // return code (from VB/ISAM function calls)
> static HANDLE nDatasetNumber;
> static short cur_index;
> static short GetOpt;
> static BSTR Dummy;
> static BSTR Throwaway;
> static BSTR PrimaryKey;
> static MYREC myrec;
>
> rc = VmxGet(&nDatasetNumber, // int DatasetNumber // VIS_BUSY ?
> &cur_index, // int SelectedIndex
> &GetOpt, // int OptionParameter
> &Dummy, // Don't need a Selector param with
> these options
> &Throwaway, // Don't need returned index entry in
> this case
> &PrimaryKey, // LPBSTR lpPriKey
> (void *) &myrec); // LPVOID lpRecordStructure
>
> For someone who knows C and Python, this may provide a clue to the
> BSTR passing/return problem.
I received two emails from Dennis, included below (for completeness):
++++++++++++++++++++++++ Start of email #1 ++++++++++++++++++++++++
I'm back at work, hence the email.
-=-=-=-=-=-=-
So, now the problem is with:
VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, LPSTR, LPSTR,
LPSTR]
I am not sure what to put for arguments 5 and 6 now.
I realized the need to release the BSTRs myself, and already included:
windll.oleaut32.SysFreeString(SrchKey)
windll.oleaut32.SysFreeString(SecKey)
windll.oleaut32.SysFreeString(PriKey)
at the end of the program.
I have no problem using .string_at() to extract the returned values -
once the .argtypes let the call execute.
-=-=-=-=-=-=-=-
Might I suggest plain old c_int (or c_int32)
After all, you don't want a pointer /to/ the integer value
that was returned by the SysAlloc...; the integer value itself /is/ the
pointer (to the BSTR memory) that needs to be passed, as is, to the
DLL function. c_int should do just that -- take the integer value
"out" of the Python integer object, and put it on the stack for the
DLL function.
I must state that I'm surprised some parts of Windows is using
a hybridized BCPL string. Last place I saw using those was on the
Amiga -- in which, while the Exec, Intuition, Graphics, and other low-
level libraries were in C, the command line user interface (utility
commands, environment, etc.) were ported from Tripos -- and Tripos had
been written in BCPL.
The hybrid comment is that Windows is combining the BCPL
length value with C null terminations, so the string can be used in
both ways.
+++++++++++++++++++++++++ End of email #1 +++++++++++++++++++++++++
++++++++++++++++++++++++ Start of email #2 ++++++++++++++++++++++++
-=-=-=-=-=-=-
TypeDef = create_string_buffer(128)
TypeDef.raw = "X".center(128, "X")
-=-=-=-=-=-=-
typedef struct tagMYREC
{
char name[51]; // 1 xref generated name key
char lastname[31]; // 2 last name
char firstname[21]; // 3 first name
char address[41]; // 4 address
char city[31]; // 5 xref city key
char state[3]; // 6 xref state
char zip[11]; // 7 xref zip code
char phone[21]; // 8 xref phone number
BSTR notes; // 9 notes
} MYREC;
-=-=-=-=-=-=-
Might I suggest increasing the length of your string buffer.
Adding up the allocation in MYREC comes to:
179 bytes BEFORE including "notes" -- and as a BSTR
declaration, notes is going to be a 4-byte length, 1 or 2 nulls (the
spec for a unicode result implied a 2-byte null) PLUS whatever value
length is itself. You may need to find out what the maximum length
allowed for the notes field is in advance and add that to the buffer
size. Pity its not a BSTR *notes -- that would imply memory allocated
elsewhere and an address -- the .string_at() could use the pointer to
retrieve the real notes.
+++++++++++++++++++++++++ End of email #2 +++++++++++++++++++++++++
Sorry for the confusion on "MYREC" - this was just used in the sample
C program provided with the dll. It is not applicable to the files I
am using. For my file, the 128 is correct.
I tried:
VmxGet.argtypes = [LPHANDLE, LPSHORT, LPSHORT, LPBSTR, c_int32,
c_int32, LPSTR]
SrchKey =
windll.oleaut32.SysAllocStringByteLen("MSD19DN
\x00", 41)
SecKey =
windll.oleaut32.SysAllocStringByteLen("1234567890123456789012345678901234567890\x00",
41)
PriKey =
windll.oleaut32.SysAllocStringByteLen("ABCDEFGHIJKLMNOPQRSTUVWXYZ12345678901234\x00",
41)
TypeDef = create_string_buffer(128)
TypeDef.raw = "X".center(128, "X")
res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
byref(c_void_p(SrchKey)), SecKey, PriKey, TypeDef )
And got:
Traceback (most recent call last):
File "C:\temp\vbisam_test_2.py", line 156, in <module>
res = VmxGet( byref(hwmcb), byref(SecIndex), byref(Option),
byref(c_void_p(SrchKey)), SecKey, PriKey, TypeDef )
WindowsError: exception: access violation reading 0x3433322D
That was what I was getting at earlier. We know that it is a c_int32
type, but the .argtypes is not doing something right (or I am not
specifying it correctly).
How do I correctly pass what needs to be passed?
More information about the Python-list
mailing list