Passing a memory address (pointer) to an extension?

[Philip Semanchuk]

On Oct 22, 2008, at 8:33 PM, Robert Kern wrote:

>
> Philip Semanchuk wrote:
>> I'm writing a Python extension in C that wraps a function which  
>> takes a void * as a parameter. (The function is shmat() which  
>> attaches a chunk of shared memory to the process at the address  
>> supplied by the caller.) I would like to expose this function to  
>> Python, but I don't know how to define the interface.
>> Specifically, when calling PyArg_ParseTuple(), what letter should I  
>> use to represent the pointer in the format string? The best idea I  
>> can come up with is to use a long and then cast it to a void *, but  
>> assuming that a long is big enough to store a void * is a shaky  
>> assumption. I could use a long long (technically still risky, but  
>> practically probably OK) but I'm not sure how widespread long longs  
>> are.
>
> I recommend not giving the user access to that argument. Just use  
> NULL and let shmat() pick a starting address. I don't think it's  
> really safe to let the user pick, even in C. Perhaps if you're doing  
> *really* low level stuff. Of course, being that low level seems to  
> be the point of posix_ipc, so maybe I should let you get on with it.

I agree. To deny users of this module access to this param of shmat()  
would be design arrogance on my part. To do so would be to claim that  
I know better than everyone who might want to use my module. Using  
values other than NULL might be unwise (the man page I'm looking at  
states that clearly) but isn't one of the core design philosophies of  
Python "we're all consenting adults"?

> Anyways, the format "n" in Python >= 2.5 will correspond to a  
> Py_ssize_t integer, which will always be the size of a pointer.

Thanks, I had looked at that but rejected it for two reasons. First,  
it requires Python 2.5. (But I could make that a minimum requirement  
if necessary.) Second, PEP 353 says this:

"Conceptually, Py_intptr_t and Py_ssize_t are different things:  
Py_intptr_t needs to be the same size as void*, and Py_ssize_t the  
same size as size_t. These could differ, e.g. on machines where  
pointers have segment and offset. On current flat-address space  
machines, there is no difference, so for all practical purposes,  
Py_intptr_t would have worked as well."

I guess what they're saying is that presently there's no practical  
difference between the size of a Py_intptr_t (which seems to be what I  
need) and a Py_ssize_t. I can go with that, but to me it still smacks  
of the cowboy coding I did on 16-bit platforms that assumed that longs  
and pointers were the same size and happily cast between them. It  
works *now*, but...

Thanks also to Gabriel Genellina who wrote much the same.


Thomas Heller's suggestion of using PyLong_AsVoidPtr() is perfect; I  
was not aware of that function before.


> You can return the void* that you get from shmat() with a PyCPointer  
> object or make a new, small type that encapsulates a pointer  
> attached via shmat(). The benefit of a real type is that you can  
> type-check the input to shmdt() for safety. I strongly recommend  
> either of those approaches over returning an integer.

Fortunately shmdt() will be wrapped inside detach() which is a method  
object so I can use the pointer stored inside my internal C data  
structure. But you're right, I will need to expose the attached  
address to the caller as an attribute if nothing else. That's been a  
feature request for this module.

bye
Philip