[Cython] How to define C-consts in python module scope

[Lisandro Dalcin]

On 20 July 2011 15:32, mark florisson <markflorisson88 at gmail.com> wrote:
> On 20 July 2011 20:04, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>> On 20 July 2011 13:51, mark florisson <markflorisson88 at gmail.com> wrote:
>>> On 20 July 2011 18:06, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>>>> On 19 July 2011 20:48, Robert Bradshaw <robertwb at math.washington.edu> wrote:
>>>>> On Tue, Jul 19, 2011 at 3:02 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>>>>>> On 19 July 2011 02:24, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>>>>>>> 2011/7/18 Robert Bradshaw <robertwb at math.washington.edu>:
>>>>>>>> Trevor King and I discussed this quite a while back, but every time I
>>>>>>>> got around to looking at his code (I don't think he ever created a
>>>>>>>> formal pull request) something came up. The idea was that we could
>>>>>>>> support cpdef structs and extern functions as well.
>>>>>>>>
>>>>>>>
>>>>>>> That's interesting, I think I shouldn't hurry with my pull request.
>>>>>>>
>>>>>>> 2011/7/19 Robert Bradshaw <robertwb at math.washington.edu>:
>>>>>>>> On Mon, Jul 18, 2011 at 4:34 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>>>>>>>>> My suggestion is
>>>>>>>>>
>>>>>>>>>  cdef exposed enum:
>>>>>>>>>    ...
>>>>>>>>
>>>>>>>> I agree, public is an overloaded word. This meaning is analogous to
>>>>>>>> its use for cdef class members. Perhaps we should use "api" for api
>>>>>>>> generation, and public used for Python-level access, with cpdef being
>>>>>>>> the preferred form for declaring Python-accessible types.
>>>>>>>>
>>>>>>>
>>>>>>> It seems to me that cpdef is more cythonic but exposed could be used
>>>>>>> in this case:
>>>>>>>
>>>>>>> cdef extern from "ev.h":
>>>>>>>    exposed enum:
>>>>>>>        EV_READ
>>>>>>>        EV_WRITE
>>>>>>>
>>>>>>
>>>>>> And what about this?
>>>>>>
>>>>>> cdef extern from "ev.h":
>>>>>>   cpdef enum:
>>>>>>       EV_READ
>>>>>>       EV_WRITE
>>>>>
>>>>> Yep, exactly.
>>>>>
>>>>>> BTW, how is this supposed to work with *.pxd files? I think the values
>>>>>> will be exposed just in the matching implementation .pyx file, and not
>>>>>> with cimport, right?
>>>>>
>>>>> It would be an error unless there's an corresponding .pyx file (which
>>>>> could be empty). The idea is that one could also define extern cpdef
>>>>> functions and structs and wrappers would be provided.
>>>>>
>>>>
>>>> It would be an error? What do you mean? if you cpdef enum in foo.pxd,
>>>> and have foo.pyx, then the enumerations should be exposed in the 'foo'
>>>> module. But then if you "cimport foo" in bar.pyx, that should succeed
>>>> but the enumerations should not be exposed in the "bar" module... Am I
>>>> missing something?
>>>>
>>>
>>> I believe Robert confirmed what you said and added to that that it
>>> would be an error to have a cpdef extern enum in a .pxd without a
>>> corresponding .pyx file.
>>>
>>
>> But what an "error" means? Cython is going to fail compilation?
>> Suppose I develop a package, and at setup.py install time my package
>> installs somewhere a .pxd full of definitions in such a way that
>> third-party code can cimport it. Then, while writing the third party
>> code, you cimport the my package .pxd, and obviously there is no .pyx
>> for my package, as you are writing YOUR code. What's going to happen
>> in this case?
>
> If you want to use cpdef in your .pxd, it means you want to expose
> those to Python. This means they must be importable from Python, which
> means you need an extension module that exposes the constants, which
> means you need a .pyx. So you ship a (possibly empty) .pyx file along
> with your .pxd. If you don't want to expose them to Python but only to
> Cython, don't use cpdef. This compile time error would be issued
> whenever a user does a cimport of a .pxd file that has a cpdef enum
> with no corresponding .pyx file, i.e. the pxd is unusable, you
> wouldn't ship it in the first place.
>

So supose I want to write a small wrapper for dlopen(), then I do:

# dl.pxd
cdef from extern from "dlfcn.h":
    cpdef enum: RTLD_GLOBAL
    void * dlopen()(const char *filename, int flag);

# dl.pyx
def open(filename, mode):
    cdef void *handle = c_dlopen(fiename,handle)
    return <long>handle

Then the "dl" module namespace contains "dlopen" and "RTLD_GLOBAL"

Now, suppose I code my setup.py to build the ext module and install
dl.pxd as package data.

Now a user runs "pip install dl" and installs my package. She wants to
reuse my .pxd (installed somewhere) for calling dlopen() in its own
Cython code:

# user.pyx
from dl cimport RTLD_GLOBAL, c_dlopen
dlopen("somelibrary.so", RTLD_GLOBAL)


So the user code is going to fail? Then I cannot take advantage of
cpdef enum for developing my  "dl" module, I have to go back to
manually exposing the enumerations





-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169