CFFI distribution questions

[Israel Brewster]

So I have a python module that I have written which uses CFFI to link against a C library I have compiled. Specifically, it is a Database driver for the 4th dimension database, using an open-source C library distributed by the 4D company. I have tested the module and C code on a couple of different platforms, but I have a few questions regarding where to go from here.

1) Currently, I am manually compiling the C library, and placing it in a subfolder of the module. So the top level of the module directory contains the python code and the __init__.py file, and then there is a sub-directory (lib4d_sql) containing the C code and compiled C library. I point CFFI to the library using a construct like this:

_CWD = os.path.dirname(os.path.realpath(__file__))
ffi.verifier(..., Library_dirs=["{}/lib4d_sql/].format(_CWD), ...)

Obviously I have left out a lot of code there. Is this sort of construct considered kosher? Or is there a better way to say "this directory relative to your location"? I can't just use relative paths, because that would be relative to the execution directory (I think).

2) What is the proper way to compile and distribute the C library with the python module? 

The examples I've found about distributing a CFFI module all assume you are are using some brain-dead-simple built-in C command where you don't have to worry about compiling or installing a library. I found some documentation related to building C and C++ extensions with distutils, and following that managed to get the library to compile, but I can't figure out what it does with the compiled library, or how to get it into the proper location relative to my module. I also found some more "raw" distutil code here: https://coderwall.com/p/mjrepq/easy-static-shared-libraries-with-distutils that I managed to use by overriding the finalize_options function of the setup tools install class (using the cmdclass option of setup), and this allowed me to build the library in the proper location in the tmp install directory, but it doesn't seem to keep the library when installing the module in the final location. So far, the only way I have found to work around that is by including a "dummy" copy of the library in the package_data option to setup, such that when the library is built it replaces this dummy file. Is there a better/more canonical way to do this?

3) The majority of the setup/packing procedure I pulled from here: https://caremad.io/2014/11/distributing-a-cffi-project/ and it seems to work - mostly. The one problem I am running into is with the implicit compile that CFFI does. Some of the code (most, really) on that page is designed to work around this by doing the compile on install so it doesn't have to be done at runtime, however this doesn't appear to be working for me. I see signs that it is doing the compile at install time, however when I try to run the module it still tries to compile at that time, an operation that fails due to lack of permissions. Might anyone have some thoughts on how I can fix this?

Finally, let me ask you guys this: is distributing this as a CFFI module even the right approach? Or should I be looking at something else entirely? C code isn't needed for speed - the database itself will be way slower than the interface code. It's just that the driver is distributed as C code and I didn't want to have to reverse engineer their code and re-write it all. I've read over https://docs.python.org/2/extending/extending.html, but I think I'm missing something when it comes to actually interfacing with python, and dealing with the non-python types (pointers to a struct, for example) that the C library uses. Would I have to put 100% of my code, including things like the cursor and connection classes, in C code? When I call the connect function from python (which should return an instance of a connection class), what exactly should my C function return? All the examples on that page show basic C types, not pointers to class instances or the like. Maybe I just need to read over the documentation a few more times :-)

Thanks for any help anyone can provide on any of these questions! :-) Sorry for being so long-winded.


-----------------------------------------------
Israel Brewster
Systems Analyst II
Ravn Alaska
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
-----------------------------------------------




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20141125/d98ce72d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Israel Brewster.vcf
Type: text/directory
Size: 417 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20141125/d98ce72d/attachment.bin>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20141125/d98ce72d/attachment-0001.html>