[Python-Dev] SSL support in _socket

[M.-A. Lemburg]

Tim Peters wrote:
> 
> [MAL]
> > Side-note: I've added the "inter-module dynamic C API linking
> > via Python trick" from the mx tools to the _socket module. _ssl
> > only uses it to get at the type object, but the support can easily
> > be extended if this should be needed for more C APIs from
> > _socket.
> >
> > Also note: the non-Unix build process files need to be updated.
> 
> I don't know what "inter-module dynamic C API linking via Python trick"
> means, but the Windows build doesn't compile anymore despite that it didn't
> and doesn't support SSL.  I suspect it's because "inter-module" wrt sockets
> is really "cross-DLL" on Windows, and clever tricks are going to bite hard
> because of that. 

No it's not (and that's the main advantage of the "trick").

Some explanation:

The _ssl module needs access to the type object defined in 
the _socket module. Since cross-DLL linking introduces a lot of
problems on many platforms, the "trick" is to wrap the
C API of a module in a struct which then gets exported to
other modules via a PyCObject.

The code in socketmodule.c defines this struct (which currently
only contains the type object reference, but could very
well also include other C APIs needed by other modules)
and exports it as PyCObject via the module dictionary
under the name "CAPI".

Other modules can now include the socketmodule.h file
which defines the needed C APIs to import and set up
a static copy of this struct in the importing module.

After initialization, the importing module can then
access the C APIs from the _socket module by simply
referring to the static struct, e.g.

        /* Load _socket module and its C API; this sets up the global
           PySocketModule */
        if (PySocketModule_ImportModuleAndAPI())
                return;

        ...
        if (!PyArg_ParseTuple(args, "O!|zz:ssl",

                              PySocketModule.Sock_Type,

                              (PyObject*)&Sock,
                              &key_file, &cert_file))
                return NULL;

(Perhaps I should copy the above explanation into the source
 files ?!)

> It's griping here:
> 
> static PyTypeObject PySocketSock_Type = {
> C:\Code\python\Modules\socketmodule.c(1768) : error C2491:
>     'PySocketSock_Type' : definition of dllimport data not allowed
> 
> and here:
> 
>     &PySocketSock_Type,
> C:\Code\python\Modules\socketmodule.c(2650) : error C2099:
>     initializer is not a constant

Ah, you're right, the export of the type object is not
needed anymore since this is now done using the PyCObject.
Sorry, my bad.
 
> The changes to socketmodule.h pretty much baffle me.  Why is the body of the
> function PySocketModule_ImportModuleAndAPI included in the header file?  Why
> is the body of this function skipped unless PySocket_BUILDING_SOCKET is
> defined?  All in all, this appears to be an extremely confusing way to
> define a function named PySocketModule_ImportModuleAndAPI in the new _ssl.c
> alone.  So why isn't the function just defined in _ssl.c directly?  There
> appears no reason to put it in the header file, and it's confusing there.

The reason for putting the code in the header file is
to avoid duplication of code. The import API is needed by
all modules wishing to use the C API of the socket module.

Currently, only _ssl needs this, but I think it would be
a good strategy to extend this technique to other modules
as well (esp. the array module would be a good candidate).
 
> This shows signs of adapting a complicated framework to a situation too
> simple to require most of what the framework does.  If so, since there is no
> other use of this framework in Python, and the framework isn't documented in
> the Python codebase, the framework should be tossed, and something as simple
> as possible done instead.

I don't think it's overly complicated. It's been in use in
mxDateTime and various database modules including mxODBC 
for many years and I haven't received any complaints about
it in the last few years. It would be nice if we could
integrate better support for it into the Python core.
Then we wouldn't need the header file source code
definition anymore.

IMHO, it's a very useful way of doing cross-DLL "linking"
in a platform independent manner. Note that the whole idea
originated from a discussion I had with Jim Fulton some
years ago. As I understand, the PyCObject was invented for
just this purpose.
 
> I can't make more time to sort this out now.  It would help if the code were
> made more transparent (see last paragraph), so it consumed less time to
> figure out what it's intending to do.  In the meantime, the Windows build
> will remain broken.

As I read the checkins, you've remove the type object 
export. 

I am curious why the test_socket still fails on Windows
though. Both test_socket and test_socket_ssl work just fine on
Linux.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/