[pyOpenSSL] Arbitrary extension to X.509 certificate

[exarkun at twistedmatrix.com]

On 05:22 pm, philip.kershaw at stfc.ac.uk wrote:
>Hi Jean-Paul,
>
>It may be possible to get some of this work done.  Can you provide some 
>pointers + any preferences how you would want the relevant OpenSSL 
>interfaces exposed through Python?

Hiya Phil,

I'd love to give you some hints on this one.  First though, I want to 
point out that your question is a bit more of a stumper than you might 
have expected.  Though I'm the pyOpenSSL maintainer, I'm far from an 
expert on all of the OpenSSL APIs.

What I *mostly* know is that the OpenSSL APIs are terrible, and 
generally each in their own unique way.  Coming up with an approach to 
wrap a new OpenSSL API in pyOpenSSL usually involves stumbling around 
the documentation for a while, hoping to come across a nice looking 
function, then giving up on that and wandering through the source for a 
while (sometimes you can even find the implementation of a function), 
then giving up on that and looking around for other open source 
applications that do roughly the same kind of thing you think you want 
to do and reading their source instead.  Between those three sources of 
information, it's sometimes possible to understand what APIs exist to 
accomplish your goal and how they are used.

With a tenuous grip on that material, it's a question of deciding how 
the behavior could be exposed to Python.  Sometimes this is obvious, as 
in the case of SSL_write.  Other times it's obvious but admitting the 
reality is difficult, as in the case of the PKey APIs where OpenSSL 
reference counting and CPython reference counting contend with each 
other, requiring careful multi-library reference counting tricks to 
avoid double frees and other memory corruption.

All of this may sound like a lot of work, and it is.  Unfortunately it's 
hard to come to any sound decisions about what the Python API should 
look like until a lot of the background work has been done.  pyOpenSSL 
is an extremely leaky abstraction: wherever it diverges very far from 
the behavior of OpenSSL, things get difficult, so my philosophy since 
taking over the project has been to avoid any divergence from OpenSSL 
except those that are unavoidable (and grandfathering in some 
divergences, such as the class-oriented API).

That said (and apologies for that bit of ranting, but I needed to get it 
out), here are some suggestions I can make with respect to improving the 
x509 extension support:

  1. Some extensions require a X509V3_CTX structure to supply additional 
configuration/parameters.  It may be necessary to represent this 
structure in Python in order to provide an API which can really create 
arbitrary extensions (or I could be wrong).

  2. X509V3_EXT_nconf seems like an important extension API which 
pyOpenSSL currently uses as part of the existing extension support, but 
does not exactly "expose".  Many of its features are hidden and 
unavailable from Python.  This might be an area in which to make 
improvements (or I could be wrong).

  3. The current extension API which operates in terms of strings is 
broken and hides some features of arbitrary extensions.  I forget 
exactly how, though.  It is something like "Extension data is treated as 
a string value, prohibiting the use of any extensions which have non- 
string data", but perhaps it's extension names that are the problem and 
not data, or perhaps it's only one of the accessor APIs where this 
mistake is made, I can't remember, and looking at the code now, I don't 
understand/remember the OpenSSL APIs well enough to be able to tell.

  4. Apparently only extensions with names recognized by OpenSSL are 
supported now.  It looks like you want to specify an extension by its 
OID, which I presume requires the use of a different API than pyOpenSSL 
is currently using (ie, perhaps you cannot do this with 
X509V3_EXT_nconf).  So the approach taken to implement extension get/set 
may require exploring an alternate API.

I realize this probably provides more questions than answers, but off 
the top of my head it's the best I can do.  Perhaps someone on the list 
more familiar with the OpenSSL extension APIs can help answer some of 
these.

Jean-Paul