What's so funny? WAS Re: rotor replacement

[phr at localhost.localdomain]

[Again I'm having news server trouble and made a previous attempt to
post this, so sorry if you see it twice.  This version is edited
somewhat from the previous.]

"Martin v. Löwis" <martin at v.loewis.de> writes:
> This is not possible - whether the module is included in Python or not.
> People *will* have to download something besides your application,
> namely Python - and in the version your require, too. If they cannot
> compile Python themselves, they will have to find somebody who did that
> for them.

Sorry, the presumption is that they already have Python installed.

> > So, if your modules are generally useful, I hope you
> > will try to get them into the core.
> 
> Right - the critical condition is "if the modules are generally
> useful". I cannot possibly know whether they are generally useful
> until a general public has commented that they are.

There is considerable demand for a crypto module.  Note how this thread
started: someone who had been using the rotor module complained that it's
now missing.

> Hmm. Most applications don't have any crypto needs. Those that do
> typically need TLS, or secure password hashing (crypt(3) specifically).
> So I somehow doubt that an AES module would be generally useful.

I believe it's feasible to do TLS in pure Python with acceptable
performance, given a DES module and the existing sha module.  I think
Trevor Perrin has already written something like that.  It's feasible
because outside of bulk crypto operations using DES and SHA, TLS's
execution time is dominated by public-key math calculations, which are
acceptably fast using Python's built-in longs.  The rest of the stuff
like certificate parsing happens just once per session and doing it in
Python isn't too awful.  On the other hand, it's impossible to do a
general purpose TLS stack acceptably without a DES module.  According
to RFC 2246 sec. 9, supporting TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA is
mandatory unless the application profile specifies otherwise.  SHA is
already in the core and DHE/DSS can be done with Python longs, so DES
is the missing critical piece.

Also, since DES is in the process of being decertified and AES is its
replacement, supporting DES without supporting AES is silly.  GnuPG
for example uses AES by default, so writing an interoperating Python
application (I have some parts of one working) needs AES.

If by crypt(3) you mean the old thing of DES with the modified E
table, I don't think it's used much any more.  I don't see any reason
against adding it if people want it, but it's not an encryption
function, it's an authentication function, and shouldn't be subject to
the legal concerns of a general purpose encryption module.

I do think it would be nice to have TLS in the core someday, just like
it's already in the Java distro (JSSE).  However, that's much more
ambitious and anyway it would rely on the block cipher module, so the
block cipher module is needed anyway.  Meanwhile, a pure-Python TLS
implementation outside the core is ok as a workaround.  Getting more
pure Python modules into the core is in general less valuable than
getting more C modules into the core.

> For an AES module with an interface nobody has ever used but yourself,
> it would be very difficult to argue that the module is generally useful.

Eh?  Lots of people have asked for general purpose crypto modules, and
the interface follows the approach used by any number of existing
libraries.  It just does exposes the simplest possible interface to
the block cipher and then implements the FIPS operating modes in a
natural and obvious way.  I've used a number of other such libraries
and am pretty familiar with what they need to do.  The main thing it
tries to do differently from PEP 272 is separate the block ciphers
from the FIPS modes, so that each new cipher module doesn't have to
implement all the different modes.

I hadn't thought there was any controversy over the technical side of
this.  There's the legal/political issue which I can gently suggest is
overblown, but since I'm not the one who would have to take the heat
if some government bigwig somewhere flipped their lid over crypto in
the Python distro, I don't feel entitled to be very vociferous about
this aspect.

> I'm primarily looking for the user testing. I require that this happens
> *outside* the Python distribution. Make a distutils package, and report
> the user feedback. Then propose inclusion into the core.

I'm happy to have that kind of testing (and I requested it), given
that the goal is inclusion in the core, and the core developers have
told me (as they did) that the proposal looks good and they'd like to
have the module, so I can reasonably expect it to go into the core if
it meets its technical expectations.

If the developers instead say (as they seemed to somewhat later) that
because of legal/political concerns, there's no way the module can
possibly go into the core no matter how good it is technically, then
my motivation for writing the module dries up quite a bit.  If someone
wants to pay me a salary to write it anyway, I might be interested,
but people who work as volunteers get to pick their own reasons for
doing things or not doing them.

> It very much depends on the details of the target system. Yes,
> it is a bit more tricky to install the additional library - but
> that is the case with any additional library, whether it is C or not.

I don't see this.  If a library is pure Python, I can always include
it with the application.  If it's in C, somebody has to compile and
install it, and I as the app writer may not have access to compilers
for the target platform.  Of course a Python library might have
platform-dependent OS calls or other configuration hair, but that
shouldn't apply to a pure mathematical function that's written
portably and makes no OS calls at all.

> For the CGI user who is not allowed to install binaries into
> /usr/local, she can still install the module in her home
> directory, no?

Evidently not always.  And how would the CGI user create a binary
anyway, even given a way to install it, if the web hosting service is
using a platform that the CGI user doesn't have a compiler for?  Think
of a Mac user whose web host runs Windows, or vice versa.

> See, this is the critical point: "commonly-used functions", not
> "functions I believe would be commonly used". You must have
> *existing* users for a function to be commonly-used.

You're going around in circles.  DES and AES have millions of users.
They have few Python users because the functions aren't available in
Python.  To fix that, they must be added to Python.  How many users
do you think the Python sha module had before it went into Python?

--Paul