What's so funny? WAS Re: rotor replacement

[phr at localhost.localdomain]

Skip Montanaro <skip at pobox.com> writes:
>     phr> It is not a whizbang module.  It is a stripped-down, basic
>     phr> implementation of a well-accepted set of standards that are being
>     phr> used in thousands of other applications in other languages.
> 
> Then there should be a library already out there already.  All you
> should need to do is wrap it (manually, with SWIG, whatever).

I'm currently using something wrapped with SWIG, but my understanding
is that core modules are not supposed to depend on SWIG.  So a core
module will likely use some existing primitives wrapped by hand.  That
is what I've offered to do, along with providing a somewhat more
Pythonic interface (though still a straightforward one) than directly
wrapping a C library intended for use from C applications.

>     phr> There is demand for it.  Look at how this thread started: some
>     phr> crypto user noticed that rotor was gone and wanted to know what to
>     phr> do instead.
> 
> Yes, and putting rotor back would be the wrong thing to do.

Correct.  The right thing is to replace rotor with something
reasonable that follows standards.

>     phr> The issue of whether there's enough desire for a crypto module to
>     phr> warrant including one in the stdlib was decided a long time ago.
>     phr> The proof of that somebody decided to accept the rotor module into
>     phr> the distro.
> 
> No, rotor was added in Python's early days (July 1992).  Times have changed.

I don't see that.  There's surely even more demand for crypto now than
there was in 1992.

> As long as we are discussing cryptography, what's wrong with m2crypto?
> 
>     http://sandbox.rulemaker.net/ngps/m2/ 

It's a good package but it's pretty heavyweight.  It depends on both
SWIG and OpenSSL.  I think it's still under development--there's an
occasional flurry of messages about it on python-crypto, but I haven't
been following it closely.  I'd have a hard time making a case for
accepting it into the core given the difficulty I'm having making the
case for something as simple as a block cipher wrapper.
m2crypto+OpenSSL is at least 100 times as much code as the module I've
proposed.  I think the Python lib should someday have its own
pure-Python SSL/TLS implementation sort of like the one Java has.  But
if m2crypto went into the lib, I'd use it.

> Why not incorporate it into the standard distribution?

I don't have the authority to incorporate anything into the standard
distribution.  All I can do is offer to contribute stuff that I write,
and let the distro maintainers decide whether to incorporate it.

I don't have the authority to offer m2crypto, since I'm not the author.
Only the authors can do that.  They haven't done so, as far as I know.

> Or, what about Andrew Kuchling's crypto toolkit?
> 
>     http://www.amk.ca/python/code/crypto.html

This is perhaps more suitable than m2crypto but as far as I know,
Andrew hasn't offered to contribute it.  Whatever his reasons are, I
have to respect them.  I think it has more stuff than a core crypto
module really needs (e.g. numerous semi-obsolete algorithms that
aren't in widespread use so aren't needed for interoperability) but
the extra stuff doesn't really get in the way.  If it were in the
core, it would completely fulfill my desires and I would be
transported with ecstacy.  But I've never seen any indication that
it's headed there.

> I believe both have been around awhile.  If crypto-in-the-core is
> really what's needed why not see if one of them is ready to go?

I don't think m2crypto is the answer, though maybe I'm wrong.  And if
Andrew's package is the answer, he would have submitted it already.

>     phr> Have you ever used a crypto library in a serious way?  
> 
> Nope, never directly.  Don't make this about me.  I'm interested in the
> Python development process and how you'd like to turn it on its head.

Either you're the one turning the actual process on its head, or else
it's already on its head and needs to be turned rightside up.  Either
way, the existing process has certainly been a total failure so far at
producing good crypto support in the lib.

>     phr> It's already the category king, because there are precisely zero
>     phr> other entrants in the category.  
> 
> See my above references.  Note, I don't use crypto at all, yet I was aware
> of both of these (no Googling required).

The authors have not offered to contribute them, so they're not in the
category.  The category consists of crypto modules that have actually
been offered.  As I keep saying, I'd love it if someone else offered
one.  I'm not eager for this headache.  I just saw that somebody
really ought to do it, and nobody was doing it, so I decided I was
elected.

> My guess would be they are substantially more mature than your
> proposed module.

This may sound a little strange, since my module is not yet written,
but if we take maturity to mean lower future maintenance needs, then I
wouldn't say m2crypto is precisely more mature.  m2crypto uses SWIG
and is intimately connected to an immensely complex package (OpenSSL)
anpd has to track new OpenSSL and SWIG releases.  OpenSSL has all
kinds of OS dependencies, has high performance math subsystems written
in assembly code for various different cpu's, etc.  So my guess is
that m2crypto has a longer development curve and will always need
significant amounts of maintenance.  My module is more like a basic
portable math library that computes square roots, cosines, etc. and is
OS and CPU independent.  Once written and tested, it's intended to be
pretty solid and not need much attention.

Andrew's package is along the same lines as mine, except Andrew's is
already done, so Andrew's is definitely more mature.  I like to think
that my module's API incorporates some lessons from Andrew's and
improves on it, but that's not that important in practice.

>     phr> I read those as saying that no crypto module will be
>     phr> considered for inclusion whether or not it's the category
>     phr> king, because any such module might conflict with crypto
>     phr> regulations in some countries.
> 
> That may be a problem, sure.  I'm not sure how the discussion here changes
> that.  That's just life as we know it.

It doesn't change it.  However, it does change my level of interest in
writing a module, as is reasonable since the only important
characteristics of my module that are different from existing ones is
that mine is designed specifically to be suitable for the core and has
been offered for distribution in the core.  Outside the core, the
existing modules work ok so I use them.

> Why in the heck is inclusion in the standard library a requirement
> for you to write this thing?

There's no other reason to write it.  Outside the stdlib, there are
other modules that are already useable, like the ones you mentioned,
so I use those.

> If it's useful to you, write it and get on with your life.

It's not worth doing unless it's intended for the stdlib.  How many
times do I have to explain this?

>     phr> Tell me again how rotor got into the distribution.
> 
> Python's user community was probably a few hundred people.  Guido
> likely had no thoughts of world domination with the little language that
> could.  Times have changed.

If Python is aiming for world domination, it needs to be able to
compete with Java in security features.  Java ships with the whole
JSSE system and a much fancier low level crypto architecture than the
one we've been discussing.  So I think the Python folks should figure
out how to incorporate crypto without getting crushed under the iron
boot (or at least the wooden shoe) of the Dutch government or whatever
it is that they're worried about.

> You seem think there was a PEP process and distutils and Wikis.  I
> suspect some of the algorithms one might include in a robust crypto
> toolkit today weren't even invented in 1992.

AES wasn't invented then.  DES became a US federal standard around 1977.

> So throw away the rotor crutch and put your money where your mouth is.

If you mean write yet another crypto module knowing ahead of time that
it won't go into the stdlib, that would be more like throwing my money
down the drain.