[Cryptography-dev] "Streaming" APIs

[Donald Stufft]

So one thing that's really handy to do with encryption is to be able to (de|en)crypt things without needing to load the entire thing into memory. Currently we support this in the encrypt API by doing:

cipher = BlockCipher()

enciphered = cipher.encrypt(block1)
enciphered += cipher.encrypt(block2)
enciphered += cipher.encrypt(block3)
enciphered += cipher.finalize()

We needed to do this because we need to be able to call finalize() before the encryption is "done". 

When I was messing with padding I ended up with an API that (for padding) got around the need for an explicit finalize step but instead it required passing the entire data stream into the function. However it supports generators/iterators so you can still efficiently process large datasets.

This api looks something like

padder = Padding()

padded1 = "".join(padder.pad("1234"))
padded2 = "".join(padder.pad(c for c in "1234"))

However the downside of this API is that You need to call "".join() to get actual strings or you need to do some ugly hacks inside of the pad() function so it returns a string if given a string and returns a generator if given a generator.

A third option is similar to dictionaries on Python 2.x where you have something like iterpad() and pad(). This could work for encryption as well so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt().

So I guess the question is how do we want to handle these streaming sorts of APIs?

1) Thing.action() + Thing.finalize()
2a) "".join(Thing.action(iterator))
2b) Thing.action(terator OR string) - Magic return types
3) Thing.action and Thing.iter_action
4) ????

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20130911/43e8cdef/attachment.sig>