From phr-pycrypt at nightsong.com Wed Jun 12 01:36:47 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Tue, 11 Jun 2002 23:36:47 -0000 Subject: [PYTHON-CRYPTO] credit card encryption Message-ID: <20020611233647.11322.qmail@brouhaha.com> We want to do authorize/settle stuff asynchronously from the customer placing an order, so this is not an issue. It will be a batch process, but the question is whether that batch requires human intervention to succeed (e.g., the human types in the passphrase that unlocks the private key for one batch). In this case you can disconnect the payment box from the web server completely (e.g. transfer orders daily by floppy disc), or (less daily nuisance) connect it by a two-wire (i.e. unidirectional) serial cable so that the web server can send stuff to the payment box but the payment box can't send anything back to the web box (like card numbers, if it gets broken into). If you're running a retail site, though, sooner or later you'll want synchronous processing. If a credit card AVS fails because the customer entered a phone number different than the one on file with the card issuer (that happens all the time, people enter their work phone# or something) and you reject the payment right away, the customer will probably fix the problem and resubmit. But if they don't find out til the next day that the order failed, chances are they'll give up and you lose the order. I'll try to upload my encryption lib soon. Mainly that means I have to figure out again how to notify the stupid US export authorities that it's online. Sigh. From marklists at MCEAHERN.COM Wed Jun 12 02:19:56 2002 From: marklists at MCEAHERN.COM (Mark McEahern) Date: Tue, 11 Jun 2002 19:19:56 -0500 Subject: [PYTHON-CRYPTO] credit card encryption In-Reply-To: <20020611233647.11322.qmail@brouhaha.com> Message-ID: [Paul Rubin] > I'll try to upload my encryption lib soon. Mainly that means I have > to figure out again how to notify the stupid US export authorities > that it's online. Sigh. This might be of help: http://www.cdt.org/crypto/admin/ // m - From rsalz at DATAPOWER.COM Wed Jun 12 04:09:45 2002 From: rsalz at DATAPOWER.COM (Rich Salz) Date: Tue, 11 Jun 2002 22:09:45 -0400 Subject: [PYTHON-CRYPTO] credit card encryption In-Reply-To: <20020611215025.8253.qmail@brouhaha.com> Message-ID: > For example, the original PKCS1 has been replaced with an "improved" > PKCS1 which was replaced with OAEP which is now deprecated in favor of > RSA-KEM. Use a scheme that's already been vetted, even PKCS1 (which > has some theoretical weaknesses). I believe that RSA is advocating RSA-KEM, but they're pretty much alone. The IETF, for example, is unconvinced that there is any real reason to replace OAEP. > You're supposed to do this if you're processing a lot of transactions > or storing a lot of card numbers (like 100's of thousands). The primary reason security for using h/w crypto devices is that you can easily tell if it's been stolen, while you can't tell if someone's copied your private key stored on disk. Performance can also be a reason, of course. Your other advice is all quite good. /r$ From phr-pycrypt at nightsong.com Wed Jun 12 04:28:10 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Wed, 12 Jun 2002 02:28:10 -0000 Subject: [PYTHON-CRYPTO] credit card encryption Message-ID: <20020612022810.14822.qmail@brouhaha.com> [Rich Salz] I believe that RSA is advocating RSA-KEM, but they're pretty much alone. The IETF, for example, is unconvinced that there is any real reason to replace OAEP. When this has come up on sci.crypt, it's more like people don't see much point to OAEP, and RSA-KEM is the main alternative I've seen suggested. Last I checked (6 months ago?), RSAS was advocating OAEP, but that may have changed since then. > You're supposed to do this if you're processing a lot of transactions > or storing a lot of card numbers (like 100's of thousands). The primary reason security for using h/w crypto devices is that you can easily tell if it's been stolen, while you can't tell if someone's copied your private key stored on disk. It's not just a matter of detecting someone deliberately copying your secret key. Key material is like toxic waste and it's very difficult to keep it from leaking by accident in a hectic production environment where there are constantly staff changes, emergency software patches, sudden overnight bugouts of whole remote installations when the hosting ISP goes bankrupt and shuts down with no notice (that actually happened where I used to work and we're talking about truckloads of equipment that had to be got out and brought to another site), backup tapes with your key files on them that are supposed to go to offsite storage but go to surplus dealers instead, etc. Using hardware crypto modules is contractually required in order to connect to some payment networks and we were told we should be using them by the time we had 100,000 cards online. The modules eliminate a lot of variables, which is a good thing. I'm a firm believer in crypto hardware and I sleep much easier when I don't have any important private keys on general purpose computers. The library I mentioned earlier tries to provide HSM-like functionality as sort of a low-rent substitute for real HSM's. Performance can also be a reason, of course. These days crypto hardware is generally slower than workstation software except for maybe some specialized applications. That's especially true of lower cost devices, like the Java iButton which can do about one RSA decryption per second (vs 100's per second for a reasonably fast PC these days). From phr-pycrypt at nightsong.com Tue Jun 11 23:50:25 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Tue, 11 Jun 2002 21:50:25 -0000 Subject: [PYTHON-CRYPTO] credit card encryption Message-ID: <20020611215025.8253.qmail@brouhaha.com> Both offer an implementation of RSA. In the preliminary searching I've done on RSA, there seems to be an issue related to padding. The M2Crypto encrypt/decrypt methods have an option to specify a type of padding (e.g., pkcs1_padding). amk's Crypto offers no such options. Do I have to worry about padding? Yes, you have to worry about padding. pkcs1 is sort of disfavored these days but it should be ok for what you're doing. Simplistically, I would probably add a fixed amount of random information to the credit card number before encrypting it. Is that sufficient? No, don't do that. RSA padding is a complicated subject and there's been a flurry of academic papers about it in the past year or two. For example, the original PKCS1 has been replaced with an "improved" PKCS1 which was replaced with OAEP which is now deprecated in favor of RSA-KEM. Use a scheme that's already been vetted, even PKCS1 (which has some theoretical weaknesses). The thing I'm most leery about is key management. M2Crypto provides a way to protect the private key with a passphrase. amk's Crypto suggests using pickle to save the key(s). I suppose I could use SHA to generate a key from a passphrase, and then use that key to symmetrically encrypt the private key data before I save it to a file. Of course, how does the batch process for sending payment information to the payment processor get access to the private key? I haven't resolved that issue. Worst case scenario is the batch has to wait until someone types in the passphrase to unlock the private key. It sounds to me like you're dealing with two different problems: 1) protecting the secret key on the web server--that's easy, don't have a secret key there. That's presumably why you're using public key encryption. 2) Protecting the key on the separate server that talks to the payment processor. Is that done over the internet? Is it a manual process? The simplest approach is keep that server disconnected from the internet except when processing payments (I'm imagining a daily upload). Unfortunately that means you can't do online card verification when the customer places the order (it's nice to tell them right away that their card was declined, because they entered the wrong zip code or something like that). If you have to keep the payment server online, firewall it so that it can't talk to anything except the card processor and the web server. Better yet, don't connect it to the web server by TCP/IP. Connect it by serial port instead, to further lower the chance of anyone rooting the payment server by breaking through some TCP service. The really hardcore way protect online secret keys is by generating and storing them in a crypto hardware module. See for example www.ibutton.com (look at the java crypto ibuttons) for examples. You're supposed to do this if you're processing a lot of transactions or storing a lot of card numbers (like 100's of thousands). I don't mean to be looking for opportunities to plug my own code but I also have a library documented at http://www.nightsong.com/phr/python/crypto.txt which is written in pure Python. It currently doesn't support RSA, but supports a public key scheme based on Diffie-Hellman key exchange whose padding issues are hopefully simpler. Its claim to fame is running in a separate process from your application, so you can (for example) start the cryptography ("server") process running and enter a passphrase to unlock the secret key, then leave the process running so the application can use the key in memory, but the key is never written to disc. You can also put the crypto server on a separate computer and store your keys on the separate computer. My intention is to set up a Sharp Zaurus (Linux PDA) as a crypto coprocesor using this library but I haven't gotten around to this yet. The design of the library is inspired by the key management features of crypto hardware modules. The library is still unreleased and I wouldn't advise using it for production in its current state, but if you want to look at it and play with it, that's fine. In any case, its docs might give you some idea of how to deal with these issues. I also don't mean to disparage M2 or AMK's libraries in the least--they both look pretty good, I'm just not familiar in detail with either of them. Paul From marklists at MCEAHERN.COM Wed Jun 12 00:18:36 2002 From: marklists at MCEAHERN.COM (Mark McEahern) Date: Tue, 11 Jun 2002 17:18:36 -0500 Subject: [PYTHON-CRYPTO] credit card encryption In-Reply-To: <20020611215025.8253.qmail@brouhaha.com> Message-ID: [Paul Rubin] > It sounds to me like you're dealing with two different problems: > > 1) protecting the secret key on the web server--that's easy, don't > have a secret key there. That's presumably why you're using public > key encryption. Yes, that's correct. My plan is to store the public key on the web server--NOT the private key. > 2) Protecting the key on the separate server that talks to the payment > processor. Is that done over the internet? Is it a manual process? The payment processor provides a binary that talks to it via SSL. We have a Python C extension that talks to that. Ideally, the process could be both automated and relatively secure. Those may be mutually exclusive. ;-) > The simplest approach is keep that server disconnected from the > internet except when processing payments (I'm imagining a daily > upload). Unfortunately that means you can't do online card > verification > when the customer places the order (it's nice to tell them right > away that their card was declined, because they entered the wrong > zip code or something like that). We want to do authorize/settle stuff asynchronously from the customer placing an order, so this is not an issue. It will be a batch process, but the question is whether that batch requires human intervention to succeed (e.g., the human types in the passphrase that unlocks the private key for one batch). > If you have to keep the payment > server online, firewall it so that it can't talk to anything except > the card processor and the web server. Better yet, don't connect > it to the web server by TCP/IP. Connect it by serial port instead, > to further lower the chance of anyone rooting the payment server > by breaking through some TCP service. This is helpful. We haven't figured out the exact network layout, but it will most certainly be locked down more than the web servers. ;-) > The really hardcore way protect online secret keys is by generating > and storing them in a crypto hardware module. See for example > www.ibutton.com (look at the java crypto ibuttons) for examples. > You're supposed to do this if you're processing a lot of transactions > or storing a lot of card numbers (like 100's of thousands). If we get to that point, we'll worry about more fancy stuff. I'm trying to design something that is reasonably secure for thousands, tens of thousands, of credit cards. For what we're doing, that scale would be wildly successful. > I don't mean to be looking for opportunities to plug my own code but I > also have a library documented at > > http://www.nightsong.com/phr/python/crypto.txt > > which is written in pure Python. Awesome. I'm least concerned about performance at this point. I just want to make sure I take proper precautions, etc. Thanks, // mark - From marklists at MCEAHERN.COM Tue Jun 11 17:35:55 2002 From: marklists at MCEAHERN.COM (Mark McEahern) Date: Tue, 11 Jun 2002 10:35:55 -0500 Subject: [PYTHON-CRYPTO] /dev/random vs. /dev/urandom Message-ID: I recently learned the difference between /dev/random and /dev/urandom--the former blocks waiting for sufficient entropy while the latter never blocks. Fwiw, there's a decent explanation here: http://old.lwn.net/2001/0823/kernel.php3 There's no internal anchor to the section on /dev/random, but you can search for this heading: Feeding entropy from network devices. Also, I've appended the text to the bottom of this message. Apparently, the practical effect is that sometimes using /dev/random takes longer, but no one has demonstrated an attack against /dev/urandom. Nonetheless, I noticed in the recently released pycrypto-1.0a2 (http://www.amk.ca/python/code/crypto.html), the randpool.py initializes with /dev/urandom rather than /dev/random. That leads to two questions: 1. Should the selection of which random device to use be optional? 2. Should the code try to use /dev/random first, and if that doesn't exist, use /dev/urandom? Or am I being overly sophomoric? E.g., import os preferred_device = "/dev/random" alternate_device = "/dev/urandom" random_device = None if os.path.exists(preferred_device): random_device = preferred_device elif os.path.exists(alternate_device): random_device = alternate_device if random_device: try: f = file(random_device) data = f.read(self.bytes) f.close() self._addBytes(data) ... Cheers, // mark Excerpted from http://old.lwn.net/2001/0823/kernel.php3 Feeding entropy from network devices. The Linux kernel provides two pseudo-devices which generate random numbers: /dev/random and /dev/urandom. They both provide (seemingly) random numbers to applications, but they differ in one regard: /dev/random works much harder to ensure that the returned numbers are truly random. The random number generator works through the maintenance of an "entropy pool," a collection of random data which has been collected from outside sources. The most common source of entropy (randomness) in Linux systems is device interrupts; the time periods between keystrokes or disk interrupts is unpredictable enough to provide a degree of true randomness that can not be had from a software-only random number generator. Each random event adds a certain amount of entropy to the pool. If an application reads random data from /dev/random, the kernel will make sure that there is sufficient entropy in the pool to return truly random numbers; if the entropy is inadequate, the read will block until sufficient entropy has been generated. /dev/urandom, instead, will generate numbers (using a secure hash algorithm) regardless of whether sufficient entropy exists; it never blocks waiting for entropy. In theory, that difference means that a sufficiently clever attacker could, perhaps, predict the random numbers that will be generated by /dev/urandom. Using the predicted numbers, the attacker could proceed to make a mess of any cryptographic or security code using /dev/urandom. Such an attack remains entirely theoretical, however; it would be in no way easy, and nobody has ever demonstrated a way of successfully predicting Linux's random numbers. Nonetheless, people worry, and many applications will only use random data from /dev/random. On some systems, this can lead to problems if the system is not generating enough entropy; suddenly ssh connections take a long time to start up, and things get unresponsive in general. Network firewalls, with no keyboard and little or no disk activity can be especially susceptible to this problem. The answer, seemingly, would be to use the arrival of network packets as another source of entropy. Historically, this source of entropy has been avoided, since network traffic is susceptible to observation and manipulation by an attacker. In a highly paranoid world, one might worry about an attacker watching network traffic in an effort to predict the contents of the entropy pool on a target system; the attacker could also feed precisely-timed packets to the target in the hopes of influencing random number generation there. Once again, nobody has ever gotten close to demonstrating an attack of this nature, but if security people didn't worry they would have little to do. Now, however, Robert Love has submitted a patch which allows the system to use entropy from network traffic, subject to a kernel configuration option. There is some real opposition to the patch; some people feel that network entropy should not be treated as entropy at all, and that applications should just be using /dev/urandom in these cases. The wider consensus, however, is that sometimes network entropy is the best you can get, and that it makes sense to give the user a choice of whether to use it. After all, when, ten years from now, some super cracker develops a network entropy exploit, you can always turn the feature off. - From bram at GAWTH.COM Tue Jun 11 18:26:04 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Tue, 11 Jun 2002 09:26:04 -0700 Subject: [PYTHON-CRYPTO] /dev/random vs. /dev/urandom In-Reply-To: Message-ID: Mark McEahern wrote: > Apparently, the practical effect is that sometimes using /dev/random takes > longer, but no one has demonstrated an attack against /dev/urandom. There are possible attacks on /dev/urandom if there's no source of entropy on the machine at all - in that case /dev/random will instead block forever, while /dev/urandom will return predictable data. (like, if you reboot the machine it will return the same stuff the second time.) The core of /dev/urandom really should be better - it's not as secure as it should be against some theoretical attacks, and it should use aes for performance reasons, but it's basically sound. Why then, doesn't /dev/random block until it's got a sufficient initial seed, and then return data immediately? Paranoid security wankery. That's the only way to put it. There is no security justification. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From marklists at MCEAHERN.COM Tue Jun 11 19:38:46 2002 From: marklists at MCEAHERN.COM (Mark McEahern) Date: Tue, 11 Jun 2002 12:38:46 -0500 Subject: [PYTHON-CRYPTO] encrypting credit card numbers Message-ID: I need to encrypt credit card numbers. When the card number arrives at the web server, I plan to encrypt it with a public key and send it to a database the web server only has write access to. Then, later, a separate server reads the encrypted card number, decrypts it with the private key, and sends it to the payment processor. I'm not a cryptography expert. I've found two packages that offer public key encryption for Python: M2Crypto (http://www.post1.com/home/ngps/m2/) amk's Crypto toolkit (http://www.amk.ca/python/code/crypto.html) Both offer an implementation of RSA. In the preliminary searching I've done on RSA, there seems to be an issue related to padding. The M2Crypto encrypt/decrypt methods have an option to specify a type of padding (e.g., pkcs1_padding). amk's Crypto offers no such options. Do I have to worry about padding? Are there recommended strategies for padding in amk's Crypto with RSA? Simplistically, I would probably add a fixed amount of random information to the credit card number before encrypting it. Is that sufficient? The thing I'm most leery about is key management. M2Crypto provides a way to protect the private key with a passphrase. amk's Crypto suggests using pickle to save the key(s). I suppose I could use SHA to generate a key from a passphrase, and then use that key to symmetrically encrypt the private key data before I save it to a file. Of course, how does the batch process for sending payment information to the payment processor get access to the private key? I haven't resolved that issue. Worst case scenario is the batch has to wait until someone types in the passphrase to unlock the private key. Can anybody point me to resources for key management? Any other feedback, comments, suggestions would be most helpful. Thanks, // mark - From akuchlin at mems-exchange.org Tue Jun 11 19:50:31 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Tue, 11 Jun 2002 13:50:31 -0400 Subject: [PYTHON-CRYPTO] /dev/random vs. /dev/urandom In-Reply-To: References: Message-ID: <20020611175031.GE24388@ute.mems-exchange.org> On Tue, Jun 11, 2002 at 10:27:31AM -0500, Mark McEahern wrote: >Nonetheless, I noticed in the recently released pycrypto-1.0a2 >(http://www.amk.ca/python/code/crypto.html), the randpool.py initializes >with /dev/urandom rather than /dev/random. That leads to two questions: It uses /dev/urandom simply because urandom won't block; using /dev/random, the test suite could just hang, waiting for enough entropy to accumulate. >1. Should the selection of which random device to use be optional? Quite possibly; this is after all a choice of mechanism that users might not agree with. I haven't yet come to a decision on this, and it might well change in 1.9alpha4. (1.9alpha3 will be coming out tonight, in order to fix a refcounting error in 1.9a2 that causes core dumps.) --amk (www.amk.ca) HAMLET: Let me not think on't -- Frailty, thy name is woman! -- _Hamlet_, I, ii From itamarst at YAHOO.COM Wed Jun 12 11:23:21 2002 From: itamarst at YAHOO.COM (Itamar S.-T.) Date: Wed, 12 Jun 2002 02:23:21 -0700 Subject: [PYTHON-CRYPTO] encrypting credit card numbers In-Reply-To: Message-ID: <20020612092321.47372.qmail@web13005.mail.yahoo.com> --- Mark McEahern wrote: > I need to encrypt credit card numbers. When the > card number arrives at the > web server, I plan to encrypt it with a public key > and send it to a database > the web server only has write access to. Then, > later, a separate server > reads the encrypted card number, decrypts it with > the private key, and sends > it to the payment processor. Maybe use GPG with some wrapper library - I've done this, and it has the benefit of letting humans easily decrypt the data on Windows as well using PGP for Windows. This one looks nice: http://py-gnupg.sourceforge.net/ ===== Itamar Shtull-Trauring, itamar(at)shtull-trauring.org __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com