pickle/marshal internal format 'life expectancy'/backward compatibility

[Philippe C. Martin]

> How complicated is your data structure?  Might you just store:

that's the thing: I don't want to know: My goal is to be able to design
my SC applications without having to change the file system every time I
wish to add/subtract data to be store - I should say 'constant' data to be
stored for this pickle solution would only work with data that the SC
application just stores but does not understand/nor needs to.
For instance, the data structure could simple have a name and address to
start with (when the cards get issued) then the customer might want to add
the person's pictures. In such a case, I would not have to release a new
smart card application (could be burned in the silicon) but simply load
the new structure at issuance time.

> Of course, the security of this scheme is dependent on a lot, including
> the strength of sha, your ability to keep your secret key secret, the
> correctness of what I'm saying, etc etc etc.

The crypto/mutual authentification scheme is handled by the smart
card/HSM or in some _very_ light applications, by the smart card and
pycrypto (no HSM or batch card on the "terminal" side)

Regards,

Philippe





On Sun, 06 Feb 2005 02:20:34 -0500, Adam DePrince wrote:

> On Sat, 2005-02-05 at 17:04, Tim Peters wrote:
>> [Philippe C. Martin]
>> > I am looking into using the pickle format to store object/complex data
>> > structures into a smart card as it would make the design of the embedded
>> > application very simple.
>> >
>> > Yet the card might have to stay in the pocket of the customer for a few
>> > years, during which the back office application responsible for
>> > retrieving the information from the card might evolve as well as the
>> > python release it relies upon.
>> >
>> > Is there a commitment for python releases to be able to interpret
>> > 'older' pickle/marshal internal formats ?
>> 
>> Pickle and marshal have nothing in common, and there's no
>> cross-release guarantee about marshal behavior.  Pickles are safe in
>> this respect; indeed, each Python released to date has been able to
>> load pickles created by all earlier Python releases.  Of course a
>> pickle produced by a later Python may not be loadable by an earlier
>> Python, although even in that direction you can get cross-release
>> portability by forcing the newer Python to restrict itself to produce
>> obsolete pickle formats.  Reading Lib/pickletools.py in a current
>> release will explain all this.
> 
> 
> How complicated is your data structure?  Might you just store:
> 
> repr( <mydatastructure> )
> 
> and eval it later?  Trust is an issue; you are vulnerable to malicious
> code, but no more so than with pickle or marshal.
> 
> One quick and dirty way to be a little safer is to "sign" the data you
> store.
> 
> # To save your data
> import sha
> import cPickle
> mysecret = "abc"
> mydata = {"what":"my data structure"}
> f = open( "/tmp/myfile.txt", "w+" )
> mydata = cPickle.dumps( mydata, protocol=0 )
> # I'm assuming this is a flash device ... lets be safe and not assume
> # that write is buffered ... 
> f.write( sha.new( mysecret + mydata ).digest() + mydata)
> 
> 
> 
> # To load your data
> import sha
> import cPickle
> mysecret = "abc
> f = open( "/tmp/myfile.txt", "w+" )
> hash = f.read( sha.digest_size )
> mydata = f.read()
> if sha.new( mysecret + mydata ).digest() != hash:
> 	raise "Somebody is tring to hack you!" 
> mydata = cPickle.loads( mydata )
> 
> Of course, the security of this scheme is dependent on a lot, including
> the strength of sha, your ability to keep your secret key secret, the
> correctness of what I'm saying, etc etc etc.
> 
> 
> 
> 
> Adam DePrince