[python-list] python application file format

[Prasad, Ramit]

Benjamin Jessup wrote:
> Hello all,
> 
> What do people recommend for a file format for a python desktop
> application? Data is complex with 100s/1000s of class instances, which
> reference each other.
> 
> Write the file with struct module? (Rebuild object pointers, safe,
> compact, portable, not expandable without reserved space)
> 
> Use cPickle with a module/class whitelist? (Can't easily port, not
> entirely safe, compact enough, expandable)
> 
> Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
> 
> Any advice is greatly appreciated!

I would think your options are pickle, json or database (either sqlite or 
something like Postgres). I am unfamiliar with the struct module so I 
cannot comment on its applicability.

I would guess that your data would be best saved by using a sqlite 
database. Your biggest problem might be how the different classes are 
referencing each other. If you are using identifiers then any of these 
options will probably work. If you are using aggregation then I know
that pickle will work (at least for built-in types). JSON will
keep the structure but duplicate elements.


>>> a = [ 1,2,3 ]
>>> b = [ 'a', 'b', 'c' ]
>>> a.append( b )
>>> e = [ a,b ]
>>> s = json.dumps( e ) 
>>> eret = json.loads( s )
>>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson
(329443808, 327677272) 
>>> eret[0][3].append( 'o')
>>> eret[0][3], eret[1]
([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])

So pickle will be your easiest option, but I am not sure how well it
will scale with a large number items. Using sqlite/db should scale well
but it will take you longer/more effort to create a system for converting 
your objects to and from the DB.



This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.