Alternatives for pickle?

[Bengt Richter]

On 11 Oct 2004 08:26:12 GMT, Antoon Pardon <apardon at forel.vub.ac.be> wrote:

>I'm writing a little game, a gridler application, where you
>can turn pixmaps into puzzle's and try to solve them. I already
>have the data structure for such a puzzle worked out, one of
>the problems is writing it to a file and reading it back in.
>
>I first went to the pickle module but there I read this.
>
>| Warning: The pickle module is not intended to be secure against
>| erroneous or maliciously constructed data. Never unpickle data
>| received from an untrusted or unauthenticated source.
>
>But since this is for a game and people should be able to
>exchange puzzles, it seems a heavy requirement to ask of
>the users to check a puzzle file for security hazards.
>
>
>I also thought about writing out a string that, when read
>back in and fed to eval would recreate the structure. But
>that seems to be just as insecure if not more so.
>
>So how do you serialize data in python, when you want
>a somewhat secure mechanisme. Preferably where a user
>can make a puzzle file by hand in a text editor.
>
I would consider saving and retrieving your puzzle info in a simple csv format.
You can invent your own very simple interpreter based on lines of the
form
    cmd, whatever...

the csv module has methods and options to control delimiters and quoting etc,
but e.g., by default:

 >>> import csv
 >>> lines = """
 ... this, can, be, a, command, format
 ... cmd, easy to edit with editor, note what happened to spaces
 ... cmd, arg, "quoted arg", etc
 ... do, something, else
 ... push, something on a stack
 ... set, something,to,a,value
 ... call, afunction, arg1, arg2, etc
 ... etc
 ... """
 >>> rdr = csv.reader(lines.splitlines())
 >>> for row in rdr: print row
 ...
 []
 ['this', ' can', ' be', ' a', ' command', ' format']
 ['cmd', ' easy to edit with editor', ' note what happened to spaces']
 ['cmd', ' arg', ' "quoted arg"', ' etc']
 ['do', ' something', ' else']
 ['push', ' something on a stack']
 ['set', ' something', 'to', 'a', 'value']
 ['call', ' afunction', ' arg1', ' arg2', ' etc']
 ['etc']

Anything that will iterate by lines should be ok to pass to csv.reader
so you can pass an open file, e.g., file('mypuzzlesetup.txt')

Obviously, you can interpret row[0] as operation and do what you like, e.g.,


 >>> rdr = csv.reader("""\
 ... abs, 123
 ... abs, -123
 ... sum, 1,2,3
 ... sum, 100,200,5
 ... xxx, what?
 ... """.splitlines())

 >>> for row in rdr:
 ...     cmd = getattr(__builtins__, row[0], None)
 ...     if cmd is None: print 're-edit your info:',row
 ...     else:
 ...         args = map(int, row[1:])
 ...         if len(args)==1: args=args[0]
 ...         print row,'=>', cmd(args)
 ...
 ['abs', ' 123'] => 123
 ['abs', ' -123'] => 123
 ['sum', ' 1', '2', '3'] => 6
 ['sum', ' 100', '200', '5'] => 305
 re-edit your info: ['xxx', ' what?']

The csv module also has stuff to control delimiters and a writer method, etc.
See help(csv) interactively, after importing csv.

Note that no one can introduce an xxx do do anything weird, and if you
validate all the command formats, and don't do int conversions etc without
try/except, etc, you should be able to reject anything not safe, and write
informative error messages. You don't necessarily have to abort on any and
all errors, but you can have options for that if you want to get fancy.

HTH

Regards,
Bengt Richter