Python - remote object protocols and security

[Burak Arslan]

On 07/15/13 13:51, Chris Angelico wrote:
> So the only bit you still need is: How do you transmit this across the
> network? Since it's now all just bytes, that's easy enough to do, eg
> with TCP. But that depends on the rest of your system, and is a quite
> separate question - and quite probably one you already have the answer
> to.

For Json, you need to have a way of delimiting messages -- to my
knowledge, Python's json library does not support parsing streams.

You can send the json document in the body of a Http POST, or a ZeroMQ
message, or in a UDP datagram (if you can guarantee it fits inside one)
or in a simple TCP-based encapsulation mechanism that e.g. prepends the
length of the message to the document.

e.g.

'\x00\x00\x00\x07{"a":1}'

As MessagePack already does this, you can send MessagePack documents via
an ordinary TCP socket and easily recover them on the other side of the
pipe.

>>> import msgpack; from StringIO import StringIO
>>> s = StringIO(msgpack.dumps({"a":1}) + msgpack.dumps({"b":2}))
>>> for doc in msgpack.Unpacker(s):
...     print doc
...
{'a': 1}
{'b': 2}

This won't work with Json:

>>> import json; from StringIO import StringIO
>>> s = StringIO(json.dumps({"a":1}) + json.dumps({"b":2}))
>>> for doc in json.load(s): # or whatever ???
...     print doc
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 368, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 9 - line 1 column 17 (char 8 - 16)

Note that this is a limitation of python's Json parser, not Json itself.

There seems to be a json.scanner module that *sounds* like it provides
this functionality,
but I couldn't find any documentation about it.

Alternatively, PyYaml can also parse streams. yaml.{dump,load}_all()
provide pickle-like unsafe (de)serialization support and
yaml.safe_{dump,load}_all provide msgpack-like safe-but-limited stream
parsing support.


also;

On 07/15/13 13:57, Chris Angelico wrote:
> But what I meant was that the [Json] protocol itself is designed with
> security restrictions in mind. It's designed not to fetch additional
> content from the network (as XML can),

Can you explain how parsing XML can fetch data from the network?


Best,
Burak