multiple JSON documents in one file, change proposal

[Marko Rauhamaa]

Chris Angelico <rosuav at gmail.com>:
> On Sat, Dec 1, 2018 at 9:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
>> The need for the format to be "typable" (and editable) is essential
>> for ad-hoc manual testing of components. That precludes all framing
>> formats that would necessitate a length prefix. HTTP would be
>> horrible to have to type even without the content-length problem, but
>> BEEP (RFC 3080) would suffer from the content-length (and CRLF!)
>> issue as well.
>
> I dunno, I type HTTP manually often enough that it can't be all *that*
> horrible.

Say I want to send this piece of JSON:

   {
       "msgtype": "echo-req",
       "opid": 3487547843
   }

and the framing format is HTTP. I will need to type something like this:

   POST / HTTP/1.1^M
   Host: localhost^M
   Content-type: application/json^M
   Content-length: 54^M
   ^M
   {
       "msgtype": "echo-req",
       "opid": 3487547843
   }

That's almost impossible to type without a syntax error.

>> Finally, couldn't any whitespace character work as a terminator? Yes,
>> it could, but it would force you to use a special JSON parser that is
>> prepared to handle the self-delineation. A NUL gives you many more
>> degrees of freedom in choosing your JSON tools.
>
> Either non-delimited or newline-delimited JSON is supported in a lot
> of tools. I'm quite at a loss here as to how an unprintable character
> gives you more freedom.

As stated by Paul in another context, newline-delimited is a no-go
because it forces you to restrict JSON to a subset that doesn't contain
newlines (see the JSON example above).

Of course, you could say that the terminating newline is only
interpreted as a terminator after a complete JSON value, but that's not
the format "supported in a lot of tools".

If you use any legal JSON character as a terminator, you have to make it
contextual or add an escaping syntax.

> I get it: you have a bizarre set of tools and the normal solutions
> don't work for you. But you can't complain about the tools not
> supporting your use-cases. Just code up your own styles of doing
> things that are unique to you.

There are numerous tools that parse complete JSON documents fine.
Framing JSON values with NUL-termination is trivial to add in any
programming environment. For example:

   def json_docs(path):
       with open(path) as f:
           for doc in f.read().split("\0")[:-1].
               yield json.loads(doc)


Marko