[IPython-dev] [sympy] Re: using reST for representing the notebook cells+text

[Ondrej Certik]

On Wed, Feb 24, 2010 at 12:10 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> On Wed, Feb 24, 2010 at 11:15 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>> I will do reST import/export, but having it the default format, that I
>>> think sucks. With xml, anyone can easily write a parsers for it in any
>>> language.
>>>
>>> Besides, I need to push the worksheet to the browser. In my case, I
>>> use pyjamas, so it's pure Python, however, exec/eval are not evailable
>>> there (as far as I know), but I am sure xml would work, here is some
>>> code to handle xml in pyjamas:
>>
>> A point of clarification.  The Notebook and Cell objects would not be
>> responsible
>> for the execution of python code.  The execution logic would simply
>> pull InputCells
>> out of the Notebook, extract the python code (as a str) and return the
>> OutputCell.
>>
>> [Thinking out loud about what this might look like in a web
>> browser....more thought needed]
>>
>> The Python Notebook and Cell objects would not be pyjamas entities - that
>> part of the python code would not be compiled to Javascript.  The
>> Notebook and Cell
>> objects would live in the web server and be capable of rendering
>> themselves as json
>> or html.
>>
>> The pyjamas classes would contain the logic for the user interface,
>> and calling out to the
>> Notebook/Cell web service and pass XML/json back and forth....
>
> That's right. I just did more studying of this and I don't want to
> parse xml by hand in javascript/pyjamas. I would like to use the
> builtin functionality
>  for RPC, essentially, you just call python methods on both sides and
> things get automatically transfered (using json/xml RPC) behind the
> scene, it's very convenient.
>
>>
>>> I think, that it might work to push the whole Python objects somehow
>>> (that's the nice thing about pyjamas, that since you have python on
>>> both sides, you can do lots of cool stuff), but maybe not.
>>
>> Yes, that might work.  I have been thinking about the Cell objects as having
>> all the logic needed to render itself in various formats.  But maybe a
>> Cell object
>> itself could be lighter weight - the rendering logic could be a separate class.
>> Then the Cell objects could be compiled to JS using pyjamas.  But I am
>> not familiar
>> with how pyjamas works enough to see if this would work.
>>
>>> I like Brian's idea to be able to just import it in Python. But I
>>> think I'll just stick to xml for the first iteration of my notebook,
>>> to get the job done.
>>
>> Definitely don't wait for me to implement my idea...I do agree with
>> your statement about
>> the ease of parsing XML though.
>
> Here is my current code, that runs in the browser:
>
> http://github.com/certik/sympy_gamma/blob/login/templates/nb.py
>
> and here is the code, that runs on the server, listens at some url
> (/eval_cell/...) and evaluates the cell:
>
> http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126
>
> it's just 15 lines, 7 of those are debug prints.
>
>
> So there is no way to remember sessions, worksheets anything yet. So I
> will have Notebook/Cells on the server, the eval_cell() method will
> (besides the actual evaluation) keep the tree structure of
> Notebook/Cells updated. This structure will know how to export/import
> to/from many formats (reST, xml, ...), that part is clear.
>
> The rendering itself is done using pyjamas, so the Notebook/Cells
> classes don't need to know anything about it, they just contain the
> logic.
>
> On the browser part, I will also need some
> Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for
> rendering and I need somehow to initialize them from the server. I
> think I will do that by implementing Notebook_pyjamas.add_cell()
> method, that will be exported using RPC to the server, so the server
> will call this method as many times as needed, and add there
> parameters (input_text, output_text, type_of_the_cell="text"/"eval").

The browser part is actually already implemented (see the nb.py file
above), here is how I call it:

CellWidget
Worksheet

I think I will rename Worksheet to WorksheetWidget, then it will be
clear that it is just a widget.

Hm, so it is more complex. The WorksheetWidget currently contains
methods to join cells, etc. Currently all this "technical" logic is
determined in pyjamas (e.g. the cells can only by joined sometimes,
etc.). So I think the WorksheetWidget would just call
Notebook.join_cell() RPC call on the server, when the cells were
joined.

Note sure now, what to do with synchronization. E.g. let's say the RPC
call fails (no net connection), should the user wait until the RPC
call finishes to make sure the cells can be joined, or should they be
joined in the browser immediately and then somehow remember all the
pending changes to push them to the server? I will ignore this problem
for now.

I will start by implementing this RPC logic --- currently I don't use
RPC to evaluate cells, here is what I do now:

http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L335

I simply do an asynchronous html request, pass in my Loader()
instance, and its onCompletion() method gets called, I decode the JSON
data and that's it. So I will start by rewriting this part of the code
to use RPC, and then slowly increase the RPC interface on both sides
and implement Notebook and other things on the server and simply
communicate between the server and the browser more often.

Here is a demo how to use RPC in pyjamas:

http://pyjs.org/examples/jsonrpc/output/JSONRPCExample.html

and here is the code:

http://pyjs.org/examples/jsonrpc/JSONRPCExample.py

And more documentation is here:

http://pyjs.org/book/output/Bookreader.html#Rest%20of%20the%20World


So far this is how to call methods (implemented on the server) in
pyjamas, e.g. stuff like for evaluating cells. So after pyjamas
starts, I will call the server method "give_me_the_cells", and I guess
it can just pass me a python dictionary with the cells data, and I
initialize it according to it.

So it seems the RPC is always initiated from the browser.

> Yes, you will have to add urls and handlers that know how to add
> cells, delete cells,
> etc.  I would follow a RESTful design of those URLS:
>
> http://en.wikipedia.org/wiki/Representational_State_Transfer

Actually, from the pyjamas side, I will just call methods, no need to
bother with URLs, and on the server side (django), there is also a
functionality in django, so that I don't have to bother with URLs
either. It's explained in the pyjamas docs link, above.

Ok, I think I know what to do now, there are still couple details that
are not clear to me, but it will sort itself out, once I find some
time to implement it.

Ondrej