[IPython-dev] ipython with sync

[Brian Granger]

Thanks for this link, that is helpful.

Cheers,

Brian

On Thu, Sep 12, 2013 at 4:04 PM, William Stein <wstein at gmail.com> wrote:
> On Thu, Sep 12, 2013 at 2:53 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> William,
>>
>> Thanks for notes.  This is really exciting and we are very interested
>> in getting live sync to work with IPython itself.  I am traveling this
>> week, but will try to have a look soon.  Is the code available
>> somewhere to look at?
>
> Unfortunately, this implementation is very entangled as part of
> something I'm building as part of a startup company with UW's Center
> for commercialization, and I can't just open source everything.  But
> the algorithm is here:
>
>   https://neil.fraser.name/writing/sync/
>
> and the author of that paper has an open source Python implementation,
> I think...
>
>
>>
>> Cheers,
>>
>> Brian
>>
>> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>>> Hi,
>>>
>>> Somewhat by accident I spent the last two weeks implementing hosted
>>> IPython notebooks with sync for https://cloud.sagemath.com.
>>> Initially I had just plan to simplify the port forwarding setup, since
>>> what Ondrej Certik was doing with multiple forward and reverse port
>>> forwards seemed complicated.  But then I became concerned about
>>> multiple users (or users with multiple browsers) overwriting each
>>> other's notebooks, because cloud.sagemath projects are frequently
>>> shared between multiple people, and everything else does realtime
>>> sync.    I had planned just to add some very minimal merge-on-save
>>> functionality to avoid major issues, but somehow got sucked into
>>> realtime sync (even with the other person's cursor showing).
>>>
>>> It would be enormously helpful to me if a couple of expert IPython
>>> users were to try out what I implemented and just ask a bunch of
>>> questions.
>>>
>>> 1. Go to https://cloud.sagemath.com and make an account; this is
>>> completely free, and is hosted on computers at University of
>>> Washington.
>>>
>>> 2. Create a new project.
>>>
>>> 3. Click +New, then click "IPython" (or paste in a link to an ipython
>>> notebook, or upload a file).
>>>
>>> 4. An IPython notebook server will start, the given .ipynb file should
>>> load in a same-domain iframe, and then some of the ipython notebook
>>> code is and iframe contents are monkey patched, in order to support
>>> sync and better integration with https://cloud.sagemath.com.
>>>
>>> 5. Open the ipynb file in multiple browsers, and see that changes in
>>> one appear in the other, including moving cells around, creating new
>>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>>> (evaluating a cell that creates no output doesn't set this flag, at
>>> least in 1.0.0, which is a bug in IPython, I guess).
>>>
>>> Since this is all very new and the first (I guess) realtime sync
>>> implementation on top of IPython, there are probably a lot of issues.
>>>  Note that if you click the "i" info button to the right, you'll get a
>>> link to the standard IPython
>>>
>>> The other thing of interest is a little Python script called
>>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>>> an IPython notebook server as a daemon, get the port it is running on,
>>> etc.  It's pretty simple but satisfies my use case, and has
>>> JSON-output, to make it web friendly.    As I've written it, my script
>>> passes several base_url options through by default, which are needed
>>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>>> BSD license) in case there is any interest.
>>>
>>> Regarding the monkey patching in 4 above, the right thing to do would
>>> be to explain exactly what hooks/changes in the IPython html client I
>>> need in order to do sync, etc., make sure these makes sense to the
>>> IPython devs, and send a pull request (or have a coding sprint in
>>> Seattle or Berkeley?).  As an example, in order to do sync
>>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>>> critical to do this in place when possible, since the overhead of
>>> creating a new cell is huge (due probably to the overhead of creating
>>> CodeMirror editors); however, the fromJSON method in IPython assumes
>>> that the cell is brand new -- it would be nice to add an option to
>>> make a cell fromJSON without assuming it is empty.
>>>
>>> The ultimate outcome of this could be a clean well-defined way of
>>> doing sync for IPython notebooks using any third-party sync
>>> implementation.  IPython might provide their own sync service and
>>> there are starting to be others available these days -- e.g., Google
>>> has one: https://developers.google.com/drive/realtime/, and maybe
>>> Guido van Rosum helped write one for Dropbox recently?
>>>
>>> Subdirectories:   I noticed, incidentally, that the wakari version of
>>> the IPython notebook server allows one to load ipynb files that are in
>>> any subdirectory, whereas the standard IPython notebook server
>>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>>> server for each directory that a user accesses files in right now.
>>> This seems cludgy, so I'm interested in the situation regarding adding
>>> support for subdirectories.
>>>
>>> -- William
>>>
>>>
>>>
>>> --
>>> William Stein
>>> Professor of Mathematics
>>> University of Washington
>>> http://wstein.org
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com