[IPython-dev] custom web notebook cells

[Jason Grout]

On 12/15/12 2:00 PM, Brian Granger wrote:

> I think there is a big design pattern that is emerging in the
> notebook.  Currently CodeCells have an input area and output area.
> Obviously, these were originally created for code and the output of
> the code.  But, with the Javascript plugins, the output area has
> really become more than merely output.  In fact here is how I view it
> now...
>
> * The input area is used for python code that initializes a "widget".
> That Python code is used to create the widget, configure it, provide
> input to it, etc.
> * The output area is used for the widget UI.  Widgets can be arbitrary
> JS/HTML/CSS.  Widgets can make calls back to the kernel and receive
> further JSON data back.  Widgets can even be nested.

That's a nice way to think about it.  Can we extend it one level 
further?  One thing I've long dreamed for Sage is to have javascript 
widgets in the *input* area as well, and a way to flip back and forth 
between a text representation and other representations for the widgets. 
  So, for example, you could have:

a=<<numpy array>>

and the <<numpy array>> would be a small javascript spreadsheet control. 
  Hit a button or key, and the spreadsheet control converts to an 
equivalent text representation (which is what is actually sent over the 
wire to the kernel).  Or I'd like to have:

n=<<slider>>

where <<slider>> is a jquery slider, letting me pick the value for n.

Or have:

f=<<math expression>>

where <<math expression>> is a mathjax-enabled input widget that lets me 
type in normal mathematics.  Hit a button and it gets converted to the 
text equation representation.

Or:

Process(<<image>>)

and <<image>> is an actual picture (which can be flipped to a text 
representation, which might be ascii encoded, or might be a statement 
downloading an image, or something else).




>
> Here is an example that I have started to work on.  The new Web Audio
> API allows Javascript to access the raw audio data from a microphone.
> Using this, it is possible to create a widget that records audio and
> processes and visualizes the audio in JS+Python+matplotlib+d3+etc.

Wouldn't it be cool to have, in the *input*:

audio=<<widget>>
p=Process(audio)
p.save('newaudio.wav')



where widget has a record, playback, etc. buttons.  Hit the record 
button, and it records a data file and ships it to the server and sets 
the result to a.  Hit another button, and you could actually see a text 
representation of what `audio` is going to be set to, or even better, a 
waveform representing what `audio` will be set to.

These ideas also encapsulate a lot of the other cell types mentioned. 
Why have different types of cells, when you can just have different 
widgets inside the same cell?  What if I want to have a spreadsheet 
widget representing a numpy array and an audio widget representing an 
audio file in the same input cell?

Thanks,

Jason