[IPython-dev] custom web notebook cells

[Nicholas Bollweg]

sorry, my initial setup of digest makes it hard to get at the thread
structure directly, so hopefully this gets in the right place.

the feedback has been very good, impressive community (and product, of
course)
you have built here!

    > * We need to maintain the ability to install ipython without
setuptool.

whoa, very "there is no spoon." just went back and read some tickets that
mention setuptools/distutils. will respect the community's desires, here!

so i guess, if primarily-javascript packages *were* to be installed via
python,
the IPython Way could be to have some %load_ext cells at the
beginning of the notebook that would somehow change the static
lookup path at runtime. i'll revisit my investigation of the extensions!
once
there is one example like this, it should be easy to repeat the pattern.

    > Before we start actually coding on the require.js stuff or
    > reorganizing that stuff, we need to figure out what we want to do
    > exactly.  Here is an IPEP for us to work all of that out:

thanks for the pointer to the IPEP5, very good scope! while i am learning
to
love node, i might avoid bringing in the node dependency until
someone really has that itch to scratch... i could imagine the 0mq layer
being
perfect for interacting between python and node... under the correct
conditions.

ghost.py can handle the testing requirements, though it will crank up the
travis test runs by a pretty serious amount, due to the installation of qt
and
friends... ah! but The Other Notebook already uses qt, so maybe this isn't
as
bad of a hit. here's the ghost.py .travis.yml that is the recipe i have used
for getting it running:

    https://github.com/jeanphix/Ghost.py/blob/master/.travis.yml

having node around to do npm might not actually meet the needs of
the notebook plugin builder (see below)... webassets looks good, but
doesn't
handle the dependency tree. tough choices ahead.

    > * We have to serve these files on the web.  I don't want to have our
    >   web server walking all over a users system to pull files as that is
a
    >   huge security risk.  We need to isolate the location of these files.

yes, definitely a concern... need to wrap my head around this... what makes
the
most sense? options:

- copy stuff into the ipython location
  - seems bad
- have multiple static_paths
  - my current implementation, relies on runtime setuptools
- collect all static deps into a tmpdir which only contains the files needed
  - has potential...

    > * Some plugins don't have any python code so I am not sure it makes
    >   sense to bundle things into a python package.

indeed, i have fought with this demon a few times, most recently with
apache
allura's homerolled asset management system. putting my sysadmin hat on,
i'd have the fewest dependency management systems possible... apt-get/yum +
pip.
it's an immediate turnoff if i have to do much more than that. packaging as
python does have the whole pypi advantage, though.

of course, what could end up happening is that the plugins would have
versions
as well, and it could be that two different .ipynb's you wanted to look at
would use different versions. this suggests, perhaps, that the plugins
should
be installed into the notebook itself, which also chills me to the bone.

i guess we need to understand how a user would want to use these...
definitely
have to do more thinking.

    > But, I would like to learn more about your usage cases.  There might
    > be other ways of supporting what you want to do:
    > 1) Use an existing cell type with metadata.
    > 2) Provide a custom UI for an existing cell type but don't change the
    >    notebook format.
    > 3) Actually add a new cell type to the notebook format officially if
it
    >    is something that a majority of our users would use often.

my long-term vision is to build on what mathematica/jmp/tableau can do:
support
freeform exploration by a developer/researcher/student of a data-driven
problem, which gradually solidifies into a dashboard (for lack of a better
term) that allows an end user to play in a constrained data space, but
access
advanced features.

while getting there, i want to better be able to document what i am
thinking
about it one, context-switching-free space that versions/merges well.

i never thought of doing anything that would do (3): i am indeed
sensitive to the long-term viability of data formats, and want to do
whatever i
can to avoid contributing to "data death"... all the stuff i want to do is
very
web-focused (for both UI and output), and is therefore inherently
text-based...
it might just be Really Boring text if you are at the command line (hi,
SVG!).
i was thinking about it primarily from a UI point of view, the current
implementation of which centers around the IPython.Cell subclasses... so
basically what i want is what you describe in (2).

my use cases for custom cells are sort of summed up (mostly as
implementation ideas) in this ticket on my "write some d3 visualizations
with
blockly" project:

    https://github.com/bollwyvl/blockd3/issues/10

but here's some riffing on what i've learned in the last 24 hours :)

use cases:

- VisualCodeCell http://code.google.com/p/blockly/
    - motivation
        - i've helped students reach the the "a ha" moment with
          blockly's spiritual ancestor, scratch, but feel like the problems
          that python is able to tackle are much greater. i see the ipython
          notebook as a great home for the blockly python functionality, as
it
          can be learned by the absolute novice... when i was a python
novice,
          i think i had the "a ha" moment about five years ago when i
learned
          about tab completion and ? and ?? in the ipython console!

          i have been experimenting with blockly (which can generate
python) a
          good deal, but exclusively for javascript. presently blockly is
kind
          of lame for a python workflow.
    - implementation
        - CodeCell. blockly xml would fit nicely inside cell metadata,
which i have yet to explore, but need to!
    - challenges
        - the downside here is that any editing of the python outside of
the
          web environment (or if you didn't have the ipython-blockly plugin
i
          envision), the blockly environment can't reverse-engineer its xml
          from python. however, since blockly is more aimed at the novice,
this
          might not cause that much of an issue.
- DiagramCell http://code.google.com/p/svg-edit/
    - motivation
        - i love me some svg... it is deeply ingrained in my day-to-day
design
          and communication workflow with inkscape, and has become
increasingly
          important from my web application development tasks with the
meteoric
          rise of d3. svgedit is Almost As Good as inkscape, and for a lot
of
          things i want to do, it would be sufficient for a lot of quick
          vector-based documentation things that i'd want to attach.
    - implementation
        - MarkdownCell
    - challenges
        - none, really
- SpreadsheetCell https://github.com/bollwyvl/blockd3/issues/SlickGrid
    - motivation
        - some data really is tabular. i would love to be able to whip up a
          quick table (not using markdown!) and then pop that into a numpy
          calculation chain.

          also, being able to quickly walk around higher-order numpy arrays
          would be great, suggesting an excel-like sheet navigation for a
given
          array.
    - implementation
        - CodeCell. perhaps some metadata for formatting
    - challenges
        - not trying to implement Resolver One here, but a little bit of
          functions would be cool... i like some of the things that the
google
          spreadsheet has done to blow out a calculation into a bunch of
cell
          from input into one cell
- TangleCell http://worrydream.com/Tangle/
    - motivation
        - the tangle reactive document paradigm is awesome. i have done
some work with integrating this with markdown, but it never felt right,
          and took a lot of `exec` to work properly, which always gave me
the
          willies. writing constraints directly on the backend and then
just
          tying them to frontend in markdown seems like the high road.
    - implementation
        - MarkdownCell
- ControlCell
    - motivation
        - sometimes, i just want a button or a slider for changing a value.
    - implementation
        - CodeCell. metadata.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/379874d8/attachment.html>