[IPython-dev] display protocol: animation and video

[Fernando Perez]

On Fri, Dec 16, 2011 at 8:18 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> So we have not thought about animiation in the display protocol yet.
> Definitely room for improvement.

I tried to hack up something quick for Chris using the html5 video tag
with an mp4 file, but it's not easy to make it work b/c the notebook
doesn't expose access to the local filesystem for html tags (that's
why we encode inline the image data in the Image() object).

Which got me thinking about that issue in a more general sense.  I
think we should offer a way for users to refer to local files in the
starting directory of the notebook, so that they could expose them to
the browser in situations like this one.  I know there are a few
important caveats to keep in mind, namely:

Caveat 1: in the most general case, we must not assume that the
notebook server and the kernel reside on the same host.  Just
yesterday chatting with Titus Brown he had a good use case for running
the notebook server locally on a laptop but accessing kernels living
in a remote Amazon EC2 instance.  This would let him save notebooks
locally and would eliminate upstream traffic of images while letting
him work on large, amazon-hosted datasets.

Still, I think that for the common case where the notebook server and
the kernels do reside on the same filesystem, offering an easy
solution is worthwhile.  It won't apply to scenarios such as Titus'
above, but it will be very useful in many cases.

Caveat 2: the working directory of the kernel can vary at runtime (the
user can run %cd/os.chdir() at any time). So the filesystem served by
the notebook over http would be rooted where it started the kernel.
If that kernel later moves to a different working directory, so be it.
 It's up to the user to keep track of this and adjust his/her code
accordingly.

Caveat 3: security.  The notebook server needs to sanitize paths to
avoid accessing remote paths, which means always checking that a
requested path is below the starting point via abspath().

Caveat 4..n: everything else I haven't thought of yet.



But even given those, I think it would be very useful to offer a
handler that would allow people to return objects whose repr includes
things like:

<video src="file/movie.webm" poster="file/movie.jpg" controls>
       Text for non-video browsers.
</video>

or using the Markdown image syntax to show local images in Markdown
cells.  We'd write a handler for the 'file/' URL that would simply
search the rest of that URL as a local file in the directory where the
notebook started a local kernel.  For remote kernels, it would default
to the server's filesystem since the server can't access the kernel FS
(this would be useful in a scenario such as Titus', for showing local
files like images or video in Markdown areas).

Thoughts?

Cheers,

f