[IPython-dev] tracking nbconvert-generated images in version control

[Nathan Goldbaum]

Hi Greg,

A while back I came up with this:

https://github.com/ngoldbaum/RunNotebook

RunNotebook consists of two sphinx extensions that take an unevaluated
notebook and convert it into a form suitable for inclusion in sphinx HTML
documentation.  That way we get small notebooks that contain only text
added to our documentation version control but also full evaluated
notebooks in the documentation we publish online.  As a bonus, this is an
additional level of testing for our project since code is evaluated every
time the docs are built.

The real work is being done by runipy (https://github.com/paulgb/runipy/)
which allows you to run notebooks without going into the notebook web
interface.

You could modify Jekyll to do this operation.  You wouldn't need to worry
about converting the notebook to HTML, I think, so your job would be a bit
simpler than what RunNotebook does.

-Nathan


On Fri, Apr 18, 2014 at 12:00 PM, Greg Wilson <gvwilson at third-bit.com>wrote:

> The answer to this question may well be, "You shouldn't be trying to do
> that," but here goes anyway:
>
> 1. The websites for Software Carpentry bootcamps are hosted on GitHub,
> which generates them automatically by running a tool called Jekyll
> whenever content is committed to a repository's gh-pages branch.
>
> 2. Jekyll knows how to convert compile Markdown and HTML, but doesn't
> understand IPython Notebooks, so if people have notebooks in their
> bootcamp repository, they have to run nbconvert on their own machine and
> add the generated .md file to the repository.  (Yes, we could do
> something clever with post-commit hooks and continuous integration
> systems, but this seems simpler for our users.)
>
> 3. When nbconvert runs, it creates image files on disk for the plots and
> other code-generated visuals in the notebook.  These image files have
> auto-generated names like 01-numpy_76_0.png, and the Markdown/HTML
> generated by nbconvert links to them.
>
> 4. We can easily add those images to the version control repository as
> well - but if we move cells around in the notebook, nbconvert will give
> them different names the next time it runs.  We can add *those* images
> to version control too, but what do we do about cleaning out the old
> ones?  One suggestion is to 'git rm' all the generated images before
> re-running nbconvert and trust git to detect the new image and infer
> that we meant to 'git mv', but that feels dangerous.
>
> Is there a cleaner solution?  One that we can explain and justify to
> people who are relatively new to both the notebook and version control,
> and is unlikely to go horribly, horribly wrong (which 'git rm' with
> wildcards well could)?
>
> Thanks,
> Greg
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140418/9687e944/attachment.html>