[SciPy-Dev] ways of storing notebook tutorials?

Tue Oct 15 10:21:40 EDT 2019

On Fri, Oct 11, 2019 at 7:36 PM Matthias Geier <matthias.geier at gmail.com>
wrote:

> OK, nbsphinx has already been mentioned, so I don't have to (I'm its
> author, BTW).
>

Cool. Thanks for sharing your expertise here Matthias.

> I just wanted to reinforce what has been said about not storing cell
> outputs in the repository.
> That's a good idea!
>
> Notebooks can be "cleaned" automatically, see:
>
> https://nbsphinx.readthedocs.io/en/0.4.3/usage.html#Using-Notebooks-with-Git
>
> It's also possible to have most of the repository without outputs, but
> only one branch containing executed notebooks. I've written about this
> recently: https://mg.readthedocs.io/git-jupyter.html
>
> Even without output, Jupyter notebooks are a bit annoying in version
> control, because all strings are quoted in the JSON representation and
> there are often plenty of escaped characters.
> To avoid this, something like jupytext is great, but it adds quite
> some of complexity for users.
> Ideally, the native Jupyter notebook format should be changed, but I
> don't know if that's likely to happen. FWIW, I've made a suggestion
> (https://jupyter-format.readthedocs.io/), but I guess that's a topic
> for the far future ...
>

That does look interesting. Hopefully in the not-too-far-future ....

> Regarding what Ralf said:
>
> > (a) have a "download as notebook" link in the html docs
>
> That's easy, just look at the nbsphinx docs, e.g.
> https://nbsphinx.readthedocs.io/en/0.4.3/code-cells.html.
> At the top of the page there is a download link to the notebook on
> Github and a link for launching it on Binder.
> Arbitrary such links can be created, see
> https://nbsphinx.readthedocs.io/en/0.4.3/prolog-and-epilog.html.
>

This does look like the way to go.

Cheers,
Ralf

> cheers,
> Matthias
>
> On Wed, Oct 9, 2019 at 2:26 PM Andrew Nelson <andyfaff at gmail.com> wrote:
> >
> > The overview for nbsphinx is at
> https://nbsphinx.readthedocs.io/en/0.4.3/
> >
> > On Wed, 9 Oct 2019, 12:57 Ralf Gommers, <ralf.gommers at gmail.com> wrote:
> >>
> >>
> >>
> >> On Tue, Oct 8, 2019 at 8:27 AM Andrew Nelson <andyfaff at gmail.com>
> wrote:
> >>>
> >>> Another alternative is to use `jupyter-sphinx` and `nbsphinx` sphinx
> extensions. You can include a link to the unexecuted notebook in RST (e.g.
> https://github.com/refnx/refnx/blob/master/doc/index.rst), and when the
> doc build occurs it'll execute the notebook + output as part of the run.
> >>
> >>
> >> That looks like the closest fit so far.
> https://refnx.readthedocs.io/en/latest/getting_started.html looks nice.
> The parts your setup doesn't seem to do yet are (a) have a "download as
> notebook" link in the html docs, and (b) get rid of notebook output (e.g.
> https://raw.githubusercontent.com/refnx/refnx/master/doc/getting_started.ipynb
> contains the png images) in the repo. Both of those may not be too hard to
> add.
> >>
> >>>
> >>>
> >>> On Tue, 8 Oct 2019, 17:06 Matthew Brett, <matthew.brett at gmail.com>
> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> On Tue, Oct 8, 2019 at 3:55 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >>>> >
> >>>> >
> >>>> >
> >>>> > On Tue, Oct 8, 2019 at 3:22 PM Eric Larson <larson.eric.d at gmail.com>
> wrote:
> >>>> >>
> >>>> >> I've used sphinx-gallery in a number of projects (am now a
> maintainer). Basically you write Python code and it renders to html with
> links to downloadable notebooks in the page, automatic linking/cross-refs
> to function and class defs, nice index pages, etc.
> >>>> >
> >>>> >
> >>>> > That doesn't really answer my question I think. The starting point
> is: notebooks are nice to write a long-form tutorial in, ReST is not. And
> sphinx-gallery is nice for a set of independent small to medium sized
> examples, but not the best fit for a long tutorial on a whole module (e.g.
> http://scipy.github.io/devdocs/tutorial/optimize.html).
> >>>>
> >>>> Right - I think sphinx-gallery is the right choice for these small
> >>>> examples and where you want to develop the tutorial in text.  It
> >>>> solves the ReST output nicely.
> >>>>
> >>>> >>
> >>>> >>
> >>>> >> Eric
> >>>> >>
> >>>> >>
> >>>> >> On Tue, Oct 8, 2019 at 9:20 AM Matthew Brett <
> matthew.brett at gmail.com> wrote:
> >>>> >>>
> >>>> >>> Hi,
> >>>> >>>
> >>>> >>> I use Jupytext all the time now, and I prefer the RMarkdown input
> >>>> >>> format to Jupytext.
> >>>> >>>
> >>>> >>> Jupytext can convert .ipynb files to .Rmd files, and saves
> notebooks
> >>>> >>> as both .ipynb and .Rmd.
> >>>> >>>
> >>>> >>> The workflow is pretty good - after installing Jupytext, you open
> your
> >>>> >>> .Rmd file in the notebook interface, and run anything you want,
> in the
> >>>> >>> standard interface.
> >>>> >>>
> >>>> >>> Then you save, if you need to, which saves to both .ipynb and
> .Rmd.
> >>>> >>>
> >>>> >>> You edit the .Rmd using your nice text editor.
> >>>> >>>
> >>>> >>> Then you can reload the notebook in Jupyter, with just a browser
> >>>> >>> refresh, to see the changes, and to run any extra code you've
> added.
> >>>> >>>
> >>>> >>> I then store the .Rmd files only, in version control, keeping the
> >>>> >>> .ipynb as outputs from the final build step.
> >>>> >
> >>>> >
> >>>> > Thanks, that sounds like a reasonable workflow. Seems like we can
> roundtrip notebook -> plain text format -> notebook.
> >>>>
> >>>> Right - exactly - and you can do that interactively, so you can save
> >>>> from the Jupyter interface, load in a text editor, save in the text
> >>>> editor and just reload the page in Jupyter to get the result.  It will
> >>>> even keep the outputs as long as you haven't done very substantial
> >>>> edits.
> >>>>
> >>>> > We'll still need to integrate that into the built html docs then,
> so the plain text format should be ReST. That doesn't seem supported yet:
> https://jupytext.readthedocs.io/en/latest/using-cli.html#testing-the-round-trip-conversion.
> Or am I missing something?
> >>>>
> >>>> No, that is (currently) a problem that I haven't been working on -
> >>>> because I'm using Markdown output documents like jupyter-book and
> >>>> bookdown. One of my collaborators uses notebooks a lot and I was
> >>>> going to suggest to him that we have a separate site for the notebooks
> >>>> and the main docs, or at least, we build them separately, and perhaps
> >>>> combine them after building to HTML.    Maybe jupyter-book would help
> >>>> here?
> >>
> >>
> >> If we would start over, something like jupyter-book may be a better
> option. I'd prefer to avoid a major overhaul of all tutorials though,
> because there's a *lot* of content written in reST files already. Ideally
> we keep what we have in reST, and add the ability to write new ones in
> notebooks, which could be downloaded standalone but also integrate nicely
> with the existing Sphinx html build.
> >>
> >>>> Chris Holdgraf is the maintainer, and he's very responsive.
> >>
> >>
> >> Thanks for the suggestion. Yes Chris is great, may ask him for advice.
> >>
> >> Cheers,
> >> Ralf
> >>
> >>
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20191015/35b61293/attachment.html>