[SciPy-Dev] ways of storing notebook tutorials?

[Matthew Brett]

Oh - sorry - I should say that I have also used my own nb2plots package:

http://matthew-brett.github.io/nb2plots/

Examples here:

https://matthew-brett.github.io/teaching

In particular - here's a page with the nbplots Sphinx extension:

https://matthew-brett.github.io/teaching/glm_intro.html

This is somewhere between Jupytext (where you can keep using the
notebook as a format for editing) and sphinx-gallery (where you start
and stay in text for editing).

The workflow is:

* Draft up a tutorial in the notebook
* Convert to a RST page with the nb2plots command
* Hand edit the RST a bit to get it into shape
* Continue editing the RST to finish up the document
* The extension builds the RST as a notebook for download, at the
build stage, and adds a link to the notebook in the built page.

See http://matthew-brett.github.io/nb2plots/worked_example.html for a
short example.

Cheers,

Matthew


On Wed, Oct 9, 2019 at 11:57 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Tue, Oct 8, 2019 at 8:27 AM Andrew Nelson <andyfaff at gmail.com> wrote:
>>
>> Another alternative is to use `jupyter-sphinx` and `nbsphinx` sphinx extensions. You can include a link to the unexecuted notebook in RST (e.g. https://github.com/refnx/refnx/blob/master/doc/index.rst), and when the doc build occurs it'll execute the notebook + output as part of the run.
>
>
> That looks like the closest fit so far. https://refnx.readthedocs.io/en/latest/getting_started.html looks nice. The parts your setup doesn't seem to do yet are (a) have a "download as notebook" link in the html docs, and (b) get rid of notebook output (e.g. https://raw.githubusercontent.com/refnx/refnx/master/doc/getting_started.ipynb contains the png images) in the repo. Both of those may not be too hard to add.
>
>>
>>
>> On Tue, 8 Oct 2019, 17:06 Matthew Brett, <matthew.brett at gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> On Tue, Oct 8, 2019 at 3:55 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>> >
>>> >
>>> >
>>> > On Tue, Oct 8, 2019 at 3:22 PM Eric Larson <larson.eric.d at gmail.com> wrote:
>>> >>
>>> >> I've used sphinx-gallery in a number of projects (am now a maintainer). Basically you write Python code and it renders to html with links to downloadable notebooks in the page, automatic linking/cross-refs to function and class defs, nice index pages, etc.
>>> >
>>> >
>>> > That doesn't really answer my question I think. The starting point is: notebooks are nice to write a long-form tutorial in, ReST is not. And sphinx-gallery is nice for a set of independent small to medium sized examples, but not the best fit for a long tutorial on a whole module (e.g. http://scipy.github.io/devdocs/tutorial/optimize.html).
>>>
>>> Right - I think sphinx-gallery is the right choice for these small
>>> examples and where you want to develop the tutorial in text.  It
>>> solves the ReST output nicely.
>>>
>>> >>
>>> >>
>>> >> Eric
>>> >>
>>> >>
>>> >> On Tue, Oct 8, 2019 at 9:20 AM Matthew Brett <matthew.brett at gmail.com> wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> I use Jupytext all the time now, and I prefer the RMarkdown input
>>> >>> format to Jupytext.
>>> >>>
>>> >>> Jupytext can convert .ipynb files to .Rmd files, and saves notebooks
>>> >>> as both .ipynb and .Rmd.
>>> >>>
>>> >>> The workflow is pretty good - after installing Jupytext, you open your
>>> >>> .Rmd file in the notebook interface, and run anything you want, in the
>>> >>> standard interface.
>>> >>>
>>> >>> Then you save, if you need to, which saves to both .ipynb and .Rmd.
>>> >>>
>>> >>> You edit the .Rmd using your nice text editor.
>>> >>>
>>> >>> Then you can reload the notebook in Jupyter, with just a browser
>>> >>> refresh, to see the changes, and to run any extra code you've added.
>>> >>>
>>> >>> I then store the .Rmd files only, in version control, keeping the
>>> >>> .ipynb as outputs from the final build step.
>>> >
>>> >
>>> > Thanks, that sounds like a reasonable workflow. Seems like we can roundtrip notebook -> plain text format -> notebook.
>>>
>>> Right - exactly - and you can do that interactively, so you can save
>>> from the Jupyter interface, load in a text editor, save in the text
>>> editor and just reload the page in Jupyter to get the result.  It will
>>> even keep the outputs as long as you haven't done very substantial
>>> edits.
>>>
>>> > We'll still need to integrate that into the built html docs then, so the plain text format should be ReST. That doesn't seem supported yet: https://jupytext.readthedocs.io/en/latest/using-cli.html#testing-the-round-trip-conversion. Or am I missing something?
>>>
>>> No, that is (currently) a problem that I haven't been working on -
>>> because I'm using Markdown output documents like jupyter-book and
>>> bookdown. One of my collaborators uses notebooks a lot and I was
>>> going to suggest to him that we have a separate site for the notebooks
>>> and the main docs, or at least, we build them separately, and perhaps
>>> combine them after building to HTML.    Maybe jupyter-book would help
>>> here?
>
>
> If we would start over, something like jupyter-book may be a better option. I'd prefer to avoid a major overhaul of all tutorials though, because there's a *lot* of content written in reST files already. Ideally we keep what we have in reST, and add the ability to write new ones in notebooks, which could be downloaded standalone but also integrate nicely with the existing Sphinx html build.
>
>>> Chris Holdgraf is the maintainer, and he's very responsive.
>
>
> Thanks for the suggestion. Yes Chris is great, may ask him for advice.
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev