[IPython-dev] ipython sphinx directive

[John Hunter]

I have been working on a chapter in sphinx, and make liberal use of
the ipython sourecode lexer and matplotlib plot directive Michael D
wrote, with contributions from others.  I am writing in type-along
fashion, so I do a lot of this::

    .. sourcecode:: ipython

       In [193]: X = imread('moonlanding.png')

       In [195]: imshow(X, cmap='gray')
       Out[195]: <matplotlib.image.AxesImage object at 0x97d1acc>


    .. plot::

       import matplotlib.image as mimage
       import matplotlib.pyplot as plt
       X = mimage.imread('moonlanding.png')

       fig = plt.figure()
       ax = fig.add_subplot(111)
       ax.imshow(X, cmap='gray')
       plt.show()

This works great, but as I continue my example, adding new lines of
ipython, I have to always cut and paste the *full* plot code as the
figure is updated, which is tedious and difficult to maintain.  What I
really want is an ipython sphinx mode, preferably one that is pylab
aware, so all I have to do is

    .. ipython_pylab::

           X = imread('moonlanding.png')
           imshow(X, cmap='gray')

and sphinx will pass this to an embedded ipython session, render the
input and output prompts, and insert any figures that are created.
Then I can update the figure in the next block

    .. ipython_pylab::

           clim(0, 300)
           hot()

and get the updated figure in my docs....

I have taken a crack at this in the "ipython_directive" code below.  I
haven't yet tacked the pylab/figure part, being an ipython newbie, and
have just done the basic ipython part.  We have already cracked the
nut of managing pylab figures in the plot directive, so while this
won't be trivial it's not the hardest part I don't think.  What I want
to focus on here is the ipython part.

Since I suspect this will be of interest to a lot of ipython/sphinx
users, I want to get the interface right.  For example, there is a
good argument to allow the sphinx writer to pass complete ipython
blocks in rather than plain python blocks, so the reader of the plain
unrendered rest document would see both the input and output.  The
ipython directive would in this case just grab the input blocks, strip
off the prompts, execute the code, and (possibly) replace the inputs
and outputs in the rendered docs which would provide auto-renumbering.
 There are a lot of things to balance here -- auto-renumbering is nice
but it makes it hard for the doc-writer to refer to specific line
numbers....

For the simplest case, I just handle plain python inputs, and generate
rendered ipython sessions.  Eg, the following sphinx doc::

  =================
  Ipython Directive
  =================

  The ipython directive is a stateful ipython shell for embedding in
  sphinx documents.

  .. ipython::

     x = 2
     x**3

  The state from previous sessions is stored, and standard error is
  trapped

  .. ipython::

     z = x*3   # x is recalled from previous block
     z
     print z   # this doesn't produce output -- how do I capture this?
     q = z[)   # this is a syntax error -- we trap ipy exceptions

  That's it -- next step is supporting pylab with embedded figs!


The sphinx ipython_directive and rendered html are included below and
attached.  For a working directory with everything ready to build for
testing, you can grab:  http://matplotlib.sf.net/ipymode.zip

Suggestions and improvements welcome --  in addition to input on the
interface,  I could use some help from the ipython devs on on how to
configure my Shell to support magics like 'ls' or 'pwd' and 'cd', how
to fix the "print" problem above, integrating pylab plots...

JDH


import sys, os, shutil, imp, warnings, cStringIO, re
try:
    from hashlib import md5
except ImportError:
    from md5 import md5

from docutils.parsers.rst import directives
try:
    # docutils 0.4
    from docutils.parsers.rst.directives.images import align
except ImportError:
    # docutils 0.5
    from docutils.parsers.rst.directives.images import Image
    align = Image.align
import sphinx


sphinx_version = sphinx.__version__.split(".")
# The split is necessary for sphinx beta versions where the string is
# '6b1'
sphinx_version = tuple([int(re.split('[a-z]', x)[0])
                        for x in sphinx_version[:2]])


import IPython



class EmbeddedSphinxShell:
    def __init__(self):

        self.cout = cStringIO.StringIO()

        IPython.Shell.Term.cout = self.cout
        IPython.Shell.Term.cerr = self.cout
        argv = []
        self.user_ns = {}
        self.user_glocal_ns = {}

        self.IP = IPython.ipmaker.make_IPython(
            argv, self.user_ns, self.user_glocal_ns, embedded=True,
            shell_class=IPython.Shell.InteractiveShell,
            rc_override=dict(colors = 'NoColor'))

    def process(self, line):

        self.cout.write('%s%s\n'%(self.IP.outputcache.prompt1, line))
        self.IP.push(line)
        if (self.IP.SyntaxTB.last_syntax_error and
            self.IP.rc.autoedit_syntax):
            print 'OOPS'
            self.IP.edit_syntax_error()


    def astext(self):
        self.cout.seek(0)
        s = self.cout.read()
        self.cout.truncate(0)
        return s



shell = EmbeddedSphinxShell()


def ipython_directive(name, arguments, options, content, lineno,
                   content_offset, block_text, state, state_machine):

    for line in content:
        shell.process(line)

    lines = shell.astext().split('\n')
    if len(lines):
        ipy_lines = ['.. sourcecode:: ipython', '']
        ipy_lines.extend(['    %s'%line for line in lines])
        ipy_lines.append('')

        state_machine.insert_input(
            ipy_lines, state_machine.input_lines.source(0))

def setup(app):
    setup.app = app
    options = {}
    app.add_directive('ipython', ipython_directive, True, (0, 2, 0), **options)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20091106/b7169893/attachment.html>