` tags if you just want it monospaced and not
highlighted.

>
> If I put the notebook safely inside our firewall, would it be
> reasonable to just let 5-10 people use the same server or do you
> foresee problems down the road?

This should be fine, as long as it is acceptable for the 5-10 people to be
running code as the user that launched the server.  Anyone with access to
the server can run arbitrary code on the host as this user.  If this is
acceptable, there should be no problem.  One thing to be careful of is
simultaneously working with a single notebook: You can both be working with
the same file, but saving is destructive.  So if two people open a notebook,
both edit it, and both save it, the later save will clobber the first.

> I've set up a sage server partly for
> that purpose, but having the ipython notebook available could have
> some advantages. For example, we're running a computational mechanics
> seminar where it would be nice to share some simple mpi programs, say
> in the 8-16 processor range. I don't think we can do that in sage yet,
> right?
>
> In the docs, it talks about the python roundtrip not being guaranteed.
> What about a sage round trip capability for the notebooks?
>

I don't know about the Sage notebook's file formats, but it is unlikely.
 That doesn't mean that it won't work at all, or even well, but lossless
roundtrip is generally not feasible between file formats with different
capabilities.

>
> What's the latest recommendation for interactive 3D graphics in the
> notebook, or what direction should I investigate? Probably the most
> straightforward route for me would be to take our existing vtk
> representations that we use with Qt and follow the notebook example on
> the display protocol.  Also, BTW, is there a PySide qt gui example
> somewhere?
>

The IPython notebook currently has no mechanism for interactive graphics.
All the usual interactive tools should work if you are running the notebook
server *locally*, as their windows will pop up as usual.  You can publish
static HTML, SVG, PNG, or txt representations of objects, but not dynamic
(unless you write the dynamic part into the HTML/JS output).

>
> Thanks,
> Chris
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jason-sage at creativetrax.com  Fri Sep 16 18:50:29 2011
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 16 Sep 2011 17:50:29 -0500
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 
Message-ID: <4E73D2B5.7040202@creativetrax.com>

On 9/16/11 3:37 PM, Chris Kees wrote:
> Hi,
>
> I updated my source on the notebook after I read Fernando's post. Nice
> work! Couple of questions:
>
> Is there a way to use reST in the text boxes? It seems like it sort of
> works but doesn't get the ::
>
> If I put the notebook safely inside our firewall, would it be
> reasonable to just let 5-10 people use the same server or do you
> foresee problems down the road? I've set up a sage server partly for
> that purpose, but having the ipython notebook available could have
> some advantages. For example, we're running a computational mechanics
> seminar where it would be nice to share some simple mpi programs, say
> in the 8-16 processor range. I don't think we can do that in sage yet,
> right?

(answering the Sage question here...) What do you mean when you say 
"share some simple mpi programs"?  Do you mean running the mpi programs 
from the command line, or sharing the C code files, or using mpi4py to 
run the programs?  If you can do it in python or on the command line, 
you can likely do it in Sage.  Feel free to email the sage-support list; 
I'm sure we can help you there.

>
> In the docs, it talks about the python roundtrip not being guaranteed.
> What about a sage round trip capability for the notebooks?

I think that might be possible, though it would require ipython being 
able to have "text cells" with arbitrary html, rather than just 
markdown.  The Sage notebook format looks something like:

arbitrary html

{{{
input
///
output
}}}

arbitrary html

{{{
input
///
output
}}}

arbitrary html

etc.

Of course, sage-specific things like interacts wouldn't work.  (However, 
over the summer we've written a version of interacts that is independent 
of Sage and relies instead on the new ipython messaging protocol; it's 
part of the single-cell server project: 
http://utmost.aimath.org/#august-2011.)

Thanks,

Jason

--
Jason Grout

From benjaminrk at gmail.com  Fri Sep 16 19:35:11 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 16 Sep 2011 16:35:11 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: <4E73D2B5.7040202@creativetrax.com>
References: 
	<4E73D2B5.7040202@creativetrax.com>
Message-ID: 

On Fri, Sep 16, 2011 at 15:50, Jason Grout wrote:

> On 9/16/11 3:37 PM, Chris Kees wrote:
> > Hi,
> >
> > I updated my source on the notebook after I read Fernando's post. Nice
> > work! Couple of questions:
> >
> > Is there a way to use reST in the text boxes? It seems like it sort of
> > works but doesn't get the ::
> >
> > If I put the notebook safely inside our firewall, would it be
> > reasonable to just let 5-10 people use the same server or do you
> > foresee problems down the road? I've set up a sage server partly for
> > that purpose, but having the ipython notebook available could have
> > some advantages. For example, we're running a computational mechanics
> > seminar where it would be nice to share some simple mpi programs, say
> > in the 8-16 processor range. I don't think we can do that in sage yet,
> > right?
>
> (answering the Sage question here...) What do you mean when you say
> "share some simple mpi programs"?  Do you mean running the mpi programs
> from the command line, or sharing the C code files, or using mpi4py to
> run the programs?  If you can do it in python or on the command line,
> you can likely do it in Sage.  Feel free to email the sage-support list;
> I'm sure we can help you there.
>
>
>
> >
> > In the docs, it talks about the python roundtrip not being guaranteed.
> > What about a sage round trip capability for the notebooks?
>
> I think that might be possible, though it would require ipython being
> able to have "text cells" with arbitrary html, rather than just
> markdown.  The Sage notebook format looks something like:
>

since (almost) arbitrary html is valid markdown, I don't expect a problem
there.  The first notebook draft had html text cells, but we never found any
HTML we wanted to use that didn't work as-is in markdown, so we disabled it
to prevent unnecessary clutter.  If there are real HTML use cases for which
the markdown transforms html inappropriately, we can easily turn it back on.

>
> arbitrary html
>
> {{{
> input
> ///
> output
> }}}
>
> arbitrary html
>
> {{{
> input
> ///
> output
> }}}
>
> arbitrary html
>
> etc.
>

That seems simple enough, though after playing with a few sage worksheets,
it looks like if you make an HTML cell, it is indistinguishable from a code
cell in the exported file.  Presumably, IPython would only want the
Python/Sage cells as code cells, whereas all the latex, text, and html cells
should come out as text.  If Sage doesn't distinguish between these in the
file-format, then I'm not sure what we should do.  Detecting the language
based purely on content will certainly not be acceptable.

But the fact that Sage can have HTML that is part of a cell, as well as HTML
that is *not* part of a cell makes it clear that a roundtrip will not result
in an identical notebook.  It may have the same content, but it will have
changed structure at least partly.  This is what we mean when we talk about
lacking roundtrip support - not that you can't import to/export from a
format, but that doing so is not guaranteed to preserve the entire document
structure.

>
> Of course, sage-specific things like interacts wouldn't work.  (However,
> over the summer we've written a version of interacts that is independent
> of Sage and relies instead on the new ipython messaging protocol; it's
> part of the single-cell server project:
> http://utmost.aimath.org/#august-2011.)
>

Very cool!

>
> Thanks,
>
> Jason
>
> --
> Jason Grout
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From cekees at gmail.com  Fri Sep 16 23:08:32 2011
From: cekees at gmail.com (Chris Kees)
Date: Fri, 16 Sep 2011 22:08:32 -0500
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

Message-ID: 

On Fri, Sep 16, 2011 at 4:14 PM, MinRK  wrote:
>
>
> On Fri, Sep 16, 2011 at 13:37, Chris Kees  wrote:
>>
>> Hi,
>>
>> I updated my source on the notebook after I read Fernando's post. Nice
>> work! Couple of questions:
>>
>> Is there a way to use reST in the text boxes? It seems like it sort of
>> works but doesn't get the ::
>
> No, it uses markdown, which is largely similar, if simpler, and much better
> suited for
> use in a browser. ?Indent code blocks by four spaces, and it should
> behave nicely. ?Or use `` tags if you just want it monospaced and not
> highlighted.
>

I noticed that github allows you to choose between several different
languages for wiki input so I thought you might have something like
that in mind.  I've been using reST for docstrings, sphinx docs, and
github so I selfishly would prefer not to have to learn another
simplified markup language.

>>
>> If I put the notebook safely inside our firewall, would it be
>> reasonable to just let 5-10 people use the same server or do you
>> foresee problems down the road?
>
> This should be fine, as long as it is acceptable for the 5-10 people to be
> running code as the user that launched the server. ?Anyone with access to
> the server can run arbitrary code on the host as this user. ?If this is
> acceptable, there should be no problem. ?One thing to be careful of is
> simultaneously working with a single notebook: You can both be working with
> the same file, but saving is destructive. ?So if two people open a notebook,
> both edit it, and both save it, the later save will clobber the first.
>

I could set up a special user or virtual machine like is done for sage
servers, and then we could just use a convention that your always save
worksheets in notebooks ending in your initials. One flaw in that plan
is that the current notebook doesn't appear to have a "duplicate" or
"save as" capability, but I'm guessing that would be very easy to add.
 I like the integrated version control idea that Anne Archibald
mentioned in her blog.

>>
>> I've set up a sage server partly for
>> that purpose, but having the ipython notebook available could have
>> some advantages. For example, we're running a computational mechanics
>> seminar where it would be nice to share some simple mpi programs, say
>> in the 8-16 processor range. I don't think we can do that in sage yet,
>> right?
>>
>> In the docs, it talks about the python roundtrip not being guaranteed.
>> What about a sage round trip capability for the notebooks?
>
> I don't know about the Sage notebook's file formats, but it is unlikely.
> ?That doesn't mean that it won't work at all, or even well, but lossless
> roundtrip is generally not feasible between file formats with different
> capabilities.
>

I understand about the perfect lossless roundtrip being impossible in
general. I'd be happy with a convert script that goes back and forth
with warnings or comments (sort of like a failed merge). As long as
the python part works and most of the text makes it on the round trip
it seems valuable to me.  One can invest quite a bit of effort into
these notebooks, and I like the idea of being able to convert them to
high quality python code or sage with minimal effort.

>>
>> What's the latest recommendation for interactive 3D graphics in the
>> notebook, or what direction should I investigate? Probably the most
>> straightforward route for me would be to take our existing vtk
>> representations that we use with Qt and follow the notebook example on
>> the display protocol. ?Also, BTW, is there a PySide qt gui example
>> somewhere?
>
> The IPython notebook currently has no mechanism for interactive graphics.
> All the usual interactive tools should work if you are running the notebook
> server *locally*, as their windows will pop up as usual. ?You can publish
> static HTML, SVG, PNG, or txt representations of objects, but not dynamic
> (unless you write the dynamic part into the HTML/JS output).
>

OK, that gives a path forward.  Most of our graphics is done in vtk so
I asked the paraview list whether the JS support from the ParaViewWeb
project is available from python yet
(http://www.vtk.org/Wiki/ParaViewWeb).

>>
>> Thanks,
>> Chris
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

From jason-sage at creativetrax.com  Fri Sep 16 23:20:39 2011
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 16 Sep 2011 22:20:39 -0500
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 
	<4E73D2B5.7040202@creativetrax.com>

Message-ID: <4E741207.8090208@creativetrax.com>

On 9/16/11 6:35 PM, MinRK wrote:
>
>
> On Fri, Sep 16, 2011 at 15:50, Jason Grout  > wrote:
>
>     On 9/16/11 3:37 PM, Chris Kees wrote:
>      > Hi,
>      >
>      > I updated my source on the notebook after I read Fernando's post.
>     Nice
>      > work! Couple of questions:
>      >
>      > Is there a way to use reST in the text boxes? It seems like it
>     sort of
>      > works but doesn't get the ::
>      >
>      > If I put the notebook safely inside our firewall, would it be
>      > reasonable to just let 5-10 people use the same server or do you
>      > foresee problems down the road? I've set up a sage server partly for
>      > that purpose, but having the ipython notebook available could have
>      > some advantages. For example, we're running a computational mechanics
>      > seminar where it would be nice to share some simple mpi programs, say
>      > in the 8-16 processor range. I don't think we can do that in sage
>     yet,
>      > right?
>
>     (answering the Sage question here...) What do you mean when you say
>     "share some simple mpi programs"?  Do you mean running the mpi programs
>     from the command line, or sharing the C code files, or using mpi4py to
>     run the programs?  If you can do it in python or on the command line,
>     you can likely do it in Sage.  Feel free to email the sage-support list;
>     I'm sure we can help you there.
>
>
>
>      >
>      > In the docs, it talks about the python roundtrip not being
>     guaranteed.
>      > What about a sage round trip capability for the notebooks?
>
>     I think that might be possible, though it would require ipython being
>     able to have "text cells" with arbitrary html, rather than just
>     markdown.  The Sage notebook format looks something like:
>
>
> since (almost) arbitrary html is valid markdown, I don't expect a
> problem there.  The first notebook draft had html text cells, but we
> never found any HTML we wanted to use that didn't work as-is in
> markdown, so we disabled it to prevent unnecessary clutter.  If there
> are real HTML use cases for which the markdown transforms html
> inappropriately, we can easily turn it back on.

Good point about streamlining things.  We've found it useful to be able 
to put arbitrary html in (think of a worksheet as an html page with 
cells on it---it's a more fluid document than just text, worksheet, 
text, worksheet, etc.).  For example, one person asked about putting two 
cells side-by-side, and that was pretty easy to do by writing the 
necessary table commands before and after the cells.  I'm curious: can 
you do something like that (embed a  before a compute cell and a 
 after a compute cell to put the compute cell in a table)?  Or are 
your text cells self-contained snippets in the document?

I like the idea of a worksheet simply being a webpage (with all the 
possible html capabilities), with special cells that evaluate code 
somewhere in the page, or insert WYSIWYG editors, or do interactive 
graphics, etc.

>
>
>     arbitrary html
>
>     {{{
>     input
>     ///
>     output
>     }}}
>
>     arbitrary html
>
>     {{{
>     input
>     ///
>     output
>     }}}
>
>     arbitrary html
>
>     etc.
>
>
> That seems simple enough, though after playing with a few sage
> worksheets, it looks like if you make an HTML cell, it is
> indistinguishable from a code cell in the exported file.  Presumably,
> IPython would only want the Python/Sage cells as code cells, whereas all
> the latex, text, and html cells should come out as text.  If Sage
> doesn't distinguish between these in the file-format, then I'm not sure
> what we should do.  Detecting the language based purely on content will
> certainly not be acceptable.
>

I think you may be confusing %html cells (which are just code cells 
whose output is spliced into the html document, exactly as if we had 
called the html() command, and similar to what would happen if your 
output was a text/html mimetype and spliced into the document) and what 
I'm calling arbitrary html above.  Anything that is typed into the code 
cell is considered a code cell, including %html cells, etc.  To edit the 
arbitrary html that I'm talking about above, shift-click on the blue 
insert-cell bar to open up what we call a "text cell" (as compared to a 
code cell).  A TinyMCE WYSIWYG html editor will pop up.

> But the fact that Sage can have HTML that is part of a cell, as well as
> HTML that is *not* part of a cell makes it clear that a roundtrip will
> not result in an identical notebook.  It may have the same content, but
> it will have changed structure at least partly.  This is what we mean
> when we talk about lacking roundtrip support - not that you can't import
> to/export from a format, but that doing so is not guaranteed to preserve
> the entire document structure.

Yes, of course.  For example, interacts wouldn't work very well.  But I 
think a more modest goal of having the inputs round-trip and the html 
between code cells round-trip might be achievable.  What about files 
attached to the notebooks?  In Sage, we can attach lots of files to a 
worksheet in a DATA directory.  Also, the graphics are not inlined png 
(as I notice the ipython notebook has), but are stored in cell-specific 
output directories.

Jason

From benjaminrk at gmail.com  Fri Sep 16 23:41:23 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 16 Sep 2011 20:41:23 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

Message-ID: 

On Fri, Sep 16, 2011 at 20:08, Chris Kees  wrote:

> On Fri, Sep 16, 2011 at 4:14 PM, MinRK  wrote:
> >
> >
> > On Fri, Sep 16, 2011 at 13:37, Chris Kees  wrote:
> >>
> >> Hi,
> >>
> >> I updated my source on the notebook after I read Fernando's post. Nice
> >> work! Couple of questions:
> >>
> >> Is there a way to use reST in the text boxes? It seems like it sort of
> >> works but doesn't get the ::
> >
> > No, it uses markdown, which is largely similar, if simpler, and much
> better
> > suited for
> > use in a browser.  Indent code blocks by four spaces, and it should
> > behave nicely.  Or use `` tags if you just want it monospaced and
> not
> > highlighted.
> >
>
> I noticed that github allows you to choose between several different
> languages for wiki input so I thought you might have something like
> that in mind.  I've been using reST for docstrings, sphinx docs, and
> github so I selfishly would prefer not to have to learn another
> simplified markup language.
>

That makes sense, and I expect we will get this request often.  The problem
is reST is really a horrible language
for in-browser cell markup (as opposed to full document markup, for which it
is designed), and for many simple cases
writing as if it were reST will result in the same output in markdown.

>
> >>
> >> If I put the notebook safely inside our firewall, would it be
> >> reasonable to just let 5-10 people use the same server or do you
> >> foresee problems down the road?
> >
> > This should be fine, as long as it is acceptable for the 5-10 people to
> be
> > running code as the user that launched the server.  Anyone with access to
> > the server can run arbitrary code on the host as this user.  If this is
> > acceptable, there should be no problem.  One thing to be careful of is
> > simultaneously working with a single notebook: You can both be working
> with
> > the same file, but saving is destructive.  So if two people open a
> notebook,
> > both edit it, and both save it, the later save will clobber the first.
> >
>
> I could set up a special user or virtual machine like is done for sage
> servers, and then we could just use a convention that your always save
> worksheets in notebooks ending in your initials. One flaw in that plan
> is that the current notebook doesn't appear to have a "duplicate" or
> "save as" capability, but I'm guessing that would be very easy to add.
>  I like the integrated version control idea that Anne Archibald
> mentioned in her blog.
>

Yes, integrated version control is one of the very first features we got
excited about,
long before a single line of code was written, and will hopefully support
soon.

>
> >>
> >> I've set up a sage server partly for
> >> that purpose, but having the ipython notebook available could have
> >> some advantages. For example, we're running a computational mechanics
> >> seminar where it would be nice to share some simple mpi programs, say
> >> in the 8-16 processor range. I don't think we can do that in sage yet,
> >> right?
> >>
> >> In the docs, it talks about the python roundtrip not being guaranteed.
> >> What about a sage round trip capability for the notebooks?
> >
> > I don't know about the Sage notebook's file formats, but it is unlikely.
> >  That doesn't mean that it won't work at all, or even well, but lossless
> > roundtrip is generally not feasible between file formats with different
> > capabilities.
> >
>
> I understand about the perfect lossless roundtrip being impossible in
> general. I'd be happy with a convert script that goes back and forth
> with warnings or comments (sort of like a failed merge). As long as
> the python part works and most of the text makes it on the round trip
> it seems valuable to me.  One can invest quite a bit of effort into
> these notebooks, and I like the idea of being able to convert them to
> high quality python code or sage with minimal effort.
>

Yes, one (or few) time export or import should not be a problem.  Just
don't expect to be able to reliably be going back and forth between
environments with continued edits, which clean roundtrip would get you.

>
> >>
> >> What's the latest recommendation for interactive 3D graphics in the
> >> notebook, or what direction should I investigate? Probably the most
> >> straightforward route for me would be to take our existing vtk
> >> representations that we use with Qt and follow the notebook example on
> >> the display protocol.  Also, BTW, is there a PySide qt gui example
> >> somewhere?
> >
> > The IPython notebook currently has no mechanism for interactive graphics.
> > All the usual interactive tools should work if you are running the
> notebook
> > server *locally*, as their windows will pop up as usual.  You can publish
> > static HTML, SVG, PNG, or txt representations of objects, but not dynamic
> > (unless you write the dynamic part into the HTML/JS output).
> >
>
> OK, that gives a path forward.  Most of our graphics is done in vtk so
> I asked the paraview list whether the JS support from the ParaViewWeb
> project is available from python yet
> (http://www.vtk.org/Wiki/ParaViewWeb).
>
> >>
> >> Thanks,
> >> Chris
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From gael.varoquaux at normalesup.org  Fri Sep 16 23:44:15 2011
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 17 Sep 2011 05:44:15 +0200
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

Message-ID: <20110917034415.GE13494@phare.normalesup.org>

On Fri, Sep 16, 2011 at 08:41:23PM -0700, MinRK wrote:
>      I noticed that github allows you to choose between several different
>      languages for wiki input so I thought you might have something like
>      that in mind. ?I've been using reST for docstrings, sphinx docs, and
>      github so I selfishly would prefer not to have to learn another
>      simplified markup language.

>    That makes sense, and I expect we will get this request often. ?The
>    problem is reST is really a horrible language
>    for in-browser cell markup (as opposed to full document markup, for which
>    it is designed),

Actually, reST was originally designed to write docstrings.

It seems to me wrong to go with any other markup language, given that the
standard in the Python world in reST. It is hard enough to agree on
standards, let us rejoice that there is one.

Gael

From benjaminrk at gmail.com  Sat Sep 17 00:17:05 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 16 Sep 2011 21:17:05 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110917034415.GE13494@phare.normalesup.org>
References: 

	<20110917034415.GE13494@phare.normalesup.org>
Message-ID: 

On Fri, Sep 16, 2011 at 20:44, Gael Varoquaux  wrote:

> On Fri, Sep 16, 2011 at 08:41:23PM -0700, MinRK wrote:
> >      I noticed that github allows you to choose between several different
> >      languages for wiki input so I thought you might have something like
> >      that in mind.  I've been using reST for docstrings, sphinx docs, and
> >      github so I selfishly would prefer not to have to learn another
> >      simplified markup language.
>
> >    That makes sense, and I expect we will get this request often.  The
> >    problem is reST is really a horrible language
> >    for in-browser cell markup (as opposed to full document markup, for
> which
> >    it is designed),
>
> Actually, reST was originally designed to write docstrings.

Ah, I didn't know that, given that its design is so much better suited to
large documents than small isolated text.

>
> It seems to me wrong to go with any other markup language, given that the
> standard in the Python world in reST. It is hard enough to agree on
> standards, let us rejoice that there is one.
>

We thought about this quite a bit, and concluded that markdown was just a
better
fit, despite the Python-ness of reST.  reST is just a very bad tool for
interactive, light text markup,
but if you can point me to a good javascript reST renderer, we can
reconsider.

> Gael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ellisonbg at gmail.com  Sat Sep 17 01:16:46 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 16 Sep 2011 22:16:46 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

	<20110917034415.GE13494@phare.normalesup.org>

Message-ID: 

I can outline our plans for supporting reST in the notebook.

* The problem with reST is that it can only render entire documents
and not parts of documents.  Because of this, it can't be used to
render a single reST text cell in any way that behaves as people
expect.  Examples are section headings and references, which are
contextual and require the entire document to make sense of.  A more
formal way of putting this is:

rest_render(cell[0]) + rest_render(cell[1]) != rest_render(cell[0] + cell[1])

Markdown does not suffer from these design limitations and this is why
it is our default for rich text cells in the notebook.

* Obviously reST is very important to the python community and we want
to support reST.  Here is how we plan on doing this.  The problem is
in the rendering of individual cells.  We will implement reST text
cells that simply don't render to HTML in the notebook as the Markdown
cells do.  They will remain reST source and be stored in that form in
the notebook format.

* Notebooks with reST text cells can be exported to reST easily and
then rendered to HTML/LaTeX as desired.

* reST document will be able to be imported into the notebook once
this is implemented as well.

The most important point in all of this is that the problem is reST
itself, not the IPython notebook.  I actually spent a couple of days
trying to hack reST support into the notebook using various tricks.
At the end of it all my conclusion was a somewhat depressing "wow,
reST is simply not designed for this usage case, but Markdown is a
perfect fit"

Cheers,

Brian

On Fri, Sep 16, 2011 at 9:17 PM, MinRK  wrote:
>
>
> On Fri, Sep 16, 2011 at 20:44, Gael Varoquaux
>  wrote:
>>
>> On Fri, Sep 16, 2011 at 08:41:23PM -0700, MinRK wrote:
>> > ? ? ?I noticed that github allows you to choose between several
>> > different
>> > ? ? ?languages for wiki input so I thought you might have something like
>> > ? ? ?that in mind. ?I've been using reST for docstrings, sphinx docs,
>> > and
>> > ? ? ?github so I selfishly would prefer not to have to learn another
>> > ? ? ?simplified markup language.
>>
>> > ? ?That makes sense, and I expect we will get this request often. ?The
>> > ? ?problem is reST is really a horrible language
>> > ? ?for in-browser cell markup (as opposed to full document markup, for
>> > which
>> > ? ?it is designed),
>>
>> Actually, reST was originally designed to write docstrings.
>
> Ah, I didn't know that, given that its design is so much better suited to
> large documents than small isolated text.
>
>>
>> It seems to me wrong to go with any other markup language, given that the
>> standard in the Python world in reST. It is hard enough to agree on
>> standards, let us rejoice that there is one.
>
> We thought about this quite a bit, and concluded that markdown was just a
> better
> fit, despite the Python-ness of reST. ?reST is just a very bad tool for
> interactive, light text markup,
> but if you can point me to a good javascript reST renderer, we can
> reconsider.
>
>>
>> Gael
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com

From ellisonbg at gmail.com  Sat Sep 17 01:25:08 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 16 Sep 2011 22:25:08 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

Message-ID: 

On Fri, Sep 16, 2011 at 8:08 PM, Chris Kees  wrote:
> On Fri, Sep 16, 2011 at 4:14 PM, MinRK  wrote:
>>
>>
>> On Fri, Sep 16, 2011 at 13:37, Chris Kees  wrote:
>>>
>>> Hi,
>>>
>>> I updated my source on the notebook after I read Fernando's post. Nice
>>> work! Couple of questions:
>>>
>>> Is there a way to use reST in the text boxes? It seems like it sort of
>>> works but doesn't get the ::
>>
>> No, it uses markdown, which is largely similar, if simpler, and much better
>> suited for
>> use in a browser. ?Indent code blocks by four spaces, and it should
>> behave nicely. ?Or use `` tags if you just want it monospaced and not
>> highlighted.
>>
>
> I noticed that github allows you to choose between several different
> languages for wiki input so I thought you might have something like
> that in mind. ?I've been using reST for docstrings, sphinx docs, and
> github so I selfishly would prefer not to have to learn another
> simplified markup language.
>
>>>
>>> If I put the notebook safely inside our firewall, would it be
>>> reasonable to just let 5-10 people use the same server or do you
>>> foresee problems down the road?
>>
>> This should be fine, as long as it is acceptable for the 5-10 people to be
>> running code as the user that launched the server. ?Anyone with access to
>> the server can run arbitrary code on the host as this user. ?If this is
>> acceptable, there should be no problem. ?One thing to be careful of is
>> simultaneously working with a single notebook: You can both be working with
>> the same file, but saving is destructive. ?So if two people open a notebook,
>> both edit it, and both save it, the later save will clobber the first.
>>
>
> I could set up a special user or virtual machine like is done for sage
> servers, and then we could just use a convention that your always save
> worksheets in notebooks ending in your initials. One flaw in that plan
> is that the current notebook doesn't appear to have a "duplicate" or
> "save as" capability, but I'm guessing that would be very easy to add.
> ?I like the integrated version control idea that Anne Archibald
> mentioned in her blog.

A notebook "Clone" capability is on our list as is version control
integration.  But I should mention that our model for notebooks is
quite different than that of Sage.  The notebook server stores the
notebook documents as JSON text files in the cwd of the notebook.
There is no "notebook database" involved and we encourage storing the
notebook files alongside the rest of your project files.  What this
means is that all you have to do to get version control for notebooks
is do "git init" in the directory containing the notebooks.
Obviously, we want more integration in the future, but this simple
model works today.

>>>
>>> I've set up a sage server partly for
>>> that purpose, but having the ipython notebook available could have
>>> some advantages. For example, we're running a computational mechanics
>>> seminar where it would be nice to share some simple mpi programs, say
>>> in the 8-16 processor range. I don't think we can do that in sage yet,
>>> right?
>>>
>>> In the docs, it talks about the python roundtrip not being guaranteed.
>>> What about a sage round trip capability for the notebooks?
>>
>> I don't know about the Sage notebook's file formats, but it is unlikely.
>> ?That doesn't mean that it won't work at all, or even well, but lossless
>> roundtrip is generally not feasible between file formats with different
>> capabilities.
>>
>
> I understand about the perfect lossless roundtrip being impossible in
> general. I'd be happy with a convert script that goes back and forth
> with warnings or comments (sort of like a failed merge). As long as
> the python part works and most of the text makes it on the round trip
> it seems valuable to me. ?One can invest quite a bit of effort into
> these notebooks, and I like the idea of being able to convert them to
> high quality python code or sage with minimal effort.

It should be possible to import/export IPython notebooks to Sage
notebooks, at least at a simple level.  I don't have time to work on
this right now, but I can point others in the right direction if they
want to tackle it.

>>>
>>> What's the latest recommendation for interactive 3D graphics in the
>>> notebook, or what direction should I investigate? Probably the most
>>> straightforward route for me would be to take our existing vtk
>>> representations that we use with Qt and follow the notebook example on
>>> the display protocol. ?Also, BTW, is there a PySide qt gui example
>>> somewhere?
>>
>> The IPython notebook currently has no mechanism for interactive graphics.
>> All the usual interactive tools should work if you are running the notebook
>> server *locally*, as their windows will pop up as usual. ?You can publish
>> static HTML, SVG, PNG, or txt representations of objects, but not dynamic
>> (unless you write the dynamic part into the HTML/JS output).
>>
>
> OK, that gives a path forward. ?Most of our graphics is done in vtk so
> I asked the paraview list whether the JS support from the ParaViewWeb
> project is available from python yet
> (http://www.vtk.org/Wiki/ParaViewWeb).

Sounds good.

Cheers,

Brian

>>>
>>> Thanks,
>>> Chris
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com

From satra at mit.edu  Sat Sep 17 01:34:14 2011
From: satra at mit.edu (Satrajit Ghosh)
Date: Sat, 17 Sep 2011 01:34:14 -0400
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

	<20110917034415.GE13494@phare.normalesup.org>

Message-ID: 

i don't know if you've seen this before, but might be useful:

http://rst.ninjs.org/

cheers,

satra

On Sat, Sep 17, 2011 at 1:16 AM, Brian Granger  wrote:

> I can outline our plans for supporting reST in the notebook.
>
> * The problem with reST is that it can only render entire documents
> and not parts of documents.  Because of this, it can't be used to
> render a single reST text cell in any way that behaves as people
> expect.  Examples are section headings and references, which are
> contextual and require the entire document to make sense of.  A more
> formal way of putting this is:
>
> rest_render(cell[0]) + rest_render(cell[1]) != rest_render(cell[0] +
> cell[1])
>
> Markdown does not suffer from these design limitations and this is why
> it is our default for rich text cells in the notebook.
>
> * Obviously reST is very important to the python community and we want
> to support reST.  Here is how we plan on doing this.  The problem is
> in the rendering of individual cells.  We will implement reST text
> cells that simply don't render to HTML in the notebook as the Markdown
> cells do.  They will remain reST source and be stored in that form in
> the notebook format.
>
> * Notebooks with reST text cells can be exported to reST easily and
> then rendered to HTML/LaTeX as desired.
>
> * reST document will be able to be imported into the notebook once
> this is implemented as well.
>
> The most important point in all of this is that the problem is reST
> itself, not the IPython notebook.  I actually spent a couple of days
> trying to hack reST support into the notebook using various tricks.
> At the end of it all my conclusion was a somewhat depressing "wow,
> reST is simply not designed for this usage case, but Markdown is a
> perfect fit"
>
> Cheers,
>
> Brian
>
> On Fri, Sep 16, 2011 at 9:17 PM, MinRK  wrote:
> >
> >
> > On Fri, Sep 16, 2011 at 20:44, Gael Varoquaux
> >  wrote:
> >>
> >> On Fri, Sep 16, 2011 at 08:41:23PM -0700, MinRK wrote:
> >> >      I noticed that github allows you to choose between several
> >> > different
> >> >      languages for wiki input so I thought you might have something
> like
> >> >      that in mind.  I've been using reST for docstrings, sphinx docs,
> >> > and
> >> >      github so I selfishly would prefer not to have to learn another
> >> >      simplified markup language.
> >>
> >> >    That makes sense, and I expect we will get this request often.  The
> >> >    problem is reST is really a horrible language
> >> >    for in-browser cell markup (as opposed to full document markup, for
> >> > which
> >> >    it is designed),
> >>
> >> Actually, reST was originally designed to write docstrings.
> >
> > Ah, I didn't know that, given that its design is so much better suited to
> > large documents than small isolated text.
> >
> >>
> >> It seems to me wrong to go with any other markup language, given that
> the
> >> standard in the Python world in reST. It is hard enough to agree on
> >> standards, let us rejoice that there is one.
> >
> > We thought about this quite a bit, and concluded that markdown was just a
> > better
> > fit, despite the Python-ness of reST.  reST is just a very bad tool for
> > interactive, light text markup,
> > but if you can point me to a good javascript reST renderer, we can
> > reconsider.
> >
> >>
> >> Gael
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jason-sage at creativetrax.com  Sat Sep 17 09:27:17 2011
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 17 Sep 2011 08:27:17 -0500
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 		

Message-ID: <4E74A035.2040709@creativetrax.com>

On 9/17/11 12:25 AM, Brian Granger wrote:
> A notebook "Clone" capability is on our list as is version control
> integration.  But I should mention that our model for notebooks is
> quite different than that of Sage.  The notebook server stores the
> notebook documents as JSON text files in the cwd of the notebook.
> There is no "notebook database" involved and we encourage storing the
> notebook files alongside the rest of your project files.  What this
> means is that all you have to do to get version control for notebooks
> is do "git init" in the directory containing the notebooks.
> Obviously, we want more integration in the future, but this simple
> model works today.

So at this stage, it sounds like:

* if a person is personally working on some files, the ipython notebook 
is a lightweight web interface

* if a server is being set up for lots of people, the sage notebook is a 
much heavier-weight web interface

and small groups is where the lines are blurred.

This also answers my questions about attaching files to worksheets, etc. 
  It sounds like an pynb file should be thought of as basically a .py 
file (i.e., just some code to be run), with output possibly also saved 
and other explanatory text.

Whereas in Sage, a worksheet .sws file is more like the entire directory 
containing code to be executed, supporting files, etc.

-Jason

From robert.kern at gmail.com  Sat Sep 17 19:52:09 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 17 Sep 2011 18:52:09 -0500
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110917034415.GE13494@phare.normalesup.org>
References: 

	<20110917034415.GE13494@phare.normalesup.org>
Message-ID: 

On 9/16/11 10:44 PM, Gael Varoquaux wrote:
> On Fri, Sep 16, 2011 at 08:41:23PM -0700, MinRK wrote:
>>       I noticed that github allows you to choose between several different
>>       languages for wiki input so I thought you might have something like
>>       that in mind.  I've been using reST for docstrings, sphinx docs, and
>>       github so I selfishly would prefer not to have to learn another
>>       simplified markup language.
>
>>     That makes sense, and I expect we will get this request often.  The
>>     problem is reST is really a horrible language
>>     for in-browser cell markup (as opposed to full document markup, for which
>>     it is designed),
>
> Actually, reST was originally designed to write docstrings.

Nope, it was designed for writing whole web pages in Zope.

   http://old.zope.org/Products/ZEO/Wiki/StructuredText

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

From gael.varoquaux at normalesup.org  Sat Sep 17 20:54:56 2011
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 18 Sep 2011 02:54:56 +0200
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

	<20110917034415.GE13494@phare.normalesup.org>

Message-ID: <20110918005455.GB16141@phare.normalesup.org>

On Sat, Sep 17, 2011 at 06:52:09PM -0500, Robert Kern wrote:
> > Actually, reST was originally designed to write docstrings.

> Nope, it was designed for writing whole web pages in Zope.

>    http://old.zope.org/Products/ZEO/Wiki/StructuredText

My bad.

G

From cekees at gmail.com  Mon Sep 19 00:09:55 2011
From: cekees at gmail.com (Chris Kees)
Date: Sun, 18 Sep 2011 23:09:55 -0500
Subject: [IPython-dev] minimal mpi4py notebook example
Message-ID: 

Could somebody point me to documentation on a simple example of using
mip4py from the notebook? I want to do something like

$ipcluster ...
$ipython notebook --pylab ...

and then in the notebook do a simple mpi hello world. I can start the
cluster and do it from a standard ipython client but can't seem to set
up the notebook correctly to use the engines.

Thanks,
Chris

From fperez.net at gmail.com  Mon Sep 19 16:15:19 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 19 Sep 2011 13:15:19 -0700
Subject: [IPython-dev] minimal mpi4py notebook example
In-Reply-To: 
References: 
Message-ID: 

Hi Chris,

On Sun, Sep 18, 2011 at 9:09 PM, Chris Kees  wrote:
> Could somebody point me to documentation on a simple example of using
> mip4py from the notebook? I want to do something like
>
> $ipcluster ...
> $ipython notebook --pylab ...
>
> and then in the notebook do a simple mpi hello world. I can start the
> cluster and do it from a standard ipython client but can't seem to set
> up the notebook correctly to use the engines.

Just so we understand better precisely what you'd like to see
illustrated: do you want the process running the notebook to be itself
one of the members of your MPI world (say the rank-0 node)?  Or do you
want the MPI world to start N nodes to which you then attach from a
notebook session and tell them to run various bits of code from within
the notebook?

With a bit of clarity about what you want to do, we should be able to
help you out (and will make that notebook an example in the docs).

Cheers,

f

From fperez.net at gmail.com  Mon Sep 19 22:22:45 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 19 Sep 2011 19:22:45 -0700
Subject: [IPython-dev] minimal mpi4py notebook example
In-Reply-To: 
References: 

Message-ID: 

Hi Chris,

responding on-list so this feedback benefits other users as well...

On Mon, Sep 19, 2011 at 1:41 PM, Chris Kees  wrote:
>
> I think I can do it either way, but having the notebook be the rank-0 node is probably the most natural for me, since my experience is primarily with SIMD programming. Image files are composited and written to disk by the rank-0 process, so I think they would automatically show up in the notebook (or be easy to fit into the display protocol). ? On the other hand, I can encapsulate the entire parallel program in a function call, so my guess is I can do it either way. I'm not sure it's useful, but below I've included the entire program ?that needs to run either a) in a python interpreter that has already called MPI init or b) in one that was started with mpiexec.

Well, after writing my email I realized that it's not so easy right
now to have your notebook kernel be one of the mpi ones.  There's no
fundamental problem with the architecture, it's just a bit of missing
code: right now kernels are started by the notebook server process,
which then opens up a web connection on one side that maps to that
kernel on the other side.  We haven't added the ability to create a
web notebook connected to an *existing kernel* based on its port
information (like you can wire a qt console to an existing kernel).
So the mpi processes could start a kernel all they want, there's no
way right now to control that kernel via a notebook (though you could
control it via a qtconsole, if you print/save the port data
somewhere).

If, with more experience, we find this to really be a necessary use
case we can certainly implement things: all the architecture is
designed so it could work, it's just a little bit of missing glue in
places.

But in the meantime, I've written up (attached) two notebooks for
illustrating simple MPI use.  The first is really the kind of code
you'd run normally in a terminal, but I put it in a notebook to show
that it can be a way for your users to start things if they are a bit
terminal-phobic.  They can use that notebook as a poor-man's terminal,
and the 'busy' indicator will be on as long as the cluster is active.
They can just stop it with the 'kernel interrupt' button, which
effectively sends Ctrl-C.  The same goal can be obtained with a screen
session into the server (more robust than a simple ssh login), but
this makes it nicely sit in your open notebooks next to the real work.

The 'real work' is in the 2nd notebook, which just illustrates some
basic communication with the engines and uses %autopx with the
convenience of whole-cell editing to send them a little bit of MPI
code (taken from the mpi4py docs).

Let me know if this helps, and if so, we can put it in the docs.

Cheers,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: parallel_ipcluster.ipynb
Type: application/octet-stream
Size: 3344 bytes
Desc: not available
URL: 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: parallel_mpi.ipynb
Type: application/octet-stream
Size: 6513 bytes
Desc: not available
URL: 

From hans_meine at gmx.net  Tue Sep 20 09:03:05 2011
From: hans_meine at gmx.net (Hans Meine)
Date: Tue, 20 Sep 2011 15:03:05 +0200
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110918005455.GB16141@phare.normalesup.org>
References: 

	<20110918005455.GB16141@phare.normalesup.org>
Message-ID: <201109201503.05203.hans_meine@gmx.net>

Am Sonntag, 18. September 2011, 02:54:56 schrieb Gael Varoquaux:
> On Sat, Sep 17, 2011 at 06:52:09PM -0500, Robert Kern wrote:
> > > Actually, reST was originally designed to write docstrings.
> > 
> > Nope, it was designed for writing whole web pages in Zope.
> > 
> >    http://old.zope.org/Products/ZEO/Wiki/StructuredText
> 
> My bad.

Not really.  reST == _re_structured text.

See http://docutils.sourceforge.net/docs/ref/rst/introduction.html#history

  "StructuredText was developed by Jim Fulton of Zope Corporation ? now 
released as a part of [ZOPE] ? I discovered StructuredText in late 1999 while 
searching for a way to document the Python modules in one of my projects.
? I decided that a complete rewrite was in order, and [started 
reStructuredText]"

So you were both right, but talking about two (slightly) different things. ;-)

Looking forward to the best rst support in the Notebook thats possible.
Maybe a resumable rst parser is needed, possibly even written in JS (shudder), 
like this one http://codemirror.net/1/story.html

Best,
  Hans

From cekees at gmail.com  Tue Sep 20 09:22:20 2011
From: cekees at gmail.com (Chris Kees)
Date: Tue, 20 Sep 2011 08:22:20 -0500
Subject: [IPython-dev] roundtable on python distro for hpc
Message-ID: 

I think it would be great if some ipython developers and users were
represented on this online "roundtable" on python and software
distributions for HPC platforms:

http://fixingscientificsoftwaredistribution.blogspot.com/2011/09/round-table-about-fixing-problem-of.html

Here is some preliminary discussion as well:

https://groups.google.com/d/topic/mpi4py/dOqv4smldbU/discussion

Cheers,
Chris Kees

From fperez.net at gmail.com  Tue Sep 20 14:41:44 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 20 Sep 2011 11:41:44 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110917034415.GE13494@phare.normalesup.org>
References: 

	<20110917034415.GE13494@phare.normalesup.org>
Message-ID: 

On Fri, Sep 16, 2011 at 8:44 PM, Gael Varoquaux
 wrote:
>
> It seems to me wrong to go with any other markup language, given that the
> standard in the Python world in reST. It is hard enough to agree on
> standards, let us rejoice that there is one.

I would have hoped for less of a cheap shot coming from you.

We've explained (I said it many times at Euroscipy, with lots of
detail)  that there are *technical* reasons why we weren't able to put
reST/shpinx integration into the notebook from day one, despite all of
us desperately wanting it.  We've heard of sphinx before, believe it
or not.

But the design of reST, in particular the highly stateful nature of
the parsing process that makes whole-document multi-pass processing
more or less mandatory, makes it complex and costly to implement.
It's especially hard in an environment like the notebook, where you'd
like to provide immediate feedback on a per-cell basis while keeping
the load on the server as light as possible, and the number of
asynchronous processes in flight to a minimum (these add complexity
and brittleness).

Cheers,

f

From fperez.net at gmail.com  Tue Sep 20 15:28:21 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 20 Sep 2011 12:28:21 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: <4E74A035.2040709@creativetrax.com>
References: 

	<4E74A035.2040709@creativetrax.com>
Message-ID: 

On Sat, Sep 17, 2011 at 6:27 AM, Jason Grout
 wrote:
>
>
> Whereas in Sage, a worksheet .sws file is more like the entire directory
> containing code to be executed, supporting files, etc.

Yes, and there's an important reason behind this difference: for us,
being able to type

ls

in a notebook and still getting the 'real' directory listing is very
important.  The notebook is controlling a regular ipython kernel that
works just like the cmd-line ipython, with direct access to local
files, the ability to %run scripts, etc.  We wanted this lightweight,
your-filesystem-is-still-yours model because it's a very natural
worfklow when analyzing data and working with local files.  In
contrast, Sage uses the filesystem for its own execution model, with
temporary directories created for cell execution.  This means that
ls/os.listdir('.') shows this internal filesystem structure.

There are many benefits for Sage with this approach and it's certainly
a valid one, but for our use cases and for fitting naturally with the
command-line ipython, we went with our model instead, that leaves the
filesystem/cwd unmodified, and where a notebook is just one more file
sitting in your working directory.

I hope this clarifies things somewhat.

Cheers,

f

From gael.varoquaux at normalesup.org  Tue Sep 20 17:56:13 2011
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 20 Sep 2011 23:56:13 +0200
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

	<20110917034415.GE13494@phare.normalesup.org>

Message-ID: <20110920215613.GB4207@phare.normalesup.org>

On Tue, Sep 20, 2011 at 11:41:44AM -0700, Fernando Perez wrote:
> On Fri, Sep 16, 2011 at 8:44 PM, Gael Varoquaux
>  wrote:

> > It seems to me wrong to go with any other markup language, given that the
> > standard in the Python world in reST. It is hard enough to agree on
> > standards, let us rejoice that there is one.

> I would have hoped for less of a cheap shot coming from you.

I am a bit saddened that you call this a cheap shot. I am giving a user's
point of view that differs from your developper's point of view, but is
definitely a point of view many people will have.

> We've explained (I said it many times at Euroscipy, with lots of
> detail)  that there are *technical* reasons why we weren't able to put
> reST/shpinx integration into the notebook from day one, despite all of
> us desperately wanting it.  We've heard of sphinx before, believe it
> or not.

When I was working at Enthought, in such a situation, Eric Jones would
tell me that I am letting a technical difficulty force a user interface
decision (he had a good sentence for that, but I can't remember it). Of
course, it's a tradeoff: you can put the cost on the developer, or on the
user. Depending on your target audience and the size of your development
team, you choose the right compromise in the tradeoff space.

I still expect that you will get this question many times, just like when
I work with people that I new to Python, they keep asking me why Python
cannot reload modules reliably. This is not the end of the world, but I
do think that it helps to think about usability when make core choices,
and of course balance them with the other parts of the equation. I cannot
make choices for you.

Cheers,

Gael

From takowl at gmail.com  Tue Sep 20 18:30:41 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 20 Sep 2011 23:30:41 +0100
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110920215613.GB4207@phare.normalesup.org>
References: 

	<20110917034415.GE13494@phare.normalesup.org>

	<20110920215613.GB4207@phare.normalesup.org>
Message-ID: 

On 20 September 2011 22:56, Gael Varoquaux wrote:

> Of
> course, it's a tradeoff: you can put the cost on the developer, or on the
> user. Depending on your target audience and the size of your development
>
team, you choose the right compromise in the tradeoff space
>

I don't want to add fuel to the fire here, but I'd just like to mention that
I actually like markdown from a user perspective (I wasn't involved with
notebook development). Things like the link syntax feel more natural than
reST, and the option to use HTML tags is useful. It's also widely used on
popular websites, and it wouldn't surprise me to learn that casual users are
more familiar with it than they are with reST.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Tue Sep 20 18:45:53 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 20 Sep 2011 15:45:53 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110920215613.GB4207@phare.normalesup.org>
References: 

	<20110917034415.GE13494@phare.normalesup.org>

	<20110920215613.GB4207@phare.normalesup.org>
Message-ID: 

On Tue, Sep 20, 2011 at 14:56, Gael Varoquaux  wrote:

> On Tue, Sep 20, 2011 at 11:41:44AM -0700, Fernando Perez wrote:
> > On Fri, Sep 16, 2011 at 8:44 PM, Gael Varoquaux
> >  wrote:
>
> > > It seems to me wrong to go with any other markup language, given that
> the
> > > standard in the Python world in reST. It is hard enough to agree on
> > > standards, let us rejoice that there is one.
>
> > I would have hoped for less of a cheap shot coming from you.
>
> I am a bit saddened that you call this a cheap shot. I am giving a user's
> point of view that differs from your developper's point of view, but is
> definitely a point of view many people will have.
>

> > We've explained (I said it many times at Euroscipy, with lots of
> > detail)  that there are *technical* reasons why we weren't able to put
> > reST/shpinx integration into the notebook from day one, despite all of
> > us desperately wanting it.  We've heard of sphinx before, believe it
> > or not.
>
> When I was working at Enthought, in such a situation, Eric Jones would
> tell me that I am letting a technical difficulty force a user interface
> decision (he had a good sentence for that, but I can't remember it). Of
> course, it's a tradeoff: you can put the cost on the developer, or on the
> user. Depending on your target audience and the size of your development
> team, you choose the right compromise in the tradeoff space.
>
> I still expect that you will get this question many times, just like when
> I work with people that I new to Python, they keep asking me why Python
> cannot reload modules reliably. This is not the end of the world, but I
> do think that it helps to think about usability when make core choices,
> and of course balance them with the other parts of the equation. I cannot
> make choices for you.
>

Yes, many Python people will want rST instead of markdown, and they will get
it - but not like the markdown cell is done right now, with nice, responsive
single-cell rendering in the browser.  We have actually made this choice
with user experience in mind. The design of the language makes rST a very
bad choice for light interactive text cells, and when we implement rST
support, it should be for full-document formatting, where rST's advantages
can shine.

For instance, if you have a header in a text cell, would you expect that
your formatting, links, etc. be consistent throughout the document, or
independent on a per-cell basis?  It makes sense from an rST perspective
that you would want your header levels to be respected across cells.
 However, since rST simply interprets the first header-style it sees as h1,
the second as h2, etc. this means that the only reasonable option is to
rerender all cells together every single time - cells *cannot* be rendered
independently. That, combined with rST performance, means that the
interactive user experience we want is simply not achievable.  It is simply
the wrong tool for the job, from a user-experience perspective.  When we do
use rST, it will format the cells all together, in order to preserve the
scope behavior that rST writers expect.  Choosing rST will mean forfeiting
some quality of the interactive experience for the more powerful tool.  We
*will* support it, but in an appropriate manner that actually benefits from
rST's advantages over md.

-MinRK

>
> Cheers,
>
> Gael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Tue Sep 20 18:50:52 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 20 Sep 2011 15:50:52 -0700
Subject: [IPython-dev] notebook questions
In-Reply-To: <20110920215613.GB4207@phare.normalesup.org>
References: 

	<20110917034415.GE13494@phare.normalesup.org>

	<20110920215613.GB4207@phare.normalesup.org>
Message-ID: 

On Tue, Sep 20, 2011 at 2:56 PM, Gael Varoquaux
 wrote:
>
> I am a bit saddened that you call this a cheap shot. I am giving a user's
> point of view that differs from your developper's point of view, but is
> definitely a point of view many people will have.

The problem is that you said we were making the wrong choice and
ignoring standards.  When we've made it abundantly clear that we
*want* to support reST, and we *will* support reST/sphinx, and we know
about it, but it's so complex to do that we didn't have the manpower
to do it *initially*.  You accuse us of ignoring standards and making
the wrong choices when we've explicitly said we're simply doing first
what we can do *given the limited manpower we have*, and how we plan
to tackle the harder reST problem as our resources allow.

So yes, ignoring the real-world constraints we have and that you are
well aware of, and claiming we're making invalid choices that ignore
standards and user experience instead, is a cheap shot.

Cheers,

f

From jason-sage at creativetrax.com  Tue Sep 20 19:22:50 2011
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 20 Sep 2011 18:22:50 -0500
Subject: [IPython-dev] notebook questions
In-Reply-To: 
References: 

	<4E74A035.2040709@creativetrax.com>

Message-ID: <4E79204A.3040705@creativetrax.com>

On 9/20/11 2:28 PM, Fernando Perez wrote:
> On Sat, Sep 17, 2011 at 6:27 AM, Jason Grout
>   wrote:
>>
>>
>> Whereas in Sage, a worksheet .sws file is more like the entire directory
>> containing code to be executed, supporting files, etc.
>
> Yes, and there's an important reason behind this difference: for us,
> being able to type
>
> ls
>
> in a notebook and still getting the 'real' directory listing is very
> important.  The notebook is controlling a regular ipython kernel that
> works just like the cmd-line ipython, with direct access to local
> files, the ability to %run scripts, etc.  We wanted this lightweight,
> your-filesystem-is-still-yours model because it's a very natural
> worfklow when analyzing data and working with local files.  In
> contrast, Sage uses the filesystem for its own execution model, with
> temporary directories created for cell execution.  This means that
> ls/os.listdir('.') shows this internal filesystem structure.
>
> There are many benefits for Sage with this approach and it's certainly
> a valid one, but for our use cases and for fitting naturally with the
> command-line ipython, we went with our model instead, that leaves the
> filesystem/cwd unmodified, and where a notebook is just one more file
> sitting in your working directory.
>
> I hope this clarifies things somewhat.

Perfect.  There are important use-cases for both scenarios.  I'm glad 
ipython fills this role.

Thanks,

Jason

From mark.voorhies at ucsf.edu  Tue Sep 20 20:09:49 2011
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Tue, 20 Sep 2011 17:09:49 -0700
Subject: [IPython-dev] IPython build environment on Ubuntu 10.04
Message-ID: <201109201709.50083.mark.voorhies@ucsf.edu>

Just finished submitting a manuscript and have some time to do a bit of IPython hacking.
It looks like my installed dependencies are sufficiently out of date that I should rebuild
my environment (Cython/ZMQ/PyZMQ/IPython) from scratch.

Does anyone have advice for doing a non-system install of these pieces
(e.g., to $HOME/ipython/) on Ubuntu 10.04, preferably from their VCS versions?

Thanks in advance,

Mark

From fperez.net at gmail.com  Tue Sep 20 21:42:38 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 20 Sep 2011 18:42:38 -0700
Subject: [IPython-dev] IPython build environment on Ubuntu 10.04
In-Reply-To: <201109201709.50083.mark.voorhies@ucsf.edu>
References: <201109201709.50083.mark.voorhies@ucsf.edu>
Message-ID: 

Hi Mark,

On Tue, Sep 20, 2011 at 5:09 PM, Mark Voorhies  wrote:
> Just finished submitting a manuscript and have some time to do a bit of IPython hacking.
> It looks like my installed dependencies are sufficiently out of date that I should rebuild
> my environment (Cython/ZMQ/PyZMQ/IPython) from scratch.

Unless you plan to hack on pyzmq itself, you don't need cython to just
install pyzmq.

> Does anyone have advice for doing a non-system install of these pieces
> (e.g., to $HOME/ipython/) on Ubuntu 10.04, preferably from their VCS versions?

My standard procesure is just to install things in $HOME/usr/local via

python setup.py --prefix=$HOME/usr/local

or

./configure --prefix=$HOME/usr/local && make && make install

I have $PATH, $PYTHONPATH, $LD_LIBRARY_PATH and other such variables
configured for this to work out of the box.  If you need a hand with
this I'm happy to send over my bash utils that take care of it.

I'd recommend you go with released versions of zmq/pyzmq, as there's
no need to use the bleeding edge of those.  If you're interested in
the web notebook you'll need tornado from git, that's currently our
only non-released dependency.

Looking forward to having you contribute again!  Please let us know if
you get stuck on anything.

Cheers,

f

From benjaminrk at gmail.com  Wed Sep 21 00:01:01 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 20 Sep 2011 21:01:01 -0700
Subject: [IPython-dev] IPython build environment on Ubuntu 10.04
In-Reply-To: 
References: <201109201709.50083.mark.voorhies@ucsf.edu>

Message-ID: 

On Tue, Sep 20, 2011 at 18:42, Fernando Perez  wrote:

> Hi Mark,
>
> On Tue, Sep 20, 2011 at 5:09 PM, Mark Voorhies 
> wrote:
> > Just finished submitting a manuscript and have some time to do a bit of
> IPython hacking.
> > It looks like my installed dependencies are sufficiently out of date that
> I should rebuild
> > my environment (Cython/ZMQ/PyZMQ/IPython) from scratch.
>
> Unless you plan to hack on pyzmq itself, you don't need cython to just
> install pyzmq.
>

It's true - the only dev version of anything we currently depend on is
tornado (for the notebooks only), since browsers have made websocket
protocol changes in the their most recent versions, that are not compatible
with the tornado 2.0-release.

And pyzmq is easy_installable *iff* libzmq is in a default place like
/usr/local.  Otherwise, you have to run the configure step first, to tell it
where libzmq is.

However, if you do feel like tracking Cython, PyZMQ, IPython, and Tornado,
your wish is my command:
https://gist.github.com/1231171

I just created a brand new Ubuntu 11.04 VM, with no bashrc or anything, and
ran:

$> sudo apt-get install git uuid-dev python-dev python-qt4 python-pygments #
the last two are only for the qtconsole
$> export PATH=$HOME/.local:$PATH # add this, and optionally the matching
LD_LIBRARY_PATH to your .bashrc
$> wget https://raw.github.com/gist/1231171/ipdev.sh
$> bash ipdev.sh

And I have the IPython cluster, notebook server, qtconsole, etc. all ready
to run (and git repos for Cython, PyZMQ, IPython, and Tornado in a new
`code` directory).

If you don't want to keep the repos around, it's actually even easier, as
you can always install the current git snapshot of projects on github with:

$> easy_install --user https://github.com/ipython/ipython/tarball/master

> > Does anyone have advice for doing a non-system install of these pieces
> > (e.g., to $HOME/ipython/) on Ubuntu 10.04, preferably from their VCS
> versions?
>
> My standard procesure is just to install things in $HOME/usr/local via
>
> python setup.py --prefix=$HOME/usr/local
>
> or
>
> ./configure --prefix=$HOME/usr/local && make && make install
>

> I have $PATH, $PYTHONPATH, $LD_LIBRARY_PATH and other such variables
> configured for this to work out of the box.  If you need a hand with
> this I'm happy to send over my bash utils that take care of it.
>

I do a similar thing, but just use '--user' for Python installs, which go to
~/.local, which I symlink to ~/usr/local, using the two paths
interchangeably.  This way, only PATH and LD_LIBRARY_PATH are needed, as
Python (>= 2.6) already searches that location by default.  in fact, even
LD_LIB.. is often not needed, as rpaths are coded into dylibs, so pyzmq will
still work, even without setting the LD_LIB path, but I set it anyway.

-MinRK

>
> I'd recommend you go with released versions of zmq/pyzmq, as there's
> no need to use the bleeding edge of those.  If you're interested in
> the web notebook you'll need tornado from git, that's currently our
> only non-released dependency.
>
> Looking forward to having you contribute again!  Please let us know if
> you get stuck on anything.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From cekees at gmail.com  Wed Sep 21 00:24:02 2011
From: cekees at gmail.com (Chris Kees)
Date: Tue, 20 Sep 2011 23:24:02 -0500
Subject: [IPython-dev] minimal mpi4py notebook example
In-Reply-To: 
References: 

Message-ID: 

Hi Fernando,

Success! I ran your notebook example and turned it into a parallel
test problem from proteus.

I think %autopx is all one needs to make nice notebooks for parallel
PDE initial/boundary problems. Just bracket your SPMD parts of the
program with %autopx. I will try to get the visualization hooked to
the display protocol at some point in the next few weeks.  I've got a
lot to learn about debugging and passing information back and forth to
the engines, but this is a good start.

Thanks,
Chris

On Mon, Sep 19, 2011 at 9:22 PM, Fernando Perez  wrote:
> Hi Chris,
>
> responding on-list so this feedback benefits other users as well...
>
> On Mon, Sep 19, 2011 at 1:41 PM, Chris Kees  wrote:
>>
>> I think I can do it either way, but having the notebook be the rank-0 node is probably the most natural for me, since my experience is primarily with SIMD programming. Image files are composited and written to disk by the rank-0 process, so I think they would automatically show up in the notebook (or be easy to fit into the display protocol). ? On the other hand, I can encapsulate the entire parallel program in a function call, so my guess is I can do it either way. I'm not sure it's useful, but below I've included the entire program ?that needs to run either a) in a python interpreter that has already called MPI init or b) in one that was started with mpiexec.
>
> Well, after writing my email I realized that it's not so easy right
> now to have your notebook kernel be one of the mpi ones. ?There's no
> fundamental problem with the architecture, it's just a bit of missing
> code: right now kernels are started by the notebook server process,
> which then opens up a web connection on one side that maps to that
> kernel on the other side. ?We haven't added the ability to create a
> web notebook connected to an *existing kernel* based on its port
> information (like you can wire a qt console to an existing kernel).
> So the mpi processes could start a kernel all they want, there's no
> way right now to control that kernel via a notebook (though you could
> control it via a qtconsole, if you print/save the port data
> somewhere).
>
> If, with more experience, we find this to really be a necessary use
> case we can certainly implement things: all the architecture is
> designed so it could work, it's just a little bit of missing glue in
> places.
>
> But in the meantime, I've written up (attached) two notebooks for
> illustrating simple MPI use. ?The first is really the kind of code
> you'd run normally in a terminal, but I put it in a notebook to show
> that it can be a way for your users to start things if they are a bit
> terminal-phobic. ?They can use that notebook as a poor-man's terminal,
> and the 'busy' indicator will be on as long as the cluster is active.
> They can just stop it with the 'kernel interrupt' button, which
> effectively sends Ctrl-C. ?The same goal can be obtained with a screen
> session into the server (more robust than a simple ssh login), but
> this makes it nicely sit in your open notebooks next to the real work.
>
> The 'real work' is in the 2nd notebook, which just illustrates some
> basic communication with the engines and uses %autopx with the
> convenience of whole-cell editing to send them a little bit of MPI
> code (taken from the mpi4py docs).
>
> Let me know if this helps, and if so, we can put it in the docs.
>
> Cheers,
>
> f
>

From rohit.hiwale at gmail.com  Wed Sep 21 03:07:26 2011
From: rohit.hiwale at gmail.com (rohit hiwale)
Date: Wed, 21 Sep 2011 12:37:26 +0530
Subject: [IPython-dev] Can I open ipython from another python script with
 some objects shared (like code.InteractiveConsole) ?
Message-ID: 

I have a qt application.
I want to share my MainWindow object with ipython for debugging purpose.

Currently I am using code.InteractiveConsole which allows me to initialize
the locals for the python console.
*eg.    myconsole =
code.InteractiveConsole(locals={'shared_object':MainWindow_object})*
*        myconsole.interact()*
Does ipython supports such interactive console ?
If yes, how can I do same thing with ipython ?
It will be even better if it can be done with ipython-qtconsole !!

PS : ipython-qtconsole is awesome !!

Cheers,
Rohit

-- 
Rohit Hiwale
Final year Dual Degree (Bachelors+Masters) student
Department of Computer Science & Engineering
Indian Institute of Technology, Kharagpur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From takowl at gmail.com  Wed Sep 21 13:01:42 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 21 Sep 2011 18:01:42 +0100
Subject: [IPython-dev] Tornado 2.1
Message-ID: 

I see Tornado have just made a 2.1 release, including the new websocket
protocol. Hopefully the notebook will carry on working with the released
version.

http://www.tornadoweb.org/documentation/releases/v2.1.0.html

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mark.voorhies at ucsf.edu  Wed Sep 21 13:15:28 2011
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Wed, 21 Sep 2011 10:15:28 -0700
Subject: [IPython-dev] IPython build environment on Ubuntu 10.04
In-Reply-To: 
References: <201109201709.50083.mark.voorhies@ucsf.edu>

Message-ID: <201109211015.29287.mark.voorhies@ucsf.edu>

On Tuesday, September 20, 2011 09:01:01 pm you wrote:
> On Tue, Sep 20, 2011 at 18:42, Fernando Perez  wrote:
> 
> > Hi Mark,
> >
> > On Tue, Sep 20, 2011 at 5:09 PM, Mark Voorhies 
> > wrote:
> > > Just finished submitting a manuscript and have some time to do a bit of
> > IPython hacking.
> > > It looks like my installed dependencies are sufficiently out of date that
> > I should rebuild
> > > my environment (Cython/ZMQ/PyZMQ/IPython) from scratch.
> >
> > Unless you plan to hack on pyzmq itself, you don't need cython to just
> > install pyzmq.
> >
> 
> It's true - the only dev version of anything we currently depend on is
> tornado (for the notebooks only), since browsers have made websocket
> protocol changes in the their most recent versions, that are not compatible
> with the tornado 2.0-release.
> 
> And pyzmq is easy_installable *iff* libzmq is in a default place like
> /usr/local.  Otherwise, you have to run the configure step first, to tell it
> where libzmq is.
> 
> However, if you do feel like tracking Cython, PyZMQ, IPython, and Tornado,
> your wish is my command:
> https://gist.github.com/1231171
> 
> I just created a brand new Ubuntu 11.04 VM, with no bashrc or anything, and
> ran:
> 
> $> sudo apt-get install git uuid-dev python-dev python-qt4 python-pygments #
> the last two are only for the qtconsole
> $> export PATH=$HOME/.local:$PATH # add this, and optionally the matching
> LD_LIBRARY_PATH to your .bashrc
> $> wget https://raw.github.com/gist/1231171/ipdev.sh
> $> bash ipdev.sh
> 
> And I have the IPython cluster, notebook server, qtconsole, etc. all ready
> to run (and git repos for Cython, PyZMQ, IPython, and Tornado in a new
> `code` directory).
> 

Wow!

For the record, Min's script works out of the box on Ubuntu 10.04.

Thanks, Min and Fernando.

--Mark

> If you don't want to keep the repos around, it's actually even easier, as
> you can always install the current git snapshot of projects on github with:
> 
> $> easy_install --user https://github.com/ipython/ipython/tarball/master
> 
> 
> > > Does anyone have advice for doing a non-system install of these pieces
> > > (e.g., to $HOME/ipython/) on Ubuntu 10.04, preferably from their VCS
> > versions?
> >
> > My standard procesure is just to install things in $HOME/usr/local via
> >
> > python setup.py --prefix=$HOME/usr/local
> >
> > or
> >
> > ./configure --prefix=$HOME/usr/local && make && make install
> >
> 
> > I have $PATH, $PYTHONPATH, $LD_LIBRARY_PATH and other such variables
> > configured for this to work out of the box.  If you need a hand with
> > this I'm happy to send over my bash utils that take care of it.
> >
> 
> I do a similar thing, but just use '--user' for Python installs, which go to
> ~/.local, which I symlink to ~/usr/local, using the two paths
> interchangeably.  This way, only PATH and LD_LIBRARY_PATH are needed, as
> Python (>= 2.6) already searches that location by default.  in fact, even
> LD_LIB.. is often not needed, as rpaths are coded into dylibs, so pyzmq will
> still work, even without setting the LD_LIB path, but I set it anyway.
> 
> -MinRK
> 
> 
> 
> >
> > I'd recommend you go with released versions of zmq/pyzmq, as there's
> > no need to use the bleeding edge of those.  If you're interested in
> > the web notebook you'll need tornado from git, that's currently our
> > only non-released dependency.
> >
> > Looking forward to having you contribute again!  Please let us know if
> > you get stuck on anything.
> >
> > Cheers,
> >
> > f
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> 

From fperez.net at gmail.com  Wed Sep 21 13:20:35 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 21 Sep 2011 10:20:35 -0700
Subject: [IPython-dev] Tornado 2.1
In-Reply-To: 
References: 
Message-ID: 

On Wed, Sep 21, 2011 at 10:01 AM, Thomas Kluyver  wrote:
> I see Tornado have just made a 2.1 release, including the new websocket
> protocol. Hopefully the notebook will carry on working with the released
> version.
>
> http://www.tornadoweb.org/documentation/releases/v2.1.0.html

Great news!  On my system, the tests all pass with this version, and
all the notebooks I looked at interactively work fine, both with
Firefox 6 and with Chrome.

I'll update our instructions now to point users to this version.

See, all we had to do was wait a day ;)

Cheers,

f

From ellisonbg at gmail.com  Wed Sep 21 14:32:06 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 21 Sep 2011 11:32:06 -0700
Subject: [IPython-dev] Tornado 2.1
In-Reply-To: 
References: 

Message-ID: 

Great!

On Wed, Sep 21, 2011 at 10:20 AM, Fernando Perez  wrote:
> On Wed, Sep 21, 2011 at 10:01 AM, Thomas Kluyver  wrote:
>> I see Tornado have just made a 2.1 release, including the new websocket
>> protocol. Hopefully the notebook will carry on working with the released
>> version.
>>
>> http://www.tornadoweb.org/documentation/releases/v2.1.0.html
>
> Great news! ?On my system, the tests all pass with this version, and
> all the notebooks I looked at interactively work fine, both with
> Firefox 6 and with Chrome.
>
> I'll update our instructions now to point users to this version.
>
> See, all we had to do was wait a day ;)
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com

From jtaylor.debian at googlemail.com  Wed Sep 21 15:00:07 2011
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Wed, 21 Sep 2011 21:00:07 +0200
Subject: [IPython-dev] Can I open ipython from another python script
 with some objects shared (like code.InteractiveConsole) ?
In-Reply-To: 
References: 
Message-ID: <4E7A3437.7050107@googlemail.com>

On 09/21/2011 09:07 AM, rohit hiwale wrote:
> I have a qt application.
> I want to share my MainWindow object with ipython for debugging purpose.
> 
> Currently I am using code.InteractiveConsole which allows me to
> initialize the locals for the python console.
> /eg.    myconsole =
> code.InteractiveConsole(locals={'shared_object':MainWindow_object})/
> /        myconsole.interact()/
> Does ipython supports such interactive console ?
> If yes, how can I do same thing with ipython ?
> It will be even better if it can be done with ipython-qtconsole !!
> 
> PS : ipython-qtconsole is awesome !!

yes you can embed an terminal ipython console into your application.
See an example in docs/examples/core/example-embed.py
and this page for both 0.10 and 0.11 variants + a bunch of patches for
applications already doing it:
http://wiki.ipython.org/Cookbook/Updating_code_that_uses_IPython_for_0.11

Best Regards,
Julian Taylor

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From Fernando.Perez at berkeley.edu  Fri Sep 23 13:06:55 2011
From: Fernando.Perez at berkeley.edu (Fernando Perez)
Date: Fri, 23 Sep 2011 10:06:55 -0700
Subject: [IPython-dev] Fwd: [ftpmaster@ftp-master.debian.org:
 python-tornado_2.1.0-1_amd64.changes ACCEPTED into unstable]
In-Reply-To: <20110923140642.GP5865@onerussian.com>
References: <20110923140642.GP5865@onerussian.com>
Message-ID: 

A big thank you to Yarik and Michael, the indefatigable engines behind
neurodebian, for packaging the new tornado so quickly!

---------- Forwarded message ----------
From: Yaroslav Halchenko 
Date: Fri, Sep 23, 2011 at 07:06
Subject: [ftpmaster at ftp-master.debian.org:
python-tornado_2.1.0-1_amd64.changes ACCEPTED into unstable]
To: Fernando Perez 
Cc: Michael Hanke 

oki doki -- done!

also recent Debian (squeeze, wheezy) and Ubuntu (only 11.04)
releases can get it from NeuroDebian repository

Cheers,

----- Forwarded message from Debian FTP Masters
 -----

Date: Fri, 23 Sep 2011 13:36:02 +0000
From: Debian FTP Masters 
To: Yaroslav Halchenko ,
? ? ? ?Debian Python Modules Team 
Subject: python-tornado_2.1.0-1_amd64.changes ACCEPTED into unstable
X-CRM114-Status: UNSURE (2.3293) This message is 'unsure'; please train it!

Accepted:
python-tornado_2.1.0-1.debian.tar.gz
?to main/p/python-tornado/python-tornado_2.1.0-1.debian.tar.gz
python-tornado_2.1.0-1.dsc
?to main/p/python-tornado/python-tornado_2.1.0-1.dsc
python-tornado_2.1.0-1_amd64.deb
?to main/p/python-tornado/python-tornado_2.1.0-1_amd64.deb
python-tornado_2.1.0.orig.tar.gz
?to main/p/python-tornado/python-tornado_2.1.0.orig.tar.gz

Override entries for your package:
python-tornado_2.1.0-1.dsc - source python
python-tornado_2.1.0-1_amd64.deb - optional python

Announcing to debian-devel-changes at lists.debian.org
Closing bugs: 636602

Thank you for your contribution to Debian.

----- End forwarded message -----

--
Yaroslav O. Halchenko
Postdoctoral Fellow, ? Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 ? ? ? ? ? ? ? ? ? ? ? Fax: +1 (603) 646-1419
WWW: ? http://www.linkedin.com/in/yarik

From takowl at gmail.com  Wed Sep 28 19:09:21 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 29 Sep 2011 00:09:21 +0100
Subject: [IPython-dev] IPython talk in Sheffield
Message-ID: 

I think the talk I gave last night was quite well received - I got quite a
few questions afterwards, and several people thanked me for coming to speak
about it. I had time to do brief demos of both the Qt console and the
notebook. There was particular interest in the notebook, with some of the
questions centring on "how easy is it to get that set up?" (I described the
bits necessary, and suggested EPD as a good way in from Windows).

Twitter user @allegary said "talk on ipython is nice and entertaining -
liking what they have done with the place #pysheff".

I've attached the slides in PDF format - much of the content is taken from
Fernando's slides for Euroscipy.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython presentation.pdf
Type: application/pdf
Size: 790509 bytes
Desc: not available
URL: 

From fperez.net at gmail.com  Wed Sep 28 20:13:14 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 28 Sep 2011 17:13:14 -0700
Subject: [IPython-dev] IPython talk in Sheffield
In-Reply-To: 
References: 
Message-ID: 

Hey Thomas,

On Wed, Sep 28, 2011 at 4:09 PM, Thomas Kluyver  wrote:
> I think the talk I gave last night was quite well received - I got quite a
> few questions afterwards, and several people thanked me for coming to speak
> about it. I had time to do brief demos of both the Qt console and the
> notebook. There was particular interest in the notebook, with some of the
> questions centring on "how easy is it to get that set up?" (I described the
> bits necessary, and suggested EPD as a good way in from Windows).
>
> Twitter user @allegary said "talk on ipython is nice and entertaining -
> liking what they have done with the place #pysheff".
>
> I've attached the slides in PDF format - much of the content is taken from
> Fernando's slides for Euroscipy.

Excellent slides!  And thanks a lot for making this presentation!
BTW, you should upload the slides over and link them at the
presentations page:

http://ipython.org/presentation.html

I'm behind on some urgent stuff right now, but on Friday perhaps we
can start the conversation on "how easy is it to get that set up" by
looking at plans to push 0.12 out the door...

Cheers,

f

From takowl at gmail.com  Thu Sep 29 08:52:19 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 29 Sep 2011 13:52:19 +0100
Subject: [IPython-dev] IPython talk in Sheffield
In-Reply-To: 
References: 

Message-ID: 

On 29 September 2011 01:13, Fernando Perez  wrote:

> Excellent slides!  And thanks a lot for making this presentation!
> BTW, you should upload the slides over and link them at the
> presentations page:
>
> http://ipython.org/presentation.html
>

Thanks, Fernando, I'll do that.

>  I'm behind on some urgent stuff right now, but on Friday perhaps we
> can start the conversation on "how easy is it to get that set up" by
> looking at plans to push 0.12 out the door...
>

I think this is a good idea - there's a number of fixes and new stuff that
it would be good to get to users sooner rather than later. In particular,
I'd like 0.12 to get into Ubuntu 12.04 (P cycle), which will be a long term
support release, i.e. some users will stick with it for two years. The
automatic syncing of packages from Debian will stop on 29 December (
https://wiki.ubuntu.com/PReleaseSchedule), so that means aiming for a
release before Christmas.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From hans_meine at gmx.net  Thu Sep 29 09:13:10 2011
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 29 Sep 2011 15:13:10 +0200
Subject: [IPython-dev] IPython talk in Sheffield
In-Reply-To: 
References: 

Message-ID: <201109291513.11129.hans_meine@gmx.net>

Am Donnerstag, 29. September 2011, 02:13:14 schrieb Fernando Perez:
> I'm behind on some urgent stuff right now, but on Friday perhaps we
> can start the conversation on "how easy is it to get that set up" by
> looking at plans to push 0.12 out the door...

Concerning "easy setup", I missed PPAs to be mentioned in a recent discussion 
as an easy installation method for Ubuntu systems.  In particular, Chris Lea 
(whom I don?t know personally, so trust him on your own risk) seems to offer 
all optional ipython dependencies, so that it is enough to issue:

  sudo apt-add-repository ppa:chris-lea/libpgm
  sudo apt-add-repository ppa:chris-lea/zeromq
  sudo apt-add-repository ppa:chris-lea/python-tornado

Afterwards, you can

  sudo apt-get install python-pyzmq python-tornado

and you should even get automatic updates (as soon as he provides ?em).

He seems to support Karmic, Lucid, Maverick, and Natty.

HTH,
  Hans

From takowl at gmail.com  Thu Sep 29 09:39:40 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 29 Sep 2011 14:39:40 +0100
Subject: [IPython-dev] IPython talk in Sheffield
In-Reply-To: <201109291513.11129.hans_meine@gmx.net>
References: 

	<201109291513.11129.hans_meine@gmx.net>
Message-ID: 

On 29 September 2011 14:13, Hans Meine  wrote:

> Concerning "easy setup", I missed PPAs to be mentioned in a recent
> discussion
> as an easy installation method for Ubuntu systems.  In particular, Chris
> Lea
> (whom I don?t know personally, so trust him on your own risk) seems to
> offer
> all optional ipython dependencies, so that it is enough to issue:
>
>  sudo apt-add-repository ppa:chris-lea/libpgm
>  sudo apt-add-repository ppa:chris-lea/zeromq
>  sudo apt-add-repository ppa:chris-lea/python-tornado
>

Perhaps even more conveniently, Julian Taylor made an ipython-dev ppa, which
contains IPython (albeit not a dev version at the moment), zeromq, pyzmq and
libpgm:

https://launchpad.net/~jtaylor/+archive/ipython-dev

Tornado isn't in there at the moment, but it would be an obvious thing to
add if we're packaging the version with the notebook. Hopefully in a few
months, we won't be depending on so much bleeding-edge stuff, so it will be
easier to set up.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jtaylor.debian at googlemail.com  Thu Sep 29 13:27:08 2011
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Thu, 29 Sep 2011 19:27:08 +0200
Subject: [IPython-dev] IPython ubuntu packaging.
In-Reply-To: 
References: 

	<201109291513.11129.hans_meine@gmx.net>

Message-ID: <4E84AA6C.3020700@googlemail.com>

On 09/29/2011 03:39 PM, Thomas Kluyver wrote:
> On 29 September 2011 14:13, Hans Meine  > wrote:
> 
>     Concerning "easy setup", I missed PPAs to be mentioned in a recent
>     discussion
>     as an easy installation method for Ubuntu systems.  In particular,
>     Chris Lea
>     (whom I don?t know personally, so trust him on your own risk) seems
>     to offer
>     all optional ipython dependencies, so that it is enough to issue:
> 
>      sudo apt-add-repository ppa:chris-lea/libpgm
>      sudo apt-add-repository ppa:chris-lea/zeromq
>      sudo apt-add-repository ppa:chris-lea/python-tornado
> 
> 
> Perhaps even more conveniently, Julian Taylor made an ipython-dev ppa,
> which contains IPython (albeit not a dev version at the moment), zeromq,
> pyzmq and libpgm:
> 
> https://launchpad.net/~jtaylor/+archive/ipython-dev
> 
> Tornado isn't in there at the moment, but it would be an obvious thing
> to add if we're packaging the version with the notebook. Hopefully in a
> few months, we won't be depending on so much bleeding-edge stuff, so it
> will be easier to set up.
> 
> Thomas
> 
> 

As the ppa is mentioned, here the current status and future plans for it:

The ppa contains 0.11 and its dependencies (mirrored from debian) but
currently only for oneiric and natty.

After oneiric is released (~2 weeks) I plan on pushing 0.11 or 0.12 to
the main archive quite early in the 12.04 LTS cycle.
I will then move 0.1[12] to my "stable" ipython ppa and probably support
lucid if possible too.
Then I can add more or less bleeding edge -dev versions to my
ipython-dev ppa including tornado and maybe mathjax (which hopefully
will be in debian soon:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=627518).

Debian experimental packaging will probably be a bit less up to date as
I don't have direct uploads rights there.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From takowl at gmail.com  Thu Sep 29 13:56:22 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 29 Sep 2011 18:56:22 +0100
Subject: [IPython-dev] IPython ubuntu packaging.
In-Reply-To: <4E84AA6C.3020700@googlemail.com>
References: 

	<201109291513.11129.hans_meine@gmx.net>

	<4E84AA6C.3020700@googlemail.com>
Message-ID: 

On 29 September 2011 18:27, Julian Taylor wrote:

> The ppa contains 0.11 and its dependencies (mirrored from debian) but
> currently only for oneiric and natty.
>
> After oneiric is released (~2 weeks) I plan on pushing 0.11 or 0.12 to
> the main archive quite early in the 12.04 LTS cycle.
> I will then move 0.1[12] to my "stable" ipython ppa and probably support
> lucid if possible too.
> Then I can add more or less bleeding edge -dev versions to my
> ipython-dev ppa including tornado and maybe mathjax (which hopefully
> will be in debian soon:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=627518).
>

Thanks for this, Julian. Do you have a feel for how much work it will be to
prepare 0.12 for packaging? I'm hoping it will make it into the 12.04 cycle,
so it would be good to know roughly when the window closes.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jtaylor.debian at googlemail.com  Thu Sep 29 14:13:49 2011
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Thu, 29 Sep 2011 20:13:49 +0200
Subject: [IPython-dev] IPython ubuntu packaging.
In-Reply-To: 
References: 

	<201109291513.11129.hans_meine@gmx.net>

	<4E84AA6C.3020700@googlemail.com>

Message-ID: <4E84B55D.9030107@googlemail.com>

On 09/29/2011 07:56 PM, Thomas Kluyver wrote:
> On 29 September 2011 18:27, Julian Taylor  > wrote:
> 
>     The ppa contains 0.11 and its dependencies (mirrored from debian) but
>     currently only for oneiric and natty.
> 
>     After oneiric is released (~2 weeks) I plan on pushing 0.11 or 0.12 to
>     the main archive quite early in the 12.04 LTS cycle.
>     I will then move 0.1[12] to my "stable" ipython ppa and probably support
>     lucid if possible too.
>     Then I can add more or less bleeding edge -dev versions to my
>     ipython-dev ppa including tornado and maybe mathjax (which hopefully
>     will be in debian soon:
>     http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=627518).
> 
> 
> Thanks for this, Julian. Do you have a feel for how much work it will be
> to prepare 0.12 for packaging? I'm hoping it will make it into the 12.04
> cycle, so it would be good to know roughly when the window closes.
> 
> Thanks,
> Thomas

If you don't break the API to badly it will probably not be much work.
Pretty much all packaging work that had to be done was already done for
0.11 and should help making packaging of future releases a bit easier.

There should be plenty of time to get 0.12 or higher into 12.04, the
feature freeze will probably be end of February 2012, the release will
be end of April.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From takowl at gmail.com  Thu Sep 29 14:17:59 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 29 Sep 2011 19:17:59 +0100
Subject: [IPython-dev] IPython ubuntu packaging.
In-Reply-To: <4E84B55D.9030107@googlemail.com>
References: 

	<201109291513.11129.hans_meine@gmx.net>

	<4E84AA6C.3020700@googlemail.com>

	<4E84B55D.9030107@googlemail.com>
Message-ID: 

On 29 September 2011 19:13, Julian Taylor wrote:

> There should be plenty of time to get 0.12 or higher into 12.04, the
> feature freeze will probably be end of February 2012, the release will
> be end of April.
>

Will they be cagey about taking things after the import freeze, if its going
to be an LTS release? I don't really know about the procedures, but I
thought the aim was to get it into Debian before the import freeze, so that
it would be synced automatically.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jtaylor.debian at googlemail.com  Thu Sep 29 14:58:23 2011
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Thu, 29 Sep 2011 20:58:23 +0200
Subject: [IPython-dev] IPython ubuntu packaging.
In-Reply-To: 
References: 

	<201109291513.11129.hans_meine@gmx.net>

	<4E84AA6C.3020700@googlemail.com>

	<4E84B55D.9030107@googlemail.com>

Message-ID: <4E84BFCF.5000100@googlemail.com>

On 09/29/2011 08:17 PM, Thomas Kluyver wrote:
> On 29 September 2011 19:13, Julian Taylor  > wrote:
> 
>     There should be plenty of time to get 0.12 or higher into 12.04, the
>     feature freeze will probably be end of February 2012, the release will
>     be end of April.
> 
> 
> Will they be cagey about taking things after the import freeze, if its
> going to be an LTS release? I don't really know about the procedures,
> but I thought the aim was to get it into Debian before the import
> freeze, so that it would be synced automatically.
> 

You can add new stuff after feature freeze, but it should be motivated
somehow, e.g. the new features are really useful, it was very well
tested etc. Generally whenever the gains outweighs possible regression
risks [0].
The closer one gets to release to more conservative one should be, but
as long as you have a maintainer who cares about the package and will
fix regressions also after release the rules can be relaxed quite a bit,
especially for development focused packages like ipython.

The packages will be still synced from debian whenever possible.

[0] https://wiki.ubuntu.com/FreezeExceptionProcess

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 