From benjaminrk at gmail.com  Mon Jul  1 00:19:46 2013
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 30 Jun 2013 21:19:46 -0700
Subject: [IPython-dev] MultipleInstanceError
In-Reply-To: <CAKgW=6J=2KHiAitUVk6L+dhfK=ha4YHDjW9u4+MZnAjbDDZJSg@mail.gmail.com>
References: <CAKgW=6Ksd_Xtw8mhYcAeQRcb95oE3AnXsezEL_+Tb_RG0wMAiQ@mail.gmail.com>
	<CAOvn4qjC860wjhdLbp8YccjZHHP3f5A+1QY7XZjdhoaTNeZYBA@mail.gmail.com>
	<CAHAreOoVN4fx+vdyJEnjGPi5ZHi+oBM_YJ1r2gm1TP8kQosa8A@mail.gmail.com>
	<CAKgW=6LFFd4_-2KR7KNb-YyzsExHYa+VAkbAo7RLB9KpLfH-EA@mail.gmail.com>
	<CAHAreOruxoe0nmD5KEQE5M3WbR9dfogHHDBJ3ZPkiUFQYO=3yw@mail.gmail.com>
	<CAHNn8BXcrTkYp-JkdwvjQOWHP29Q6od1r1A65L+5OYg2g0r+qw@mail.gmail.com>
	<CAKgW=6J=2KHiAitUVk6L+dhfK=ha4YHDjW9u4+MZnAjbDDZJSg@mail.gmail.com>
Message-ID: <CAHNn8BUqqEsTDbiLfo+kmyE+AYucw-GAvhnUjrSFSh8xwfGORQ@mail.gmail.com>

Does it embed with IPython.embed()?  That's probably what changed -
IPython.embed properly using the instance method, which I know it didn't at
some point.

The 'workaround' might be something like:

1. save the InteractiveShell instance
2. call ip.__class__.clear_instance()
3. proceed to embed
4. after embed, restore the instance (this will probably be the messiest
bit)

I can look into allowing embed to be called without disrupting the
singleton.


On Sun, Jun 30, 2013 at 8:50 PM, Aaron Meurer <asmeurer at gmail.com> wrote:

> I bisected it to
>
> commit c5e77f946f77dd95ab086ea3e02802c4874c5e3d
> Author: MinRK <benjaminrk at gmail.com>
> Date:   Fri Feb 8 21:41:19 2013 -0800
>
>     next release will be 1.0
>
> which means either I bisected it incorrectly (possible, since I was
> relying on python setup.py develop), or there is something in the
> version checking logic of PuDB.
>
> My use case is to add pudb.set_trace() to something in sympy and run
> isympy, which launches IPython. Then I press ! in PuDB to open IPython
> in the namespace of the function I am debugging.
>
> Aaron Meurer
>
> On Sun, Jun 30, 2013 at 10:33 PM, MinRK <benjaminrk at gmail.com> wrote:
> > This shouldn't be something that has changed since 0.11. Are you seeing a
> > difference between 0.13 and master?
> >
> > The workaround is to not use `instance` to create new instances of
> objects.
> > The result will be that methods asking for the global instance will
> retrieve
> > the original global instance, as opposed to the new one. I don't know if
> > this is acceptable or not.
> >
> >
> >
> > On Sun, Jun 30, 2013 at 7:44 PM, Fernando Perez <fperez.net at gmail.com>
> > wrote:
> >>
> >> Mmh, I'm not sure there's one at this point. Min might know, the
> >> current enforcement of the single-instance rule is wired deep inside
> >> the config machinery.
> >>
> >> On Sun, Jun 30, 2013 at 7:23 PM, Aaron Meurer <asmeurer at gmail.com>
> wrote:
> >> > So what is the workaround for PuDB?
> >> >
> >> > Aaron Meurer
> >> >
> >> > On Sun, Jun 30, 2013 at 9:21 PM, Fernando Perez <fperez.net at gmail.com
> >
> >> > wrote:
> >> >> Indeed, way back when IPython made blunt assumptions about being more
> >> >> or less in control of the user session, and created lots of global
> >> >> state. That made sense when it was only an interactive user shell,
> but
> >> >> now that it's a service to manage namespaces, its core should be much
> >> >> more constrained in how it pollutes global structures.
> >> >>
> >> >> Unfortunately as Thomas said, cleaning this up takes time and isn't
> at
> >> >> the top of the priority list.
> >> >>
> >> >> Cheers,
> >> >>
> >> >> f
> >> >>
> >> >> On Sun, Jun 30, 2013 at 8:31 AM, Thomas Kluyver <takowl at gmail.com>
> >> >> wrote:
> >> >>> Various parts of the IPython machinery assume that there's only one
> >> >>> instance
> >> >>> in a given process, and would break with multiple IPython shells in
> a
> >> >>> single
> >> >>> process. We're slowly moving away from that, but it's not high
> >> >>> priority for
> >> >>> us.
> >> >>>
> >> >>> Thomas
> >> >>>
> >> >>>
> >> >>> On 30 June 2013 05:52, Aaron Meurer <asmeurer at gmail.com> wrote:
> >> >>>>
> >> >>>> In IPython dev, if you try to start an IPython instance from within
> >> >>>> an
> >> >>>> IPython instance, it raises MultipleInstanceError. What is the
> >> >>>> purpose
> >> >>>> of this? It breaks a use-case of mine, which is to start PuDB from
> >> >>>> within IPython, then to open the IPython interpreter from within
> PuDB
> >> >>>> while debugging?  Is the new way to do things to somehow reattach
> to
> >> >>>> the same IPython instance?
> >> >>>>
> >> >>>> I've CCd the PuDB list.
> >> >>>>
> >> >>>> Aaron Meurer
> >> >>>> _______________________________________________
> >> >>>> IPython-dev mailing list
> >> >>>> IPython-dev at scipy.org
> >> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> IPython-dev mailing list
> >> >>> IPython-dev at scipy.org
> >> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Fernando Perez (@fperez_org; http://fperez.org)
> >> >> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> >> >> fernando.perez-at-berkeley: contact me here for any direct mail
> >> >> _______________________________________________
> >> >> IPython-dev mailing list
> >> >> IPython-dev at scipy.org
> >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> > _______________________________________________
> >> > IPython-dev mailing list
> >> > IPython-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >>
> >> --
> >> Fernando Perez (@fperez_org; http://fperez.org)
> >> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> >> fernando.perez-at-berkeley: contact me here for any direct mail
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130630/8999c6e7/attachment.html>

From asmeurer at gmail.com  Mon Jul  1 01:10:07 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 1 Jul 2013 00:10:07 -0500
Subject: [IPython-dev] MultipleInstanceError
In-Reply-To: <CAHNn8BUqqEsTDbiLfo+kmyE+AYucw-GAvhnUjrSFSh8xwfGORQ@mail.gmail.com>
References: <CAKgW=6Ksd_Xtw8mhYcAeQRcb95oE3AnXsezEL_+Tb_RG0wMAiQ@mail.gmail.com>
	<CAOvn4qjC860wjhdLbp8YccjZHHP3f5A+1QY7XZjdhoaTNeZYBA@mail.gmail.com>
	<CAHAreOoVN4fx+vdyJEnjGPi5ZHi+oBM_YJ1r2gm1TP8kQosa8A@mail.gmail.com>
	<CAKgW=6LFFd4_-2KR7KNb-YyzsExHYa+VAkbAo7RLB9KpLfH-EA@mail.gmail.com>
	<CAHAreOruxoe0nmD5KEQE5M3WbR9dfogHHDBJ3ZPkiUFQYO=3yw@mail.gmail.com>
	<CAHNn8BXcrTkYp-JkdwvjQOWHP29Q6od1r1A65L+5OYg2g0r+qw@mail.gmail.com>
	<CAKgW=6J=2KHiAitUVk6L+dhfK=ha4YHDjW9u4+MZnAjbDDZJSg@mail.gmail.com>
	<CAHNn8BUqqEsTDbiLfo+kmyE+AYucw-GAvhnUjrSFSh8xwfGORQ@mail.gmail.com>
Message-ID: <CAKgW=6LN+Tr4MKpROFX7cUsDEdwxYE2kATQ7X6io3zJmjzNYVw@mail.gmail.com>

This is what it does:
https://github.com/inducer/pudb/blob/master/pudb/shell.py#L106

As I noted, this is not the most common use case. Usually PuDB will be
started on its own, and there will be no issue. But quite often it is
useful to start it up from IPython rather than writing a little script
just to hit the breakpoint.

Aaron Meurer



On Sun, Jun 30, 2013 at 11:19 PM, MinRK <benjaminrk at gmail.com> wrote:
> Does it embed with IPython.embed()?  That's probably what changed -
> IPython.embed properly using the instance method, which I know it didn't at
> some point.
>
> The 'workaround' might be something like:
>
> 1. save the InteractiveShell instance
> 2. call ip.__class__.clear_instance()
> 3. proceed to embed
> 4. after embed, restore the instance (this will probably be the messiest
> bit)
>
> I can look into allowing embed to be called without disrupting the
> singleton.
>
>
> On Sun, Jun 30, 2013 at 8:50 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>>
>> I bisected it to
>>
>> commit c5e77f946f77dd95ab086ea3e02802c4874c5e3d
>> Author: MinRK <benjaminrk at gmail.com>
>> Date:   Fri Feb 8 21:41:19 2013 -0800
>>
>>     next release will be 1.0
>>
>> which means either I bisected it incorrectly (possible, since I was
>> relying on python setup.py develop), or there is something in the
>> version checking logic of PuDB.
>>
>> My use case is to add pudb.set_trace() to something in sympy and run
>> isympy, which launches IPython. Then I press ! in PuDB to open IPython
>> in the namespace of the function I am debugging.
>>
>> Aaron Meurer
>>
>> On Sun, Jun 30, 2013 at 10:33 PM, MinRK <benjaminrk at gmail.com> wrote:
>> > This shouldn't be something that has changed since 0.11. Are you seeing
>> > a
>> > difference between 0.13 and master?
>> >
>> > The workaround is to not use `instance` to create new instances of
>> > objects.
>> > The result will be that methods asking for the global instance will
>> > retrieve
>> > the original global instance, as opposed to the new one. I don't know if
>> > this is acceptable or not.
>> >
>> >
>> >
>> > On Sun, Jun 30, 2013 at 7:44 PM, Fernando Perez <fperez.net at gmail.com>
>> > wrote:
>> >>
>> >> Mmh, I'm not sure there's one at this point. Min might know, the
>> >> current enforcement of the single-instance rule is wired deep inside
>> >> the config machinery.
>> >>
>> >> On Sun, Jun 30, 2013 at 7:23 PM, Aaron Meurer <asmeurer at gmail.com>
>> >> wrote:
>> >> > So what is the workaround for PuDB?
>> >> >
>> >> > Aaron Meurer
>> >> >
>> >> > On Sun, Jun 30, 2013 at 9:21 PM, Fernando Perez
>> >> > <fperez.net at gmail.com>
>> >> > wrote:
>> >> >> Indeed, way back when IPython made blunt assumptions about being
>> >> >> more
>> >> >> or less in control of the user session, and created lots of global
>> >> >> state. That made sense when it was only an interactive user shell,
>> >> >> but
>> >> >> now that it's a service to manage namespaces, its core should be
>> >> >> much
>> >> >> more constrained in how it pollutes global structures.
>> >> >>
>> >> >> Unfortunately as Thomas said, cleaning this up takes time and isn't
>> >> >> at
>> >> >> the top of the priority list.
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> f
>> >> >>
>> >> >> On Sun, Jun 30, 2013 at 8:31 AM, Thomas Kluyver <takowl at gmail.com>
>> >> >> wrote:
>> >> >>> Various parts of the IPython machinery assume that there's only one
>> >> >>> instance
>> >> >>> in a given process, and would break with multiple IPython shells in
>> >> >>> a
>> >> >>> single
>> >> >>> process. We're slowly moving away from that, but it's not high
>> >> >>> priority for
>> >> >>> us.
>> >> >>>
>> >> >>> Thomas
>> >> >>>
>> >> >>>
>> >> >>> On 30 June 2013 05:52, Aaron Meurer <asmeurer at gmail.com> wrote:
>> >> >>>>
>> >> >>>> In IPython dev, if you try to start an IPython instance from
>> >> >>>> within
>> >> >>>> an
>> >> >>>> IPython instance, it raises MultipleInstanceError. What is the
>> >> >>>> purpose
>> >> >>>> of this? It breaks a use-case of mine, which is to start PuDB from
>> >> >>>> within IPython, then to open the IPython interpreter from within
>> >> >>>> PuDB
>> >> >>>> while debugging?  Is the new way to do things to somehow reattach
>> >> >>>> to
>> >> >>>> the same IPython instance?
>> >> >>>>
>> >> >>>> I've CCd the PuDB list.
>> >> >>>>
>> >> >>>> Aaron Meurer
>> >> >>>> _______________________________________________
>> >> >>>> IPython-dev mailing list
>> >> >>>> IPython-dev at scipy.org
>> >> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> IPython-dev mailing list
>> >> >>> IPython-dev at scipy.org
>> >> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Fernando Perez (@fperez_org; http://fperez.org)
>> >> >> fperez.net-at-gmail: mailing lists only (I ignore this when
>> >> >> swamped!)
>> >> >> fernando.perez-at-berkeley: contact me here for any direct mail
>> >> >> _______________________________________________
>> >> >> IPython-dev mailing list
>> >> >> IPython-dev at scipy.org
>> >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> > _______________________________________________
>> >> > IPython-dev mailing list
>> >> > IPython-dev at scipy.org
>> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >>
>> >> --
>> >> Fernando Perez (@fperez_org; http://fperez.org)
>> >> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>> >> fernando.perez-at-berkeley: contact me here for any direct mail
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From mail at telenczuk.pl  Mon Jul  1 08:10:26 2013
From: mail at telenczuk.pl (Bartosz)
Date: Mon, 01 Jul 2013 14:10:26 +0200
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
Message-ID: <51D171B2.1010603@telenczuk.pl>

Hi Puneeth,

Indeed, it is similar to my userscript. However, I prefer to render the 
notebook on a separate page - leaves more space for graphs.

One additional points - your script is triggered only on reload (at 
least in my case). It means that when you choose a file from GH file 
browser, it won't be automatically render. The problem is that the 
clicking triggers an AJAX request. The workaround is to trigger on a DOM 
change event as implemented in my script.

@Fernando: Do you think one of the scripts could be linked on NbViewer 
page? I could also write a short "instructions" for it.

Cheers,

Bartosz

On 06/28/2013 09:29 PM, Puneeth Chaganti wrote:
> Hi,
>
> I hadn't really looked at this Userscript, until I wanted to send this
> email.  It may have saved me some work.
>
> I've hacked up a userscript[0] that "renders" notebook files inside
> GitHub, inspired by this[1] post on GitHub's blog, about rendering
> geojson files.
>
> I started with a version that loaded all the required Notebook js
> files, and got something working (just for raw pages), but getting the
> right CSS would've been a pain.  After some poking around, I wrote a
> userscript to replace the required elements on the page with an iframe
> that renders the corresponding nbviewer url.  Some of you may find it
> useful, though it probably doesn't save you much more than a click or
> two. :)
>
> The script can be installed in Firefox using Greasemonkey, and in
> Chrom(ium|e) using Tampermonkey.
>
> [0] - https://gist.github.com/punchagan/5882879
> [1] -https://github.com/blog/1541-geojson-rendering-improvements
>
> --
> Puneeth
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From gmbecker at ucdavis.edu  Mon Jul  1 14:20:42 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Mon, 1 Jul 2013 11:20:42 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses in
	IPy. Notebook
Message-ID: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>

Hey all,

As part of my research into capturing the data analysis process in
documents, I have been working on some extensions to the IPython Notebook
as a way of implementing proof-of-concepts for certain ideas my advisors
and I have had. I think the subscribers of this list might find them
interesting and would love to hear what you guys think.

I have posted a screencast showcasing and explaining the work here:
https://www.youtube.com/watch?v=iQPagwhad_8 and will "briefly" describe it
in text below.

I've implemented 3 fundamentally different new cell types in a fork of the
IPython codebase: interactive code cells, task cells, and alternatives set
cells. To be clear, my goal is absolutely not to replace IPython Notebook.
I am simply leveraging their excellent core application to explore some new
ideas about representing data analyses in documents. Descriptions of the
cell types follow.

*interactivecode cells:* Interactive code cells are code cells which have
additional information attached to them allowing them to render a UI
control which controls one (or more) values within the code and re-executes
the code when the control is used to change the value. Example: controlling
the bandwidth of a kernel regression estimator via a slider.

*task cells*: Task cells are cells that can contain other cells (including
nested task cells or altset cells). They are used to group conceptually
linked content and can be executed in order to execute all the cells they
contain with a single command. They are primarily for organization.
Example:  the data cleaning task during a data analysis would likely
contain multiple code and exposition blocks which fit conceptually within a
single goal.

*altset cells: *Alternatives set (altset) cells represent a point in an
analysis where multiple approaches were tried before the analyst decided on
a final strategy. An altset contains two or more branches representing
these different approaches, only one of which can be active at a time. This
allows an analyst to capture the entire research process in their IPython
notebook in a structurally meaningful way, rather than just the final
approach.

Finally, when the structure of a document actually contains information
about the research process, there are a bunch of really cool things we can
do when querying, processing, executing, and rendering the document which
are difficult or impossible without this extra information. This email has
already gotten quite long, however, so I will leave discussion of those to
another time.

I'd love to hear what people think of these concepts, so please share your
thoughts.

Thanks for reading and thanks to the IPython core team for their great work.
~G


Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130701/f2a17f3f/attachment.html>

From benjaminrk at gmail.com  Mon Jul  1 15:59:12 2013
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 1 Jul 2013 12:59:12 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
Message-ID: <CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>

Very interesting!

I think your interactive code cells and task cells will be covered by plans
we already have in the works.

- with the rich widget API that is the primary milestone for 2.0, you
should be able to do all of the GUI-type controls via rich display data and
kernel callbacks.
- task cells encapsulating a segment of the notebook should be covered by
the plan to expose UI based on heading cells, to be able to treat
'sections' of a notebook as discrete entities (cut/copy/move/hide, tab
view, run, etc.)

The altset is an interesting idea that we don't have a model for. My guess
is that you will actually be able to represent this with an interactive
widget via the new API, but I'm not certain as we haven't built that yet.


On Mon, Jul 1, 2013 at 11:20 AM, Gabriel Becker <gmbecker at ucdavis.edu>wrote:

> Hey all,
>
> As part of my research into capturing the data analysis process in
> documents, I have been working on some extensions to the IPython Notebook
> as a way of implementing proof-of-concepts for certain ideas my advisors
> and I have had. I think the subscribers of this list might find them
> interesting and would love to hear what you guys think.
>
> I have posted a screencast showcasing and explaining the work here:
> https://www.youtube.com/watch?v=iQPagwhad_8 and will "briefly" describe
> it in text below.
>
> I've implemented 3 fundamentally different new cell types in a fork of the
> IPython codebase: interactive code cells, task cells, and alternatives set
> cells. To be clear, my goal is absolutely not to replace IPython Notebook.
> I am simply leveraging their excellent core application to explore some new
> ideas about representing data analyses in documents. Descriptions of the
> cell types follow.
>
> *interactivecode cells:* Interactive code cells are code cells which have
> additional information attached to them allowing them to render a UI
> control which controls one (or more) values within the code and re-executes
> the code when the control is used to change the value. Example: controlling
> the bandwidth of a kernel regression estimator via a slider.
>
> *task cells*: Task cells are cells that can contain other cells
> (including nested task cells or altset cells). They are used to group
> conceptually linked content and can be executed in order to execute all the
> cells they contain with a single command. They are primarily for
> organization. Example:  the data cleaning task during a data analysis would
> likely contain multiple code and exposition blocks which fit conceptually
> within a single goal.
>
> *altset cells: *Alternatives set (altset) cells represent a point in an
> analysis where multiple approaches were tried before the analyst decided on
> a final strategy. An altset contains two or more branches representing
> these different approaches, only one of which can be active at a time. This
> allows an analyst to capture the entire research process in their IPython
> notebook in a structurally meaningful way, rather than just the final
> approach.
>
> Finally, when the structure of a document actually contains information
> about the research process, there are a bunch of really cool things we can
> do when querying, processing, executing, and rendering the document which
> are difficult or impossible without this extra information. This email has
> already gotten quite long, however, so I will leave discussion of those to
> another time.
>
> I'd love to hear what people think of these concepts, so please share your
> thoughts.
>
> Thanks for reading and thanks to the IPython core team for their great
> work.
> ~G
>
>
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130701/f931486f/attachment.html>

From hroe.lee at gmail.com  Mon Jul  1 16:40:18 2013
From: hroe.lee at gmail.com (Hyungro Lee)
Date: Mon, 1 Jul 2013 13:40:18 -0700
Subject: [IPython-dev] IPython.core.display or IPython.display ?
Message-ID: <CAMjgkSf0wh8Fw3-0ZBcvTN764b1PVyWYwG1LmZigYhypL3CJXA@mail.gmail.com>

Hi,

This might be a stupid question but I experienced that my IPython notebook
couldn't find the display module when I imported it like this:

> from IPython.display import HTML

---------------------------------------------------------------------------
> ImportError Traceback (most recent call last)
> /home//.ipython/profile_default/<ipython-input-8-686fc631ef0a> in <module>
> ()
> ----> 1 from IPython.display import HTML
>
> ImportError: No module named display


So, I tried it again with core path like this :

> from IPython.core.display import HTML


It seems working fine with core path.

My installation of IPython is 0.12.1 and used command sudo apt-get install
ipython-notebook as guided on ipython.org/install.html

The core package has been merged to the ipython 1.0 main package, I guess?
Any guidance would be appreciated.

Thanks,
Hyungro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130701/3dd8c5bf/attachment.html>

From takowl at gmail.com  Mon Jul  1 16:53:05 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 1 Jul 2013 21:53:05 +0100
Subject: [IPython-dev] IPython.core.display or IPython.display ?
In-Reply-To: <CAMjgkSf0wh8Fw3-0ZBcvTN764b1PVyWYwG1LmZigYhypL3CJXA@mail.gmail.com>
References: <CAMjgkSf0wh8Fw3-0ZBcvTN764b1PVyWYwG1LmZigYhypL3CJXA@mail.gmail.com>
Message-ID: <CAOvn4qi8690mn8Kh-NpPQvNZ+okPKVzj1AuOjaz0+AVdLaE8uw@mail.gmail.com>

On 1 July 2013 21:40, Hyungro Lee <hroe.lee at gmail.com> wrote:

> ----> 1 from IPython.display import HTML
>>
>> ImportError: No module named display
>
>
> So, I tried it again with core path like this :
>
>> from IPython.core.display import HTML
>
>
> It seems working fine with core path.
>

Yes, 0.12.x won't have IPython.display. That was introduced later, pulling
together things from IPython.core.display and IPython.lib.display. You
should be able to carry on using the same things from IPython.core.display.

We recommend everyone upgrades from 0.12 now anyway. If you're using
Ubuntu, there's a PPA with the latest stable version in here:
https://launchpad.net/~jtaylor/+archive/ipython

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130701/cfa7b956/attachment.html>

From benjaminrk at gmail.com  Mon Jul  1 17:26:29 2013
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 1 Jul 2013 14:26:29 -0700
Subject: [IPython-dev] IPython.core.display or IPython.display ?
In-Reply-To: <CAMjgkSf0wh8Fw3-0ZBcvTN764b1PVyWYwG1LmZigYhypL3CJXA@mail.gmail.com>
References: <CAMjgkSf0wh8Fw3-0ZBcvTN764b1PVyWYwG1LmZigYhypL3CJXA@mail.gmail.com>
Message-ID: <CAHNn8BVxLqVw8x2YvLDCC58hYp=wHdZkwySnqTWTzhou03QM2g@mail.gmail.com>

display classes are defined in a few places (core.display and lib.display),
but they should be imported from IPython.display.  I think IPython.display
may be new in 0.13 from last year.


On Mon, Jul 1, 2013 at 1:40 PM, Hyungro Lee <hroe.lee at gmail.com> wrote:

> Hi,
>
> This might be a stupid question but I experienced that my IPython notebook
> couldn't find the display module when I imported it like this:
>
>> from IPython.display import HTML
>
> ---------------------------------------------------------------------------
>> ImportError Traceback (most recent call last)
>> /home//.ipython/profile_default/<ipython-input-8-686fc631ef0a> in
>> <module>()
>> ----> 1 from IPython.display import HTML
>>
>> ImportError: No module named display
>
>
> So, I tried it again with core path like this :
>
>> from IPython.core.display import HTML
>
>
> It seems working fine with core path.
>
> My installation of IPython is 0.12.1 and used command sudo apt-get install
> ipython-notebook as guided on ipython.org/install.html
>
> The core package has been merged to the ipython 1.0 main package, I guess?
> Any guidance would be appreciated.
>
> Thanks,
> Hyungro
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130701/15332850/attachment.html>

From hroe.lee at gmail.com  Mon Jul  1 18:04:54 2013
From: hroe.lee at gmail.com (Hyungro Lee)
Date: Mon, 1 Jul 2013 15:04:54 -0700
Subject: [IPython-dev] IPython.core.display or IPython.display ?
In-Reply-To: <CAHNn8BVxLqVw8x2YvLDCC58hYp=wHdZkwySnqTWTzhou03QM2g@mail.gmail.com>
References: <CAMjgkSf0wh8Fw3-0ZBcvTN764b1PVyWYwG1LmZigYhypL3CJXA@mail.gmail.com>
	<CAHNn8BVxLqVw8x2YvLDCC58hYp=wHdZkwySnqTWTzhou03QM2g@mail.gmail.com>
Message-ID: <CAMjgkSc4=nJTGap_YVTp=f8KKmkz=HV+M=t8X_8UHd9-6kDZqA@mail.gmail.com>

Thanks for the comments.

When I was following the instruction on ipython.org, I thought I was
downloading 0.13.2 as indicated, but apparently I installed 0.12.1. It
might be good if ppa added to the page.


On Mon, Jul 1, 2013 at 2:26 PM, MinRK <benjaminrk at gmail.com> wrote:

> display classes are defined in a few places (core.display and
> lib.display), but they should be imported from IPython.display.  I think
> IPython.display may be new in 0.13 from last year.
>
>
> On Mon, Jul 1, 2013 at 1:40 PM, Hyungro Lee <hroe.lee at gmail.com> wrote:
>
>> Hi,
>>
>> This might be a stupid question but I experienced that my IPython
>> notebook couldn't find the display module when I imported it like this:
>>
>>> from IPython.display import HTML
>>
>>
>>> ---------------------------------------------------------------------------
>>> ImportError Traceback (most recent call last)
>>> /home//.ipython/profile_default/<ipython-input-8-686fc631ef0a> in
>>> <module>()
>>> ----> 1 from IPython.display import HTML
>>>
>>> ImportError: No module named display
>>
>>
>> So, I tried it again with core path like this :
>>
>>> from IPython.core.display import HTML
>>
>>
>> It seems working fine with core path.
>>
>> My installation of IPython is 0.12.1 and used command sudo apt-get
>> install ipython-notebook as guided on ipython.org/install.html
>>
>> The core package has been merged to the ipython 1.0 main package, I
>> guess? Any guidance would be appreciated.
>>
>> Thanks,
>> Hyungro
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130701/7302a200/attachment.html>

From fperez.net at gmail.com  Mon Jul  1 20:04:14 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 1 Jul 2013 17:04:14 -0700
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <51D171B2.1010603@telenczuk.pl>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
Message-ID: <CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>

On Mon, Jul 1, 2013 at 5:10 AM, Bartosz <mail at telenczuk.pl> wrote:
> Hi Puneeth,
>
> Indeed, it is similar to my userscript. However, I prefer to render the
> notebook on a separate page - leaves more space for graphs.
>
> One additional points - your script is triggered only on reload (at
> least in my case). It means that when you choose a file from GH file
> browser, it won't be automatically render. The problem is that the
> clicking triggers an AJAX request. The workaround is to trigger on a DOM
> change event as implemented in my script.
>
> @Fernando: Do you think one of the scripts could be linked on NbViewer
> page? I could also write a short "instructions" for it.

That sounds like a good idea, but it's up to Matthias, who is our
nbviewer master :)

Cheers,

f


From asmeurer at gmail.com  Mon Jul  1 20:22:23 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 1 Jul 2013 19:22:23 -0500
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
	<CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
Message-ID: <CAKgW=6JyF3-505VL8GCDO3ihOGRvA7whYX1RAoSrdO8Kq4VYOw@mail.gmail.com>

Maybe someone could make it a browser extension? I wish GitHub would
just open up the API for this sort of thing.

Aaron Meurer

On Mon, Jul 1, 2013 at 7:04 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Mon, Jul 1, 2013 at 5:10 AM, Bartosz <mail at telenczuk.pl> wrote:
>> Hi Puneeth,
>>
>> Indeed, it is similar to my userscript. However, I prefer to render the
>> notebook on a separate page - leaves more space for graphs.
>>
>> One additional points - your script is triggered only on reload (at
>> least in my case). It means that when you choose a file from GH file
>> browser, it won't be automatically render. The problem is that the
>> clicking triggers an AJAX request. The workaround is to trigger on a DOM
>> change event as implemented in my script.
>>
>> @Fernando: Do you think one of the scripts could be linked on NbViewer
>> page? I could also write a short "instructions" for it.
>
> That sounds like a good idea, but it's up to Matthias, who is our
> nbviewer master :)
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From bussonniermatthias at gmail.com  Tue Jul  2 03:05:04 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 2 Jul 2013 09:05:04 +0200
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
	<CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
Message-ID: <7120088A-4FC4-4807-A820-D10147AAA654@gmail.com>


Le 2 juil. 2013 ? 02:04, Fernando Perez a ?crit :

> On Mon, Jul 1, 2013 at 5:10 AM, Bartosz <mail at telenczuk.pl> wrote:
>> Hi Puneeth,
>> 
>> Indeed, it is similar to my userscript. However, I prefer to render the
>> notebook on a separate page - leaves more space for graphs.
>> 
>> One additional points - your script is triggered only on reload (at
>> least in my case). It means that when you choose a file from GH file
>> browser, it won't be automatically render. The problem is that the
>> clicking triggers an AJAX request. The workaround is to trigger on a DOM
>> change event as implemented in my script.
>> 
>> @Fernando: Do you think one of the scripts could be linked on NbViewer
>> page? I could also write a short "instructions" for it.
> 
> That sounds like a good idea, but it's up to Matthias, who is our
> nbviewer master :)

Yes, please, open an issue on nbviewer tracker. 
I really can't work on it right now, but will later. There are a
 lot of ehencement ideas on the tracker I haven't respond to, but which are good. 

I will also migrate nbviewer to nbconvert-ipython-1.0 first.
-- 
Matthias



> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From mail at telenczuk.pl  Tue Jul  2 08:25:17 2013
From: mail at telenczuk.pl (Bartosz)
Date: Tue, 02 Jul 2013 14:25:17 +0200
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <7120088A-4FC4-4807-A820-D10147AAA654@gmail.com>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
	<CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
	<7120088A-4FC4-4807-A820-D10147AAA654@gmail.com>
Message-ID: <51D2C6AD.8070705@telenczuk.pl>

Done. The issue is here:

https://github.com/ipython/nbviewer/issues/93

Please let me know if there is anything I can help with.

Cheers,

Bartosz

> Yes, please, open an issue on nbviewer tracker.
> I really can't work on it right now, but will later. There are a
>   lot of ehencement ideas on the tracker I haven't respond to, but which are good.
>
> I will also migrate nbviewer to nbconvert-ipython-1.0 first.
>


From mail at telenczuk.pl  Tue Jul  2 08:29:23 2013
From: mail at telenczuk.pl (Bartosz)
Date: Tue, 02 Jul 2013 14:29:23 +0200
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <CAKgW=6JyF3-505VL8GCDO3ihOGRvA7whYX1RAoSrdO8Kq4VYOw@mail.gmail.com>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
	<CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
	<CAKgW=6JyF3-505VL8GCDO3ihOGRvA7whYX1RAoSrdO8Kq4VYOw@mail.gmail.com>
Message-ID: <51D2C7A3.9020800@telenczuk.pl>

> Maybe someone could make it a browser extension?

It should not be too difficult, especially with a library like Kango:

http://kangoextensions.com/

Bartek


From mdroe at stsci.edu  Tue Jul  2 12:39:27 2013
From: mdroe at stsci.edu (Michael Droettboom)
Date: Tue, 2 Jul 2013 12:39:27 -0400
Subject: [IPython-dev] matplotlib user survey 2013
Message-ID: <51D3023F.3020108@stsci.edu>

[Apologies for cross-posting]

The matplotlib developers want to hear from you!

We are conducting a user survey to determine how and where matplotlib is 
being used in order to focus its further development.

This should only take a couple of minutes.  To fill it out, visit:

https://docs.google.com/spreadsheet/viewform?fromEmail=true&formkey=dHpQS25pcTZIRWdqX0pNckNSU01sMHc6MQ 
<https://docs.google.com/spreadsheet/viewform?fromEmail=true&formkey=dHpQS25pcTZIRWdqX0pNckNSU01sMHc6MQ>

Please forward to your colleagues, particularly those who don't read 
these mailing lists.

Cheers,
Michael Droettboom, and the matplotlib team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130702/0594d117/attachment.html>

From punchagan at gmail.com  Tue Jul  2 14:42:34 2013
From: punchagan at gmail.com (Puneeth Chaganti)
Date: Wed, 3 Jul 2013 00:12:34 +0530
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <51D171B2.1010603@telenczuk.pl>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
Message-ID: <CALnw1fRCj6B0rL3eOVPmLqdCUzykH36BUhRptFsMaB9iSkQ+Yw@mail.gmail.com>

Hi Bartosz,

Thanks for trying my script out.

On Mon, Jul 1, 2013 at 5:40 PM, Bartosz <mail at telenczuk.pl> wrote:
> Hi Puneeth,
>
> Indeed, it is similar to my userscript. However, I prefer to render the
> notebook on a separate page - leaves more space for graphs.

I have a pretty wide screen, and I find my approach quite convenient.
To each, his own, I guess. :-)

> One additional points - your script is triggered only on reload (at
> least in my case). It means that when you choose a file from GH file
> browser, it won't be automatically render. The problem is that the
> clicking triggers an AJAX request. The workaround is to trigger on a DOM
> change event as implemented in my script.

Thanks for pointing this out!  I hadn't run into this use case.  I now
fixed the script, but I could get away with a relatively "careless"
approach compared to yours.


From gmbecker at ucdavis.edu  Tue Jul  2 15:56:31 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Tue, 2 Jul 2013 12:56:31 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
Message-ID: <CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>

Min (or Ben, you didn't sign your mail so I'm not sure what you go by) et
al.,

I think I may not have done a good job describing the goals and potential
uses of some of the features I've implemented. Please allow me to try
again, after which I will respond to your specific comments.

First off, I am thinking about notebooks in the context of data analysis
scripts. This is not the only use for IPython notebooks, but I think you'll
agree it is a major and important one.

Imagine if authors could easily create documents allowing themselves,
reviewers, students, and other researchers to see, explore, understand, and
reproduce the *data analysis process itself*, rather than just the final
results generated at the end of a long and complex process.

Data analyses are not simple linear/sequential affairs. It is much more apt
to view the actions taken during research/data analysis within the
framework of a directed graph. For any non-trivial analysis, there is no
single block of (non-repetitious) code which can encompass the *research
process* which lead to the generation of the final results.

Generally a large amount of code and parameter configurations exist which
contributed substantially to the analysis but would not appear in the
linear block of code which generates the final computational results of the
study from the raw data. By relaxing the linear/sequential structural
assumption on notebooks, we can gain the ability to record, represent and
explore the larger research process *without losing any existing
functionality* (nodes connected along a straight line is itself a special
case of a directed graph).

As an example see the attached screenshots.

This notebook represents a quite trivial "data analysis" with three steps:
read the data, clean the data, and create a plot of the data. As with any
real dataset, however, the analyst has numerous options in exactly how to
clean the data. In particular, this notebook records 3 particular choices
the analyst makes: whether to remove rows with missing data, and
whether/how to trim the data based on values of the two variables being
plotted (price and lsqft). These three simple choices combine to create 18
distinct data cleaning strategies.

As you can see from the screenshots the resulting plot looks substantially
different depending on the strategy that is chosen. This is a simplified,
concrete example of the implications  that choices often not explicitly
reflected in the final code/discussion can have on analysis results. With
non-sequential notebooks these choices can be represented by authors and
reproduced/assessed by readers and reviewers.

I hope that this offered a clearer picture of the goals and motivations
behind what I'm doing. In terms of your specific comments, thank you for
responding. Please see my inline comments below:

On Mon, Jul 1, 2013 at 12:59 PM, MinRK <benjaminrk at gmail.com> wrote:

> Very interesting!
>
> I think your interactive code cells and task cells will be covered by
> plans we already have in the works.
>
> - with the rich widget API that is the primary milestone for 2.0, you
> should be able to do all of the GUI-type controls via rich display data and
> kernel callbacks.
>

I had heard that but had already implemented the cell type myself. If I
understand correctly this will be possible with your 2.0, which is very
exciting. The one thing that I feel is very important here is that the code
which actually resides in the cell be identical between interactive and
non-interactive versions. If that can/will be the case then it seems like
you guys will have met this (perceived, by me) need.


> - task cells encapsulating a segment of the notebook should be covered by
> the plan to expose UI based on heading cells, to be able to treat
> 'sections' of a notebook as discrete entities (cut/copy/move/hide, tab
> view, run, etc.)
>

I'm not as sure about this one. Task cells can be nested within other task
cells, will this be true of how you treat heading cell sections?

Also, it seems like the plan is to get groups of cells to act like a single
cell. With the task cell (you could easily call them section cells, or
whathaveyou) approach all of what you're describe comes for free based on
machinery you guys have already implemented. I haven't been a party to your
discussions, so maybe I'm missing something but I'm not clear what the
benefit implementing new machinery to simulate (a special case of) complex
structured notebooks is over simply supporting the actual structure and
getting the desired functionality for free along with a lot more.


>
> The altset is an interesting idea that we don't have a model for. My guess
> is that you will actually be able to represent this with an interactive
> widget via the new API, but I'm not certain as we haven't built that yet.
>

Here I disagree that this would be the right approach even if possible. I
might be able to hack something together using the future API that I could
insert into a sequential document which will render it *as if *it had a
branching structure, though this would require cells/controls to be able to
affect the rendering of ostensibly unrelated cells which would be a bit
odd. Even if it is possible, however, rendering is not the crux of the
issue.

There are other things which are natural to want to do given a
non-sequential document. We will want generate a linear article-like view
(pdf) that is a rendering of a particular path through the document. We
might also want to add the concepts of terminal branches (important things
the analyst tried, but which don't fit back into the rest of the flow) or
assertions regarding particular combinations of branch choices at different
points in the document which must or cannot be made together.

These things are relatively straightforward to do if we have access to the
actual structure but much more complicated and difficult if the structure
is simulated solely during rendering. Furthermore, if we did find a way to
implement them we would be working very hard to simulate supporting
structured documents without actually supporting them (which is actually
much easier).

I'm looking forward to continued dialogue about these ideas with you, the
rest of the IPython team, and the IPython Notebook community at large.

Thanks,
~G


>
>
> On Mon, Jul 1, 2013 at 11:20 AM, Gabriel Becker <gmbecker at ucdavis.edu>wrote:
>
>> Hey all,
>>
>> As part of my research into capturing the data analysis process in
>> documents, I have been working on some extensions to the IPython Notebook
>> as a way of implementing proof-of-concepts for certain ideas my advisors
>> and I have had. I think the subscribers of this list might find them
>> interesting and would love to hear what you guys think.
>>
>> I have posted a screencast showcasing and explaining the work here:
>> https://www.youtube.com/watch?v=iQPagwhad_8 and will "briefly" describe
>> it in text below.
>>
>> I've implemented 3 fundamentally different new cell types in a fork of
>> the IPython codebase: interactive code cells, task cells, and alternatives
>> set cells. To be clear, my goal is absolutely not to replace IPython
>> Notebook. I am simply leveraging their excellent core application to
>> explore some new ideas about representing data analyses in documents.
>> Descriptions of the cell types follow.
>>
>> *interactivecode cells:* Interactive code cells are code cells which
>> have additional information attached to them allowing them to render a UI
>> control which controls one (or more) values within the code and re-executes
>> the code when the control is used to change the value. Example: controlling
>> the bandwidth of a kernel regression estimator via a slider.
>>
>> *task cells*: Task cells are cells that can contain other cells
>> (including nested task cells or altset cells). They are used to group
>> conceptually linked content and can be executed in order to execute all the
>> cells they contain with a single command. They are primarily for
>> organization. Example:  the data cleaning task during a data analysis would
>> likely contain multiple code and exposition blocks which fit conceptually
>> within a single goal.
>>
>> *altset cells: *Alternatives set (altset) cells represent a point in an
>> analysis where multiple approaches were tried before the analyst decided on
>> a final strategy. An altset contains two or more branches representing
>> these different approaches, only one of which can be active at a time. This
>> allows an analyst to capture the entire research process in their IPython
>> notebook in a structurally meaningful way, rather than just the final
>> approach.
>>
>> Finally, when the structure of a document actually contains information
>> about the research process, there are a bunch of really cool things we can
>> do when querying, processing, executing, and rendering the document which
>> are difficult or impossible without this extra information. This email has
>> already gotten quite long, however, so I will leave discussion of those to
>> another time.
>>
>> I'd love to hear what people think of these concepts, so please share
>> your thoughts.
>>
>> Thanks for reading and thanks to the IPython core team for their great
>> work.
>> ~G
>>
>>
>> Gabriel Becker
>> Graduate Student
>> Statistics Department
>> University of California, Davis
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130702/057e108e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: BranchingAnalysisNotebook.png
Type: image/png
Size: 255859 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130702/057e108e/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: BranchingAnalysisNotebook2.png
Type: image/png
Size: 253708 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130702/057e108e/attachment-0001.png>

From ondrej.certik at gmail.com  Tue Jul  2 18:29:05 2013
From: ondrej.certik at gmail.com (=?UTF-8?B?T25kxZllaiDEjGVydMOtaw==?=)
Date: Tue, 2 Jul 2013 16:29:05 -0600
Subject: [IPython-dev] Github - IPy Notebook button
In-Reply-To: <51D2C6AD.8070705@telenczuk.pl>
References: <51C2020E.20503@telenczuk.pl>
	<CAOvn4qg5eSKonidtRijEW3UhDiR4DmDrvVnkdbnutPjxW6TiRg@mail.gmail.com>
	<7bd974c88dce76ac2374363e26583c07@webmail.webfaction.com>
	<CALnw1fTVwYZWHOwKT6dzvEJDck-VdkMi2g=y2PL5wvYAtq0Y_w@mail.gmail.com>
	<51D171B2.1010603@telenczuk.pl>
	<CAHAreOrMJgD2v7B-7ZtzWyK0EL3_ct+qesJx7dHDkE5wxveqsg@mail.gmail.com>
	<7120088A-4FC4-4807-A820-D10147AAA654@gmail.com>
	<51D2C6AD.8070705@telenczuk.pl>
Message-ID: <CADDwiVD+RdWOhCTGy3c5xBWn9gRd+mELnWRRP9u5EVU5wukycA@mail.gmail.com>

On Tue, Jul 2, 2013 at 6:25 AM, Bartosz <mail at telenczuk.pl> wrote:
> Done. The issue is here:
>
> https://github.com/ipython/nbviewer/issues/93
>
> Please let me know if there is anything I can help with.

Thank you! I wish there was a default button in ipython where you just
click "publish" and it will appear on nbviewer. Fiddling with these
scripts (on all my computers, which currently is at east 3) is quite a
pain. I did start a PR to implement exactly that:

https://github.com/ipython/ipython/pull/2504

But I don't currently have time to finish it, plus based on my
discussion with Brian, IPython itself might not be the best place to
implement github specific features, but rather it should be
implemented as an ipython extension somehow.

Ondrej


From benjaminrk at gmail.com  Tue Jul  2 23:44:14 2013
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 2 Jul 2013 20:44:14 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
Message-ID: <CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>

On Tue, Jul 2, 2013 at 12:56 PM, Gabriel Becker <gmbecker at ucdavis.edu>wrote:

> Min (or Ben, you didn't sign your mail so I'm not sure what you go by) et
> al.,
>
> I think I may not have done a good job describing the goals and potential
> uses of some of the features I've implemented. Please allow me to try
> again, after which I will respond to your specific comments.
>
> First off, I am thinking about notebooks in the context of data analysis
> scripts. This is not the only use for IPython notebooks, but I think you'll
> agree it is a major and important one.
>
> Imagine if authors could easily create documents allowing themselves,
> reviewers, students, and other researchers to see, explore, understand, and
> reproduce the *data analysis process itself*, rather than just the final
> results generated at the end of a long and complex process.
>
> Data analyses are not simple linear/sequential affairs. It is much more
> apt to view the actions taken during research/data analysis within the
> framework of a directed graph. For any non-trivial analysis, there is no
> single block of (non-repetitious) code which can encompass the *research
> process* which lead to the generation of the final results.
>
> Generally a large amount of code and parameter configurations exist which
> contributed substantially to the analysis but would not appear in the
> linear block of code which generates the final computational results of the
> study from the raw data. By relaxing the linear/sequential structural
> assumption on notebooks, we can gain the ability to record, represent and
> explore the larger research process *without losing any existing
> functionality* (nodes connected along a straight line is itself a special
> case of a directed graph).
>
> As an example see the attached screenshots.
>
> This notebook represents a quite trivial "data analysis" with three steps:
> read the data, clean the data, and create a plot of the data. As with any
> real dataset, however, the analyst has numerous options in exactly how to
> clean the data. In particular, this notebook records 3 particular choices
> the analyst makes: whether to remove rows with missing data, and
> whether/how to trim the data based on values of the two variables being
> plotted (price and lsqft). These three simple choices combine to create 18
> distinct data cleaning strategies.
>
> As you can see from the screenshots the resulting plot looks substantially
> different depending on the strategy that is chosen. This is a simplified,
> concrete example of the implications  that choices often not explicitly
> reflected in the final code/discussion can have on analysis results. With
> non-sequential notebooks these choices can be represented by authors and
> reproduced/assessed by readers and reviewers.
>
> I hope that this offered a clearer picture of the goals and motivations
> behind what I'm doing. In terms of your specific comments, thank you for
> responding. Please see my inline comments below:
>
> On Mon, Jul 1, 2013 at 12:59 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>> Very interesting!
>>
>> I think your interactive code cells and task cells will be covered by
>> plans we already have in the works.
>>
>> - with the rich widget API that is the primary milestone for 2.0, you
>> should be able to do all of the GUI-type controls via rich display data and
>> kernel callbacks.
>>
>
> I had heard that but had already implemented the cell type myself. If I
> understand correctly this will be possible with your 2.0, which is very
> exciting. The one thing that I feel is very important here is that the code
> which actually resides in the cell be identical between interactive and
> non-interactive versions. If that can/will be the case then it seems like
> you guys will have met this (perceived, by me) need.
>
>
>> - task cells encapsulating a segment of the notebook should be covered by
>> the plan to expose UI based on heading cells, to be able to treat
>> 'sections' of a notebook as discrete entities (cut/copy/move/hide, tab
>> view, run, etc.)
>>
>
> I'm not as sure about this one. Task cells can be nested within other task
> cells, will this be true of how you treat heading cell sections?
>

Yes, headings have levels and can thus express nested hierarchy. The UI
only exposes seven levels, but there's no actual limit to the level of
nesting.


>
> Also, it seems like the plan is to get groups of cells to act like a
> single cell. With the task cell (you could easily call them section cells,
> or whathaveyou) approach all of what you're describe comes for free based
> on machinery you guys have already implemented. I haven't been a party to
> your discussions, so maybe I'm missing something but I'm not clear what the
> benefit implementing new machinery to simulate (a special case of) complex
> structured notebooks is over simply supporting the actual structure and
> getting the desired functionality for free along with a lot more.
>
>
>>
>> The altset is an interesting idea that we don't have a model for. My
>> guess is that you will actually be able to represent this with an
>> interactive widget via the new API, but I'm not certain as we haven't built
>> that yet.
>>
>
> Here I disagree that this would be the right approach even if possible. I
> might be able to hack something together using the future API that I could
> insert into a sequential document which will render it *as if *it had a
> branching structure, though this would require cells/controls to be able to
> affect the rendering of ostensibly unrelated cells which would be a bit
> odd. Even if it is possible, however, rendering is not the crux of the
> issue.
>
> There are other things which are natural to want to do given a
> non-sequential document. We will want generate a linear article-like view
> (pdf) that is a rendering of a particular path through the document. We
> might also want to add the concepts of terminal branches (important things
> the analyst tried, but which don't fit back into the rest of the flow) or
> assertions regarding particular combinations of branch choices at different
> points in the document which must or cannot be made together.
>

This, together with the Student/Teacher/Answer-key model is an approach to
the notebook that is not yet well served, but highly attractive.  There is
resistance to adding new cell types, because there is great inertia in the
cell type part of code (adding or removing cell types is a very significant
operation, with major backward-incompatible side effects).  That's why we
try to express ideas as much as we can through the dynamic output
mechanism, which is much more flexible, generic, and extensible.  It may
well be that new cell types are necessary, but we want to be sure that
existing (and planned) APIs are inadequate before making such a change.

-Min RK


>
> These things are relatively straightforward to do if we have access to the
> actual structure but much more complicated and difficult if the structure
> is simulated solely during rendering. Furthermore, if we did find a way to
> implement them we would be working very hard to simulate supporting
> structured documents without actually supporting them (which is actually
> much easier).
>
> I'm looking forward to continued dialogue about these ideas with you, the
> rest of the IPython team, and the IPython Notebook community at large.
>
> Thanks,
> ~G
>
>
>>
>>
>> On Mon, Jul 1, 2013 at 11:20 AM, Gabriel Becker <gmbecker at ucdavis.edu>wrote:
>>
>>> Hey all,
>>>
>>> As part of my research into capturing the data analysis process in
>>> documents, I have been working on some extensions to the IPython Notebook
>>> as a way of implementing proof-of-concepts for certain ideas my advisors
>>> and I have had. I think the subscribers of this list might find them
>>> interesting and would love to hear what you guys think.
>>>
>>> I have posted a screencast showcasing and explaining the work here:
>>> https://www.youtube.com/watch?v=iQPagwhad_8 and will "briefly" describe
>>> it in text below.
>>>
>>> I've implemented 3 fundamentally different new cell types in a fork of
>>> the IPython codebase: interactive code cells, task cells, and alternatives
>>> set cells. To be clear, my goal is absolutely not to replace IPython
>>> Notebook. I am simply leveraging their excellent core application to
>>> explore some new ideas about representing data analyses in documents.
>>> Descriptions of the cell types follow.
>>>
>>> *interactivecode cells:* Interactive code cells are code cells which
>>> have additional information attached to them allowing them to render a UI
>>> control which controls one (or more) values within the code and re-executes
>>> the code when the control is used to change the value. Example: controlling
>>> the bandwidth of a kernel regression estimator via a slider.
>>>
>>> *task cells*: Task cells are cells that can contain other cells
>>> (including nested task cells or altset cells). They are used to group
>>> conceptually linked content and can be executed in order to execute all the
>>> cells they contain with a single command. They are primarily for
>>> organization. Example:  the data cleaning task during a data analysis would
>>> likely contain multiple code and exposition blocks which fit conceptually
>>> within a single goal.
>>>
>>> *altset cells: *Alternatives set (altset) cells represent a point in an
>>> analysis where multiple approaches were tried before the analyst decided on
>>> a final strategy. An altset contains two or more branches representing
>>> these different approaches, only one of which can be active at a time. This
>>> allows an analyst to capture the entire research process in their IPython
>>> notebook in a structurally meaningful way, rather than just the final
>>> approach.
>>>
>>> Finally, when the structure of a document actually contains information
>>> about the research process, there are a bunch of really cool things we can
>>> do when querying, processing, executing, and rendering the document which
>>> are difficult or impossible without this extra information. This email has
>>> already gotten quite long, however, so I will leave discussion of those to
>>> another time.
>>>
>>> I'd love to hear what people think of these concepts, so please share
>>> your thoughts.
>>>
>>> Thanks for reading and thanks to the IPython core team for their great
>>> work.
>>> ~G
>>>
>>>
>>> Gabriel Becker
>>> Graduate Student
>>> Statistics Department
>>> University of California, Davis
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130702/81f9ca6a/attachment.html>

From bussonniermatthias at gmail.com  Wed Jul  3 04:25:28 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 3 Jul 2013 10:25:28 +0200
Subject: [IPython-dev] Some new cell types for describing data analyses
	in IPy. Notebook
In-Reply-To: <CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
Message-ID: <F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>

Gabriel, 

You screen shot are interesting, 
At some point I played with gridster[1]

and was more or less able to get cell to rearranges, but didn't keep the code. 
You might be interested. 

Keep in mind that the notebook browser-interface we ship is only one possible
frontend that can interpret ipynb files, nothing prevent you to write a
different frontend that display the notebook in a different format. 

This added to the fact that each cell can support arbitrary metadata, you
should be able to arrange preexisting in structure that work together. It might
be a little difficult to do it right now as our javascript is not yet modular
enough to be easily reused, but we are moving toward it.

Right now I thing storing the notebook as a directed graph is problematic in a
few way, the first being that it is incompatible with the fact that people want
to be able to run notebook in a headless manner, which if you add explicit
choice is not possible. This also contradict the fact that the notebook capture
both the input and the output of the computation. As you showed there is
actually 18 different combinations of data analysis, and they are not all
stored in the notebook. 

I really thing this is an interesting project, and reusing only our metadata in
the notebook, you should be able to  simulate it (store the dag in notebook
level metadata, and cell id in cell metadata) then reconstruct the graph when
needed. Keep in mind that at some point we might/will add cell id to the
notebook.

To sum up, I don't think the current JS client is in it's current state the
place to implement such an idea. The Dag for cell order might be an idea for
future notebook format but need to be well thought, and wait for cell IDs.

-- 
Matthias



[1] http://gridster.net/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/405c57ba/attachment.html>

From juanlu001 at gmail.com  Wed Jul  3 09:35:49 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Wed, 03 Jul 2013 15:35:49 +0200
Subject: [IPython-dev] matplotlib user survey 2013
In-Reply-To: <51D3023F.3020108@stsci.edu>
References: <51D3023F.3020108@stsci.edu>
Message-ID: <51D428B5.6040600@gmail.com>

On 07/02/2013 06:39 PM, Michael Droettboom wrote:
> [Apologies for cross-posting]
>
> The matplotlib developers want to hear from you!
>
> We are conducting a user survey to determine how and where matplotlib 
> is being used in order to focus its further development.
>
> This should only take a couple of minutes.  To fill it out, visit:
>
> https://docs.google.com/spreadsheet/viewform?fromEmail=true&formkey=dHpQS25pcTZIRWdqX0pNckNSU01sMHc6MQ 
> <https://docs.google.com/spreadsheet/viewform?fromEmail=true&formkey=dHpQS25pcTZIRWdqX0pNckNSU01sMHc6MQ>
>
> Please forward to your colleagues, particularly those who don't read 
> these mailing lists.

I am using a lot lately the PGF backend, because it works awesome 
integrated in my LaTeX documents - too bad I don't see it in the survey.

Besides, I miss the "I know what a backend is but I don't remember which 
is the deafault one" option :)

Forwarded!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/0c997011/attachment.html>

From lists at hilboll.de  Wed Jul  3 10:27:56 2013
From: lists at hilboll.de (Andreas Hilboll)
Date: Wed, 03 Jul 2013 16:27:56 +0200
Subject: [IPython-dev] IPython Notebook based journal?
Message-ID: <51D434EC.3060502@hilboll.de>

Hi,

I'm very fond of the IPython Notebook and would like to use it as a
general-purpose lab book. I'm also fan of software like rednotebook_,
which is basically a diary with the possiblity to tag and categorize
entries.

Do you see any way of integrating the IPython Notebook in a GUI similar
to rednotebook? I.e. have a calendar, have the possibility to create one
or more IPython Notebooks per day, sort them into categories and/or tag
them, and have the IPython notebook integrated in the main GUI?

Thanks for your thoughts,
Andreas.

_rednotebook: http://rednotebook.sourceforge.net/


From zvoros at gmail.com  Wed Jul  3 10:47:56 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Wed, 03 Jul 2013 16:47:56 +0200
Subject: [IPython-dev] IPython Notebook based journal?
In-Reply-To: <51D434EC.3060502@hilboll.de>
References: <51D434EC.3060502@hilboll.de>
Message-ID: <51D4399C.9050408@gmail.com>

Hi Andreas,

A couple of months ago, I faced the same problem, so I started to 
develop something along these lines. I wanted to have something that is 
usable as a lab notebook, so there are a couple of facilities that 
really target lab use. I haven't taken the whole ipython machinery, but 
I think, it should not be hard to connect to a running ipython kernel. 
My code is not completed yet, but I am working on it currently, and 
there should be something usable in the next couple of weeks. You can 
grab the code from
https://github.com/v923z/nothon

Attached you'll find a notebook that was converted to LaTeX and then pdf.

Let me know your comments, if any.

Cheers,
Zolt?n


On 03/07/13 16:27, Andreas Hilboll wrote:
> Hi,
>
> I'm very fond of the IPython Notebook and would like to use it as a
> general-purpose lab book. I'm also fan of software like rednotebook_,
> which is basically a diary with the possiblity to tag and categorize
> entries.
>
> Do you see any way of integrating the IPython Notebook in a GUI similar
> to rednotebook? I.e. have a calendar, have the possibility to create one
> or more IPython Notebooks per day, sort them into categories and/or tag
> them, and have the IPython notebook integrated in the main GUI?
>
> Thanks for your thoughts,
> Andreas.
>
> _rednotebook: http://rednotebook.sourceforge.net/
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: nothon.pdf
Type: application/pdf
Size: 165514 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/fc288599/attachment.pdf>

From fperez.net at gmail.com  Wed Jul  3 13:44:09 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 3 Jul 2013 10:44:09 -0700
Subject: [IPython-dev] Skipping dev meeting on July 4th?
Message-ID: <CAHAreOoLqOY_D_nKFR3aJ+CvpPZ-jd-awcZDHbXeEFaftjwZTQ@mail.gmail.com>

Hi folks,

given that the 4th of July is a major holiday in the US, where many
(though not all) IPython devs are based, I'm inclined to skip the
meeting tomorrow and postpone to next week.

But if there's a lot of interest in pushing forward tomorrow I'm happy
to reconsider, I'll be home working for the morning and could make it.

Cheers,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From bussonniermatthias at gmail.com  Wed Jul  3 14:02:58 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 3 Jul 2013 20:02:58 +0200
Subject: [IPython-dev] Skipping dev meeting on July 4th?
In-Reply-To: <CAHAreOoLqOY_D_nKFR3aJ+CvpPZ-jd-awcZDHbXeEFaftjwZTQ@mail.gmail.com>
References: <CAHAreOoLqOY_D_nKFR3aJ+CvpPZ-jd-awcZDHbXeEFaftjwZTQ@mail.gmail.com>
Message-ID: <2CCC6CC9-6F78-4761-8093-E3B1CD1852D3@gmail.com>


Le 3 juil. 2013 ? 19:44, Fernando Perez a ?crit :

> Hi folks,
> 
> given that the 4th of July is a major holiday in the US, where many
> (though not all) IPython devs are based, I'm inclined to skip the
> meeting tomorrow and postpone to next week.

No objection for me, I'm not even sure I'll be able to attend.

-- 
M

From ellisonbg at gmail.com  Wed Jul  3 14:20:22 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 3 Jul 2013 11:20:22 -0700
Subject: [IPython-dev] Skipping dev meeting on July 4th?
In-Reply-To: <2CCC6CC9-6F78-4761-8093-E3B1CD1852D3@gmail.com>
References: <CAHAreOoLqOY_D_nKFR3aJ+CvpPZ-jd-awcZDHbXeEFaftjwZTQ@mail.gmail.com>
	<2CCC6CC9-6F78-4761-8093-E3B1CD1852D3@gmail.com>
Message-ID: <CAH4pYpTPENRrknNbgON1dzXQcEQeh=5UxAs0_uP5dgdCs73h9Q@mail.gmail.com>

I will probably miss it...

Brian

On Wed, Jul 3, 2013 at 11:02 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 3 juil. 2013 ? 19:44, Fernando Perez a ?crit :
>
>> Hi folks,
>>
>> given that the 4th of July is a major holiday in the US, where many
>> (though not all) IPython devs are based, I'm inclined to skip the
>> meeting tomorrow and postpone to next week.
>
> No objection for me, I'm not even sure I'll be able to attend.
>
> --
> M
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Wed Jul  3 17:05:06 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 3 Jul 2013 14:05:06 -0700
Subject: [IPython-dev] Skipping dev meeting on July 4th?
In-Reply-To: <CAH4pYpTPENRrknNbgON1dzXQcEQeh=5UxAs0_uP5dgdCs73h9Q@mail.gmail.com>
References: <CAHAreOoLqOY_D_nKFR3aJ+CvpPZ-jd-awcZDHbXeEFaftjwZTQ@mail.gmail.com>
	<2CCC6CC9-6F78-4761-8093-E3B1CD1852D3@gmail.com>
	<CAH4pYpTPENRrknNbgON1dzXQcEQeh=5UxAs0_uP5dgdCs73h9Q@mail.gmail.com>
Message-ID: <CAHAreOrEovsM9DmfXCZyoSqYbj_rUS8E7gsxcFo-ZWkeSSQh9Q@mail.gmail.com>

OK, I also know Min has other stuff going on.  It seems the most
sensible thing to do for everyone is to take this week off, and resume
next week, on July 11th.

Cheers,

f

On Wed, Jul 3, 2013 at 11:20 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> I will probably miss it...
>
> Brian
>
> On Wed, Jul 3, 2013 at 11:02 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>>
>> Le 3 juil. 2013 ? 19:44, Fernando Perez a ?crit :
>>
>>> Hi folks,
>>>
>>> given that the 4th of July is a major holiday in the US, where many
>>> (though not all) IPython devs are based, I'm inclined to skip the
>>> meeting tomorrow and postpone to next week.
>>
>> No objection for me, I'm not even sure I'll be able to attend.
>>
>> --
>> M
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From benjaminrk at gmail.com  Wed Jul  3 19:05:49 2013
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 3 Jul 2013 16:05:49 -0700
Subject: [IPython-dev] Skipping dev meeting on July 4th?
In-Reply-To: <CAHAreOrEovsM9DmfXCZyoSqYbj_rUS8E7gsxcFo-ZWkeSSQh9Q@mail.gmail.com>
References: <CAHAreOoLqOY_D_nKFR3aJ+CvpPZ-jd-awcZDHbXeEFaftjwZTQ@mail.gmail.com>
	<2CCC6CC9-6F78-4761-8093-E3B1CD1852D3@gmail.com>
	<CAH4pYpTPENRrknNbgON1dzXQcEQeh=5UxAs0_uP5dgdCs73h9Q@mail.gmail.com>
	<CAHAreOrEovsM9DmfXCZyoSqYbj_rUS8E7gsxcFo-ZWkeSSQh9Q@mail.gmail.com>
Message-ID: <CAHNn8BU7LGY1p_6zbPTjiE9DRMjEVW=w+cu-MUGwn8EJf7zvzA@mail.gmail.com>

Skipping works for me, but I'll probably be around anyway.


On Wed, Jul 3, 2013 at 2:05 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> OK, I also know Min has other stuff going on.  It seems the most
> sensible thing to do for everyone is to take this week off, and resume
> next week, on July 11th.
>
> Cheers,
>
> f
>
> On Wed, Jul 3, 2013 at 11:20 AM, Brian Granger <ellisonbg at gmail.com>
> wrote:
> > I will probably miss it...
> >
> > Brian
> >
> > On Wed, Jul 3, 2013 at 11:02 AM, Matthias BUSSONNIER
> > <bussonniermatthias at gmail.com> wrote:
> >>
> >> Le 3 juil. 2013 ? 19:44, Fernando Perez a ?crit :
> >>
> >>> Hi folks,
> >>>
> >>> given that the 4th of July is a major holiday in the US, where many
> >>> (though not all) IPython devs are based, I'm inclined to skip the
> >>> meeting tomorrow and postpone to next week.
> >>
> >> No objection for me, I'm not even sure I'll be able to attend.
> >>
> >> --
> >> M
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> > --
> > Brian E. Granger
> > Cal Poly State University, San Luis Obispo
> > bgranger at calpoly.edu and ellisonbg at gmail.com
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/b34fd291/attachment.html>

From gmbecker at ucdavis.edu  Wed Jul  3 21:04:54 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Wed, 3 Jul 2013 18:04:54 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
Message-ID: <CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>

Matthias,

Thanks for your detailed response.


On Wed, Jul 3, 2013 at 1:25 AM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> Gabriel,
>
> You screen shot are interesting,
> At some point I played with gridster[1]
>
> and was more or less able to get cell to rearranges, but didn't keep the
> code.
> You might be interested.
>
> Keep in mind that the notebook browser-interface we ship is only one
> possible
> frontend that can interpret ipynb files, nothing prevent you to write a
> different frontend that display the notebook in a different format.
>
> This added to the fact that each cell can support arbitrary metadata, you
> should be able to arrange preexisting in structure that work together. It
> might
> be a little difficult to do it right now as our javascript is not yet
> modular
> enough to be easily reused, but we are moving toward it.
>

Respectfully, rolling my own frontend for ipynb files given all the work
the IPython team has done on the excellent notebook browser interface would
be an enormous and extremely wasteful duplication of effort. I don't think
its the right way to pursue these features.

Furthermore, if I were going to write an application offering the types of
features I am talking about from scratch, there wouldn't be any good reason
to base it on the unaltered ipynb format, as they don't easily support the
structure required by those features without the additional cell types I
implemented in my forked version.


> Right now I thing storing the notebook as a directed graph is problematic
> in a
> few way,
>

I'm not talking about storing the notebook as an actual directed graph data
structure. There would be benefits to that but its not necessary and it
isn't want I did in my forked version.

The ability to have nested cells (cells which contain other cells) gets us
everything we need structure wise, and is the basis of everything seen in
both the video (other than interactive code cell stuff) and screenshots I
posted. The ipynb file for the notebook pictured in the screenshot looks
exactly like a normal ipynb file except that in the json there are cell
declarations which have a cells field which contains the json descriptions
of the cells contained in that cell.



> the first being that it is incompatible with the fact that people want
> to be able to run notebook in a headless manner, which if you add explicit
> choice is not possible.
>

This isn't the case. The json saved versions of notebooks with branching
remember which version was most recently run. When an altset cell is
executed, it runs only the most recently run (or currently "selected",
though that means something else internally) branch. Thus by doing the
naive thing and looping through all top level cells and executing them, the
currently chosen path through the notebook can easily be run in a headless
environment and give the correct results.


> This also contradict the fact that the notebook capture
> both the input and the output of the computation.
>

I don't really understand what you mean by this. In the JSON representation
of an executed code cell, the input field contains the code, but not any
values of variables used by the code, nor any indication of code which was
run before executing the code cell.

Changing and rerunning an earlier code cell without re-executing the cell
in question can easily invalidate the output stored in the JSON, even
without the concept of branching or choice.



> As you showed there is
> actually 18 different combinations of data analysis, and they are not all
> stored in the notebook.
>

The notebook knows and records which choices were made. There are 18
different combinations of data analysis *but only one was chosen by analyst
as generating the final/most recent result*.

In the case of "publishing" about an analysis the notebook stores the path
most chosen by the analyst, while retaining information about what else he
or she did during the decision process.

In the case of instruction, imagine how much easier it would be to teach
data analysis if the students could actually see what data analysts do,
instead of simply the final method they choose in a particular analysis.



>
> I really thing this is an interesting project, and reusing only our
> metadata in
> the notebook, you should be able to  simulate it (store the dag in notebook
> level metadata, and cell id in cell metadata) then reconstruct the graph
> when
> needed. Keep in mind that at some point we might/will add cell id to the
> notebook.
>
> To sum up, I don't think the current JS client is in it's current state the
> place to implement such an idea. The Dag for cell order might be an idea
> for
> future notebook format but need to be well thought, and wait for cell IDs.
>

I apologize for not being clear. As I said in a response above, the
directed graph idea was intended to be conceptual for thinking about the
documents, not structural for actually storing them.

What I actually did was simply allow cell nesting and change indexing so
that it is with respect to the parent/container (cell or notebook) instead
of always with respect to the notebook. This required some machinery
changes but not too many and it is an extension in the mathematical sense
in that indexing will behave identically to the old system for notebooks
without any nesting while now meaningfully functioning for notebooks with
nesting.

~G

>
> --
> Matthias
>
>
>
> [1] http://gridster.net/
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/9ce08d73/attachment.html>

From ellisonbg at gmail.com  Wed Jul  3 21:59:48 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 3 Jul 2013 18:59:48 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
Message-ID: <CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>

Gabriel,

I watched your video and there are some nice ideas here.  We are not
headed in this direction in terms of *implementation* but I think you
will find that similar *capabilities* will show up in the notebook
over time.  A few comments about the implementation aspects:

First, the benefits of having a notebook be a linear sequence of cells
are massive:

* Simple, simple simple - this makes it very easy to reason about
notebooks in code.  Nesting leads to complexity that is not worth the
cost.
* You can get most of the benefits of nesting without the complexity.
As Min mentioned, there is an implied hirrarchy in the heading cells.
We plan on using that to allow group level actions - show/hide, run
group, move, cut/copy/paste, etc.
* It is not difficult to think about building a proper diff tool for
notebooks.  With nested cells this becomes horrific.
* Getting the UI/UX right for nested cells is a nightmare.  If you
have  ever used Mathematica, you will know this - beyond 1-2 levels,
the UI become unusable.
* Hierarchy puts a significant cognitive load on users

Because of these things we don't have any plans on changing the
notebook document format or notebook UI to allow nested cells.

Second, while it is tempting to generalize the notion of input to
include widgety things, it is more appropriate to put these things in
the output:

* Our output architecture has the notion of multiple representations.
This allows us to build rich widget as you have done, but to still
provide static representations (png, jpg, latex).
* Having the multiple representations of output allows us to build the
rich widgets, but maintain a clear path for converting notebooks to
static formats (pdf, html, word, powerpoint).
* Insisting that input cells are pure code allows you to reason in a
clear manner about how a notebook works = code runs and leads to
output.  That reasoning can be applied in an automate manner by
running notebooks in batch mode, or building a test system based on
them.
* Putting widgets in the input area forces you to do regular
expression matching to replace those variables in the code.  This
limits you to an extremely simple event model where the only possible
event you can know about is substitute the regular expression and run
all the code.  What if you want different UI controls in the browser
to trigger different bits of code in the kernels when different fine
grained events happen.  Making the UI controls live on the Python and
JS side allows us to build this in a natural way.

Th alt-cells you show bring up the issue of providence.  We have some
very initial thoughts about that, but it is way out of scope for the
project right now - we have a plates 10x overfull already.  We will
get there though eventually.

Thanks for sharing your ideas.

PS - for a bit more background about the context of our saying "no" to
this feature request, see this blog post:

http://brianegranger.com/?p=249

I also gave a talk about this at SciPy and will be posting my slides soon.

Cheers,

Brian

On Wed, Jul 3, 2013 at 6:04 PM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
> Matthias,
>
> Thanks for your detailed response.
>
>
> On Wed, Jul 3, 2013 at 1:25 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>>
>> Gabriel,
>>
>> You screen shot are interesting,
>> At some point I played with gridster[1]
>>
>> and was more or less able to get cell to rearranges, but didn't keep the
>> code.
>> You might be interested.
>>
>> Keep in mind that the notebook browser-interface we ship is only one
>> possible
>> frontend that can interpret ipynb files, nothing prevent you to write a
>> different frontend that display the notebook in a different format.
>>
>> This added to the fact that each cell can support arbitrary metadata, you
>> should be able to arrange preexisting in structure that work together. It
>> might
>> be a little difficult to do it right now as our javascript is not yet
>> modular
>> enough to be easily reused, but we are moving toward it.
>
>
> Respectfully, rolling my own frontend for ipynb files given all the work the
> IPython team has done on the excellent notebook browser interface would be
> an enormous and extremely wasteful duplication of effort. I don't think its
> the right way to pursue these features.
>
> Furthermore, if I were going to write an application offering the types of
> features I am talking about from scratch, there wouldn't be any good reason
> to base it on the unaltered ipynb format, as they don't easily support the
> structure required by those features without the additional cell types I
> implemented in my forked version.
>
>>
>> Right now I thing storing the notebook as a directed graph is problematic
>> in a
>> few way,
>
>
> I'm not talking about storing the notebook as an actual directed graph data
> structure. There would be benefits to that but its not necessary and it
> isn't want I did in my forked version.
>
> The ability to have nested cells (cells which contain other cells) gets us
> everything we need structure wise, and is the basis of everything seen in
> both the video (other than interactive code cell stuff) and screenshots I
> posted. The ipynb file for the notebook pictured in the screenshot looks
> exactly like a normal ipynb file except that in the json there are cell
> declarations which have a cells field which contains the json descriptions
> of the cells contained in that cell.
>
>
>>
>> the first being that it is incompatible with the fact that people want
>> to be able to run notebook in a headless manner, which if you add explicit
>> choice is not possible.
>
>
> This isn't the case. The json saved versions of notebooks with branching
> remember which version was most recently run. When an altset cell is
> executed, it runs only the most recently run (or currently "selected",
> though that means something else internally) branch. Thus by doing the naive
> thing and looping through all top level cells and executing them, the
> currently chosen path through the notebook can easily be run in a headless
> environment and give the correct results.
>
>>
>> This also contradict the fact that the notebook capture
>> both the input and the output of the computation.
>
>
> I don't really understand what you mean by this. In the JSON representation
> of an executed code cell, the input field contains the code, but not any
> values of variables used by the code, nor any indication of code which was
> run before executing the code cell.
>
> Changing and rerunning an earlier code cell without re-executing the cell in
> question can easily invalidate the output stored in the JSON, even without
> the concept of branching or choice.
>
>
>>
>> As you showed there is
>> actually 18 different combinations of data analysis, and they are not all
>> stored in the notebook.
>
>
> The notebook knows and records which choices were made. There are 18
> different combinations of data analysis but only one was chosen by analyst
> as generating the final/most recent result.
>
> In the case of "publishing" about an analysis the notebook stores the path
> most chosen by the analyst, while retaining information about what else he
> or she did during the decision process.
>
> In the case of instruction, imagine how much easier it would be to teach
> data analysis if the students could actually see what data analysts do,
> instead of simply the final method they choose in a particular analysis.
>
>
>>
>>
>> I really thing this is an interesting project, and reusing only our
>> metadata in
>> the notebook, you should be able to  simulate it (store the dag in
>> notebook
>> level metadata, and cell id in cell metadata) then reconstruct the graph
>> when
>> needed. Keep in mind that at some point we might/will add cell id to the
>> notebook.
>>
>> To sum up, I don't think the current JS client is in it's current state
>> the
>> place to implement such an idea. The Dag for cell order might be an idea
>> for
>> future notebook format but need to be well thought, and wait for cell IDs.
>
>
> I apologize for not being clear. As I said in a response above, the directed
> graph idea was intended to be conceptual for thinking about the documents,
> not structural for actually storing them.
>
> What I actually did was simply allow cell nesting and change indexing so
> that it is with respect to the parent/container (cell or notebook) instead
> of always with respect to the notebook. This required some machinery changes
> but not too many and it is an extension in the mathematical sense in that
> indexing will behave identically to the old system for notebooks without any
> nesting while now meaningfully functioning for notebooks with nesting.
>
> ~G
>>
>>
>> --
>> Matthias
>>
>>
>>
>> [1] http://gridster.net/
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jakevdp at cs.washington.edu  Wed Jul  3 22:37:01 2013
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Wed, 3 Jul 2013 19:37:01 -0700
Subject: [IPython-dev] nbconvert: template not found
Message-ID: <CACpqBg2BH=cTGoYsaeaL7NZNXdoC7as5OHBjJ4i65WfcagRS8A@mail.gmail.com>

Hi,
I've been working on using the new nbconvert framework from within the
ipython root directory (after doing build_ext --inplace), and it's been
really great!  But when I install IPython system-wide and try to use the
Python nbconvert API from within another directory, I get a Jinja "template
file not found" error.  It seems to go back to the default variables in
IPython/nbconvert/exporters/exporter.py

Any ideas how to best fix this?
   Jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/193d3054/attachment.html>

From ellisonbg at gmail.com  Wed Jul  3 22:44:42 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 3 Jul 2013 19:44:42 -0700
Subject: [IPython-dev] nbconvert: template not found
In-Reply-To: <CACpqBg2BH=cTGoYsaeaL7NZNXdoC7as5OHBjJ4i65WfcagRS8A@mail.gmail.com>
References: <CACpqBg2BH=cTGoYsaeaL7NZNXdoC7as5OHBjJ4i65WfcagRS8A@mail.gmail.com>
Message-ID: <CAH4pYpSZYT+PgCoqjRmFkSC2FdOpP7jMvcSLHHBTa--RhjURVw@mail.gmail.com>

Can you create an issue and ping @jdfreder about this.  He is working
on nbconvert full time right now and will know how to tackle this...

Cheers,

Brian

On Wed, Jul 3, 2013 at 7:37 PM, Jacob Vanderplas
<jakevdp at cs.washington.edu> wrote:
> Hi,
> I've been working on using the new nbconvert framework from within the
> ipython root directory (after doing build_ext --inplace), and it's been
> really great!  But when I install IPython system-wide and try to use the
> Python nbconvert API from within another directory, I get a Jinja "template
> file not found" error.  It seems to go back to the default variables in
> IPython/nbconvert/exporters/exporter.py
>
> Any ideas how to best fix this?
>    Jake
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From benjaminrk at gmail.com  Wed Jul  3 23:42:14 2013
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 3 Jul 2013 20:42:14 -0700
Subject: [IPython-dev] nbconvert: template not found
In-Reply-To: <CAH4pYpSZYT+PgCoqjRmFkSC2FdOpP7jMvcSLHHBTa--RhjURVw@mail.gmail.com>
References: <CACpqBg2BH=cTGoYsaeaL7NZNXdoC7as5OHBjJ4i65WfcagRS8A@mail.gmail.com>
	<CAH4pYpSZYT+PgCoqjRmFkSC2FdOpP7jMvcSLHHBTa--RhjURVw@mail.gmail.com>
Message-ID: <CAHNn8BWYvoYEs5GO8d8=kgmmUHWg8vY7ebpWRft-Rp4N3GDLKQ@mail.gmail.com>

They weren't being properly included in the package data.

Fixed by #3532 <https://github.com/ipython/ipython/pull/3532>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130703/89db2baa/attachment.html>

From cyclebelfast at gmail.com  Thu Jul  4 02:43:03 2013
From: cyclebelfast at gmail.com (cyclebelfast at gmail.com)
Date: Thu, 04 Jul 2013 07:43:03 +0100
Subject: [IPython-dev] Merging nbconvert **now**
In-Reply-To: <CAH4pYpSH3EaPQ4t7ZHN5BdLO8MNuGqJjoyNc189eV73W+tL3bQ@mail.gmail.com>
References: <CAH4pYpSH3EaPQ4t7ZHN5BdLO8MNuGqJjoyNc189eV73W+tL3bQ@mail.gmail.com>
Message-ID: <51D51977.1010904@gmail.com>

On 30/06/13 20:28, Brian Granger wrote:
> Hi,
>
> As everyone knows, we are thinking about releasing IPython 1.0 in the
> next few weeks - with nbconvert in the main codebase.  If we are going
> to do this, I think we should put nbconvert into IPython proper now -
> halt all work on nbconvert and just put it into IPython/nbconvert.
> There are a few reasons I say this:
>
> * There are decisions being made in nbconvert (such as the whether or
> not to include reveal.js as a submodule) that we should really be
> making in the context of IPython.
> * There will be a lot of integration work to be done (setup.py,
> testing, etc) that also should be done in the context of IPython
> * I want the discussions about nbconvert to start to happen in the
> context of IPython.
> * To this day, nbconvert can't be setup.py installed, which is
> preventing other projects such as dexy from moving forward with
> integration.
[...]
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
>

Hi, just a data point. I hacked and munged nbconvert to a single module 
here:

https://github.com/averagehuman/ipynb2html

because

1) It didn't support the creation of HTML fragments for including in 
other output (eg. static blog engine/django templatetag)
2) It wasn't an installable package.

Here is the ReST directive I wrote for including an IPython notebook in 
a Pelican blog post:

https://gist.github.com/averagehuman/5925401

Thanks to whoever is working on nbconvert, I hope my use case can be 
supported.




From bussonniermatthias at gmail.com  Thu Jul  4 03:33:10 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 4 Jul 2013 09:33:10 +0200
Subject: [IPython-dev] Merging nbconvert **now**
In-Reply-To: <51D51977.1010904@gmail.com>
References: <CAH4pYpSH3EaPQ4t7ZHN5BdLO8MNuGqJjoyNc189eV73W+tL3bQ@mail.gmail.com>
	<51D51977.1010904@gmail.com>
Message-ID: <D54983A0-DD2C-4165-B197-ED08EDBA1471@gmail.com>

Hi, 

> Hi, just a data point. I hacked and munged nbconvert to a single module 
> here:
> 
> https://github.com/averagehuman/ipynb2html
> 
> because
> 
> 1) It didn't support the creation of HTML fragments for including in 
> other output (eg. static blog engine/django templatetag)
> 2) It wasn't an installable package.
> 
> Here is the ReST directive I wrote for including an IPython notebook in 
> a Pelican blog post:
> 
> https://gist.github.com/averagehuman/5925401
> 
> Thanks to whoever is working on nbconvert, I hope my use case can be 
> supported.


nbconvert should now be installed with IPython dev automatically, also as a Library.
Jakevdp is working on pelican extension : 

https://github.com/getpelican/pelican-plugins/pull/21

and you can see how to use it as a library here :

http://nbviewer.ipython.org/urls/raw.github.com/Carreau/posts/master/06-NBconvert-Doc-Draft.ipynb

nbconvert should be highly customizable, including only exporting part of a notebook, or including other stuff in between cells.

-- 
Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/af5a13b3/attachment.html>

From cyclebelfast at gmail.com  Thu Jul  4 04:30:54 2013
From: cyclebelfast at gmail.com (cyclebelfast at gmail.com)
Date: Thu, 04 Jul 2013 09:30:54 +0100
Subject: [IPython-dev] Merging nbconvert **now**
In-Reply-To: <D54983A0-DD2C-4165-B197-ED08EDBA1471@gmail.com>
References: <CAH4pYpSH3EaPQ4t7ZHN5BdLO8MNuGqJjoyNc189eV73W+tL3bQ@mail.gmail.com>
	<51D51977.1010904@gmail.com>
	<D54983A0-DD2C-4165-B197-ED08EDBA1471@gmail.com>
Message-ID: <51D532BE.308@gmail.com>

On 04/07/13 08:33, Matthias BUSSONNIER wrote:
>
> nbconvert should now be installed with IPython dev automatically, also
> as a Library.
> Jakevdp is working on pelican extension :
>
> https://github.com/getpelican/pelican-plugins/pull/21
>
> and you can see how to use it as a library here :
>
> http://nbviewer.ipython.org/urls/raw.github.com/Carreau/posts/master/06-NBconvert-Doc-Draft.ipynb
>
> nbconvert should be highly customizable, including only exporting part
> of a notebook, or including other stuff in between cells.
>
> --
> Matthias
>


Ah, sounds good. Thanks.



From bussonniermatthias at gmail.com  Thu Jul  4 05:13:51 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 4 Jul 2013 11:13:51 +0200
Subject: [IPython-dev] Some new cell types for describing data analyses
	in IPy. Notebook
In-Reply-To: <1372844910.16126.107.camel@peregrin>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<1372844910.16126.107.camel@peregrin>
Message-ID: <495C148C-AC63-48CD-9D2C-5784C35B8922@gmail.com>

Hi all, Many responses in one Mail


Le 3 juil. 2013 ? 11:48, Russell Neches a ?crit :

> In response to Matthias' points...
> 
>> Right now I thing storing the notebook as a directed graph is
>> problematic in a few way, the first being that it is incompatible with
>> the fact that people want to be able to run notebook in a headless
>> manner, which if you add explicit choice is not possible. 
> 
> If by "headless" execution you mean converting the notebook into a
> regular .py script, I think the directed graph model isn't a problem. In
> fact, it actually elegantly solves a number of thorny problems involved
> with transforming a notebook into a script.

Not exactly, I mean stripping cells from their output, and regenerating the output
without an open browser. But if you say the notebook keep references of choice
then this is not really a problem..

> ?

> Gabe is proposing to allow (but not force!) the user to build non-linear
> code paths into their notebooks. This way, it is possible to SPECIFY a
> PARTICULAR path through the code cells, and then output it in a linear
> form. 
> 
> The example he offered is a wonderful illustration of why this is
> awesome. There are eighteen possible .py files (or PDFs, or HTML files,
> etc etc) one could generate, and the user simply needs to choose one. If
> the user wants to proceed to make another script representing a
> different path, they can just activate one of the alternative paths and
> output another script. This is extremely powerful. The user could
> perhaps elect to generate all possible linear scripts for a given
> collection of alternatives, and then dispatch them to different nodes to
> compare and contrast the results. 
> 

I understand what you want to do, but first IMHO this is too complicated.
Users are already lost by the fact that codecell can be executed out of order. 
I also think this is close to a discussion that we had with the %with magic, 
in the sense that most of the use case could be solved by using feature of 
the underlying programming language. In this some case probably an if statement. 


> There is even a sensible default behavior; simply linearize according to
> the current selection state of the task cell. The ability to swap around
> chunks of a workflow and then output linearized scripts makes the
> concept of "headless" execution vastly more powerful and interesting. If
> you've ever had to work with things like Sun Grid Engine (a lot of us
> scientists are pretty much stuck with it), this would be a Life Changing
> Ability.

> There is also the matter of incompatible assumptions. I often create
> notebooks that begin with a bunch of code cells, each of which loads a
> different data set and sets parameters related to that particular data
> set. When I use these notebooks, I execute ONE of these, skip the rest,
> and then proceed to the actual analysis. At the moment there is NO WAY
> to correctly output this kind of notebook as a script without modifying
> it. 

I get that, and I myself would really like to execute notebook in a headless
manner that generate a report based on input data. It still have to be done, 
even if it is not that hard it have to be designed.

It does not take more than 50 line to do the basic[1], you just have to start the
kernel yourself before evaluating the notebook, and execute the data loading 
before evaluating each cell of the notebook, you can even inject dynamically
 the codecell you would have like to run, and save the final notebook.

> 
> The directed graph model makes this problem simply go away. I can stick
> these incompatible alternatives into a task, and just pick one of them. 
> 
> Again, nothing forces me to use these features. As Gabe pointed out, a
> linear document is a subset of a directed graph. It should be possible
> to load old notebooks as (rather dull) directed graphs without making a
> giant mess of the JavaScript.

From a theoretical point of view I agree, nonetheless, inserting, searching and other
common operation would rapidly become rather difficult, and even if the cost would
be low here, this would mean that any software that would like to work with ipynb
should support directed graph.

> 
>> This also contradict the fact that the notebook capture both the input
>> and the output of the computation. As you showed there is actually 18
>> different combinations of data analysis, and they are not all stored
>> in the notebook. 
> 
> I haven't dug into Gabe's code, but this doesn't seem to be a problem. A
> task cell has ONE input, ONE output, and at any given time, ONE selected
> execution pathway. From the outside, it works just like a regular code
> cell. It's just got some private state information about which execution
> pathway is currently active.
> 
>> To sum up, I don't think the current JS client is in it's current
>> state the place to implement such an idea. The Dag for cell order
>> might be an idea for future notebook format but need to be well
>> thought, and wait for cell IDs.
>> 
> You mean the JS client was PREVIOUSLY not in a sate to implement such an
> idea, and so Gabe fixed it. Hooray! ;-)
> 
> 
> Russell



Le 4 juil. 2013 ? 03:04, Gabriel Becker a ?crit :

> This added to the fact that each cell can support arbitrary metadata, you
> should be able to arrange preexisting in structure that work together. It might
> be a little difficult to do it right now as our javascript is not yet modular
> enough to be easily reused, but we are moving toward it.
> 
> Respectfully, rolling my own frontend for ipynb files given all the work the IPython team
> has done on the excellent notebook browser interface would be an enormous and extremely
> wasteful duplication of effort. I don't think its the right way to pursue these features.

With current architecture, I agree, but in the end you should be able to include only one javascript file
and the rest should be pulled with require.js so you would just need to overwrite what you need. 
In a perfect world the notebook would just be a jslib you can use, so you wouldn't have to patch what we do
but pass a list of (exrta) cell type you want to support, and maybe custom read methods for the cell the core
don't know about. Not sure how far we would support that, but it should be pretty easy to make custom format On Top
of ipynb

> Furthermore, if I were going to write an application offering the types of features I am talking about from scratch,
> there wouldn't be any good reason to base it on the unaltered ipynb format, as they don't easily support the structure
> required by those features without the additional cell types I implemented in my forked version. 
> 
> 
> Right now I thing storing the notebook as a directed graph is problematic in a
> few way,
> 
> I'm not talking about storing the notebook as an actual directed graph data structure. There would be benefits
> to that but its not necessary and it isn't want I did in my forked version.
> 
> The ability to have nested cells (cells which contain other cells) gets us everything we need structure wise,
> and is the basis of everything seen in both the video (other than interactive code cell stuff) and screenshots
> I posted. The ipynb file for the notebook pictured in the screenshot looks exactly like a normal ipynb file
> except that in the json there are cell declarations which have a cells field which contains the json descriptions
> of the cells contained in that cell.

this is more or less what I called storing as a DAG (or more like a tree I guess here), this look a lot like what 
we had with worksheet, and we are moving away from this data structure because of it's complexity to handle some cases

> ?
> 
> I apologize for not being clear. As I said in a response above, the directed graph idea was intended to be conceptual
> for thinking about the documents, not structural for actually storing them.

I don't think the 2 are unrelated. thinking and storing document as graph could make sens.

> What I actually did was simply allow cell nesting and change indexing so that it is with respect to the parent/container
> (cell or notebook) instead of always with respect to the notebook. This required some machinery changes but not too
> many and it is an extension in the mathematical sense in that indexing will behave identically to the old system for
> notebooks without any nesting while now meaningfully functioning for notebooks with nesting.

I'm still curious of that, and would be a little afraid of how you handle things in UI.


Le 4 juil. 2013 ? 03:59, Brian Granger a ?crit :

> Gabriel,
> ...
> 
> Second, while it is tempting to generalize the notion of input to
> include widgety things, it is more appropriate to put these things in
> the output:


> * Putting widgets in the input area forces you to do regular
> expression matching to replace those variables in the code.  This
> limits you to an extremely simple event model where the only possible
> event you can know about is substitute the regular expression and run
> all the code.  What if you want different UI controls in the browser
> to trigger different bits of code in the kernels when different fine
> grained events happen.  Making the UI controls live on the Python and
> JS side allows us to build this in a natural way.

This is one place where I sometime disagree with Brian, where I think
input widget for codemirror would be great. To compare with Gabriel 'interactive'
code cell, I would be more inclined to provide the ability to bind with get to Codemirror
like in http://livecoding.io/3419309
through reg-exp it bind to any variable in the codecell and pop a widget to change the value.
you don't have to explicitly state which variable should be bound. 
Implementation detail CM provide a method to get token at cursor, which helps a lot?
It has also the advantage of working without changing the cell type.


> Th alt-cells you show bring up the issue of providence.  We have some
> very initial thoughts about that, but it is way out of scope for the
> project right now - we have a plates 10x overfull already.  We will
> get there though eventually.

Personally I'm torn with alt cell. I feel they should be used using function and cases with the underlying language, 
but they have a definitive advantage in teaching and exploring.

> Thanks for sharing your ideas.


Final thought, 

	Seeing interactive widget is definitively awesome, and look fantastic.
I think we can avoid having a specific cell type for that altogether,
and IMHO input methods (like livecoding.io) and interactive widget are 
2 complementary approaches. I think Gabriel way of adding specific widget
that are bound to specific line could also be done without specific cell type 
using metadata. I can't wait to be able to select my matplotlib color in a color picker
directly in CM for example.

	The task cell are nice, but think should be covered using a different mechanism.
as brian pointed out we think of using implicit grouping using headers.

	As I already said, I'm torn about alt-cell. The UI is nice for teaching, but I think it covers a
too small use case. in particular to change the path you use, you need a user interaction, 
or to use a specific tool to run the notebook by selecting a path. 
This IMHO go agains the modularity. I see that those weeks because I'm polishing a notebook
for a publication, alt cell would have definitively been usefull for test, but now I'm curing myself for not having
written function I could have reused and add if statement to select the case.
	So I would be inclined to have a semi-linear approaches she you write the functions you need, 
and the path is selected using cases in python, why not with a radio-selector ui that set the value of a variable
which set the future path, but not actually selecting a cell at ipynb level. This has also the advantage of being 
pure python compatible, without having to generate multiple script.


	Still really impressed by the work, and I really thing this is a good start for more discussion, and a nice starting point
to design stuff we will add later.

Sorry for the length and if I missed stuff.
-- 
Matthias



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/60af6d2d/attachment.html>

From jason-sage at creativetrax.com  Thu Jul  4 05:50:44 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 04 Jul 2013 02:50:44 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <495C148C-AC63-48CD-9D2C-5784C35B8922@gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<1372844910.16126.107.camel@peregrin>
	<495C148C-AC63-48CD-9D2C-5784C35B8922@gmail.com>
Message-ID: <51D54574.5070901@creativetrax.com>

On 7/4/13 2:13 AM, Matthias BUSSONNIER wrote:
>
> This is one place where I sometime disagree with Brian, where I think
> input widget for codemirror would be great. To compare with Gabriel
> 'interactive'
> code cell, I would be more inclined to provide the ability to bind with
> get to Codemirror
> like in http://livecoding.io/3419309
> through reg-exp it bind to any variable in the codecell and pop a widget
> to change the value.
> you don't have to explicitly state which variable should be bound.
> Implementation detail CM provide a method to get token at cursor, which
> helps a lot?
> It has also the advantage of working without changing the cell type.

I think input widgets like the livecoding.io examples are a powerful way 
to input/change code.  In the end, you are just having the computer help 
you write code, instead of hand-typing each letter.  Rather than looking 
up a separate website that lets me pick an html color and gives me a hex 
value, then pasting that hex value into the IPython cell, the pop-up 
widget in the cell pastes or changes the hex value right there.

For another take on input widgets, see my proof-of-concept example here: 
http://bl.ocks.org/jasongrout/5378313  It would be easy to imagine a 
slider widget replacing a number, like:

n = <slider>

and a quick key to toggle between the slider and the number inside the 
codemirror widget.

FYI, we've had a short discussion on widgets already [1]

Thanks,

Jason

[1] 
https://github.com/ipython/ipython/issues/2802#issuecomment-16711626, 
and ensuing comments



From jakevdp at cs.washington.edu  Thu Jul  4 12:10:50 2013
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Thu, 4 Jul 2013 09:10:50 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
Message-ID: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>

Hi,
I've been working on adapting the recently-merged nbconvert refactor to
work with my Pelican blogging plugin, and am having a really difficult
time.  In particular, the header content produced by the new nbconvert (via
CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to the old
version.  This is conflicting with the default blog theme, and leading to
some results which are not very pretty.

In the old iteration of the notebook plugin, I used about half a dozen
regular expression replace statements to modify the stylesheet & content
and make it play well with the blog style.  My hope was that the new
nbconvert would be flexible enough to obviate the need for this sort of
text-mangling; in reality the required text-mangling in the new version is
much more extensive.  I've been working at it for several hours, and still
don't have a suitable solution that leads to nicely-formatted notebooks
within blog posts.

For those of you familiar with the new nbconvert: what is the reason for
the changes in the default CSS styles between nbconvert 1 and 2?  Is there
a good way to recover the old style sheet within the new codebase?

Thanks,
   Jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/315f83b2/attachment.html>

From damianavila at gmail.com  Thu Jul  4 12:26:37 2013
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Thu, 04 Jul 2013 13:26:37 -0300
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
Message-ID: <51D5A23D.3040401@gmail.com>

El 04/07/13 13:10, Jacob Vanderplas escribi?:
> Hi,
> I've been working on adapting the recently-merged nbconvert refactor 
> to work with my Pelican blogging plugin, and am having a really 
> difficult time.  In particular, the header content produced by the new 
> nbconvert (via CSSHTMLHeaderTransformer) contains a lot of extra stuff 
> compared to the old version.  This is conflicting with the default 
> blog theme, and leading to some results which are not very pretty.
>
> In the old iteration of the notebook plugin, I used about half a dozen 
> regular expression replace statements to modify the stylesheet & 
> content and make it play well with the blog style.  My hope was that 
> the new nbconvert would be flexible enough to obviate the need for 
> this sort of text-mangling; in reality the required text-mangling in 
> the new version is much more extensive.  I've been working at it for 
> several hours, and still don't have a suitable solution that leads to 
> nicely-formatted notebooks within blog posts.
>
> For those of you familiar with the new nbconvert: what is the reason 
> for the changes in the default CSS styles between nbconvert 1 and 2? 
>  Is there a good way to recover the old style sheet within the new 
> codebase?
>
> Thanks,
>    Jake
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Hi Jake,

As far as I know, the changes in css are derived by the incorporation of 
the less machinery to get the final css.
The changes were done in the general IPython css styling, and nbconvert 
is just reading the new style contained in style.min.css.
Probably Mathias can go deeper in the explanation because he PRed the 
changes involving less.

You can always get the old style sheet from previous versions of 
IPython, make an old_ipython.css file and include it in your head 
instead of the new styling.
This is the way I used to do it in my plugin for Nikola, although I am 
migrating to the new styling very soon.

Hope it helps.

Dami?n.







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/912389aa/attachment.html>

From damianavila at gmail.com  Thu Jul  4 13:05:39 2013
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Thu, 04 Jul 2013 14:05:39 -0300
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
Message-ID: <51D5AB63.1090702@gmail.com>

I forgot to add the link to the ipython.css file I am using with the plugin:

https://github.com/damianavila/site-ipython-theme-for-Nikola/blob/master/site-ipython/assets/css/ipython.css

Cheers.

Dami?n.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/54e0d6c6/attachment.html>

From ellisonbg at gmail.com  Thu Jul  4 13:35:11 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 4 Jul 2013 10:35:11 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
Message-ID: <CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>

There are a couple of factors going on:

* We are now minimizing the stylesheet which obscures everything.
* We are transitioning to using bootstrap.  You may have direct
conflicts with bootstrap classes as well as ours
* Our css classes are horribly named - generic names like "cell" or
"selected".  To address this we are planning on renaming our css
classes using the following convention:

ipy-cell-selected
ipy-notebook-foo

We probably won't have time to do all of our classes before 1.0, but
we can prioritize the ones you are having problems with.  Can you open
an issue for this and provide us with a list of the ones you are
running into?

Cheers,

Brian


On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
<jakevdp at cs.washington.edu> wrote:
> Hi,
> I've been working on adapting the recently-merged nbconvert refactor to work
> with my Pelican blogging plugin, and am having a really difficult time.  In
> particular, the header content produced by the new nbconvert (via
> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to the old
> version.  This is conflicting with the default blog theme, and leading to
> some results which are not very pretty.
>
> In the old iteration of the notebook plugin, I used about half a dozen
> regular expression replace statements to modify the stylesheet & content and
> make it play well with the blog style.  My hope was that the new nbconvert
> would be flexible enough to obviate the need for this sort of text-mangling;
> in reality the required text-mangling in the new version is much more
> extensive.  I've been working at it for several hours, and still don't have
> a suitable solution that leads to nicely-formatted notebooks within blog
> posts.
>
> For those of you familiar with the new nbconvert: what is the reason for the
> changes in the default CSS styles between nbconvert 1 and 2?  Is there a
> good way to recover the old style sheet within the new codebase?
>
> Thanks,
>    Jake
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Thu Jul  4 15:04:52 2013
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Thu, 4 Jul 2013 21:04:52 +0200
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
Message-ID: <CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>

Short from my phone:

Could we add an .ipython class to body,
And wrap the all less import in a .ipython{}

CSS will then only apply to elements that are included in a div.ipython ?

Le jeudi 4 juillet 2013, Brian Granger a ?crit :

> There are a couple of factors going on:
>
> * We are now minimizing the stylesheet which obscures everything.
> * We are transitioning to using bootstrap.  You may have direct
> conflicts with bootstrap classes as well as ours
> * Our css classes are horribly named - generic names like "cell" or
> "selected".  To address this we are planning on renaming our css
> classes using the following convention:
>
> ipy-cell-selected
> ipy-notebook-foo
>
> We probably won't have time to do all of our classes before 1.0, but
> we can prioritize the ones you are having problems with.  Can you open
> an issue for this and provide us with a list of the ones you are
> running into?
>
> Cheers,
>
> Brian
>
>
> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
> <jakevdp at cs.washington.edu <javascript:;>> wrote:
> > Hi,
> > I've been working on adapting the recently-merged nbconvert refactor to
> work
> > with my Pelican blogging plugin, and am having a really difficult time.
>  In
> > particular, the header content produced by the new nbconvert (via
> > CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to the
> old
> > version.  This is conflicting with the default blog theme, and leading to
> > some results which are not very pretty.
> >
> > In the old iteration of the notebook plugin, I used about half a dozen
> > regular expression replace statements to modify the stylesheet & content
> and
> > make it play well with the blog style.  My hope was that the new
> nbconvert
> > would be flexible enough to obviate the need for this sort of
> text-mangling;
> > in reality the required text-mangling in the new version is much more
> > extensive.  I've been working at it for several hours, and still don't
> have
> > a suitable solution that leads to nicely-formatted notebooks within blog
> > posts.
> >
> > For those of you familiar with the new nbconvert: what is the reason for
> the
> > changes in the default CSS styles between nbconvert 1 and 2?  Is there a
> > good way to recover the old style sheet within the new codebase?
> >
> > Thanks,
> >    Jake
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org <javascript:;>
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu <javascript:;> and ellisonbg at gmail.com <javascript:;>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org <javascript:;>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/dd9621c8/attachment.html>

From benjaminrk at gmail.com  Thu Jul  4 15:11:28 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 4 Jul 2013 12:11:28 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
Message-ID: <CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>

Matthias is right - we don't actually have to prefix every name, we can
just scope our CSS based on the outer ipython container.


On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> Short from my phone:
>
> Could we add an .ipython class to body,
> And wrap the all less import in a .ipython{}
>
> CSS will then only apply to elements that are included in a div.ipython ?
>
> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>
> There are a couple of factors going on:
>>
>> * We are now minimizing the stylesheet which obscures everything.
>> * We are transitioning to using bootstrap.  You may have direct
>> conflicts with bootstrap classes as well as ours
>> * Our css classes are horribly named - generic names like "cell" or
>> "selected".  To address this we are planning on renaming our css
>> classes using the following convention:
>>
>> ipy-cell-selected
>> ipy-notebook-foo
>>
>> We probably won't have time to do all of our classes before 1.0, but
>> we can prioritize the ones you are having problems with.  Can you open
>> an issue for this and provide us with a list of the ones you are
>> running into?
>>
>> Cheers,
>>
>> Brian
>>
>>
>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>> <jakevdp at cs.washington.edu> wrote:
>> > Hi,
>> > I've been working on adapting the recently-merged nbconvert refactor to
>> work
>> > with my Pelican blogging plugin, and am having a really difficult time.
>>  In
>> > particular, the header content produced by the new nbconvert (via
>> > CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to the
>> old
>> > version.  This is conflicting with the default blog theme, and leading
>> to
>> > some results which are not very pretty.
>> >
>> > In the old iteration of the notebook plugin, I used about half a dozen
>> > regular expression replace statements to modify the stylesheet &
>> content and
>> > make it play well with the blog style.  My hope was that the new
>> nbconvert
>> > would be flexible enough to obviate the need for this sort of
>> text-mangling;
>> > in reality the required text-mangling in the new version is much more
>> > extensive.  I've been working at it for several hours, and still don't
>> have
>> > a suitable solution that leads to nicely-formatted notebooks within blog
>> > posts.
>> >
>> > For those of you familiar with the new nbconvert: what is the reason
>> for the
>> > changes in the default CSS styles between nbconvert 1 and 2?  Is there a
>> > good way to recover the old style sheet within the new codebase?
>> >
>> > Thanks,
>> >    Jake
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/30026113/attachment.html>

From ellisonbg at gmail.com  Thu Jul  4 15:25:35 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 4 Jul 2013 12:25:35 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
Message-ID: <CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>

Does the less syntax for nesting:

.ipython {

  .cell {}

}

map to only the immediate children or all descendents?  If it only
works for immediate children, our css stylesheets would have to become
horribly nested.

Cheers,

Brian

On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
> Matthias is right - we don't actually have to prefix every name, we can just
> scope our CSS based on the outer ipython container.
>
>
> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
> <bussonniermatthias at gmail.com> wrote:
>>
>> Short from my phone:
>>
>> Could we add an .ipython class to body,
>> And wrap the all less import in a .ipython{}
>>
>> CSS will then only apply to elements that are included in a div.ipython ?
>>
>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>>
>>> There are a couple of factors going on:
>>>
>>> * We are now minimizing the stylesheet which obscures everything.
>>> * We are transitioning to using bootstrap.  You may have direct
>>> conflicts with bootstrap classes as well as ours
>>> * Our css classes are horribly named - generic names like "cell" or
>>> "selected".  To address this we are planning on renaming our css
>>> classes using the following convention:
>>>
>>> ipy-cell-selected
>>> ipy-notebook-foo
>>>
>>> We probably won't have time to do all of our classes before 1.0, but
>>> we can prioritize the ones you are having problems with.  Can you open
>>> an issue for this and provide us with a list of the ones you are
>>> running into?
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>>
>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>>> <jakevdp at cs.washington.edu> wrote:
>>> > Hi,
>>> > I've been working on adapting the recently-merged nbconvert refactor to
>>> > work
>>> > with my Pelican blogging plugin, and am having a really difficult time.
>>> > In
>>> > particular, the header content produced by the new nbconvert (via
>>> > CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to the
>>> > old
>>> > version.  This is conflicting with the default blog theme, and leading
>>> > to
>>> > some results which are not very pretty.
>>> >
>>> > In the old iteration of the notebook plugin, I used about half a dozen
>>> > regular expression replace statements to modify the stylesheet &
>>> > content and
>>> > make it play well with the blog style.  My hope was that the new
>>> > nbconvert
>>> > would be flexible enough to obviate the need for this sort of
>>> > text-mangling;
>>> > in reality the required text-mangling in the new version is much more
>>> > extensive.  I've been working at it for several hours, and still don't
>>> > have
>>> > a suitable solution that leads to nicely-formatted notebooks within
>>> > blog
>>> > posts.
>>> >
>>> > For those of you familiar with the new nbconvert: what is the reason
>>> > for the
>>> > changes in the default CSS styles between nbconvert 1 and 2?  Is there
>>> > a
>>> > good way to recover the old style sheet within the new codebase?
>>> >
>>> > Thanks,
>>> >    Jake
>>> >
>>> > _______________________________________________
>>> > IPython-dev mailing list
>>> > IPython-dev at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >
>>>
>>>
>>>
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From benjaminrk at gmail.com  Thu Jul  4 15:38:59 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 4 Jul 2013 12:38:59 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
Message-ID: <CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>

There are two ways of applying style to descendants - immediate (with `>`)
and any level (with space, or as you posted).

So if we just put all of our style in a `.ipython {...}` block, then our
classes in a single `.ipython` div, our style would not apply outside that
context.

Illustration: http://nbviewer.ipython.org/5929801




On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Does the less syntax for nesting:
>
> .ipython {
>
>   .cell {}
>
> }
>
> map to only the immediate children or all descendents?  If it only
> works for immediate children, our css stylesheets would have to become
> horribly nested.
>
> Cheers,
>
> Brian
>
> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
> > Matthias is right - we don't actually have to prefix every name, we can
> just
> > scope our CSS based on the outer ipython container.
> >
> >
> > On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
> > <bussonniermatthias at gmail.com> wrote:
> >>
> >> Short from my phone:
> >>
> >> Could we add an .ipython class to body,
> >> And wrap the all less import in a .ipython{}
> >>
> >> CSS will then only apply to elements that are included in a div.ipython
> ?
> >>
> >> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
> >>
> >>> There are a couple of factors going on:
> >>>
> >>> * We are now minimizing the stylesheet which obscures everything.
> >>> * We are transitioning to using bootstrap.  You may have direct
> >>> conflicts with bootstrap classes as well as ours
> >>> * Our css classes are horribly named - generic names like "cell" or
> >>> "selected".  To address this we are planning on renaming our css
> >>> classes using the following convention:
> >>>
> >>> ipy-cell-selected
> >>> ipy-notebook-foo
> >>>
> >>> We probably won't have time to do all of our classes before 1.0, but
> >>> we can prioritize the ones you are having problems with.  Can you open
> >>> an issue for this and provide us with a list of the ones you are
> >>> running into?
> >>>
> >>> Cheers,
> >>>
> >>> Brian
> >>>
> >>>
> >>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
> >>> <jakevdp at cs.washington.edu> wrote:
> >>> > Hi,
> >>> > I've been working on adapting the recently-merged nbconvert refactor
> to
> >>> > work
> >>> > with my Pelican blogging plugin, and am having a really difficult
> time.
> >>> > In
> >>> > particular, the header content produced by the new nbconvert (via
> >>> > CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to
> the
> >>> > old
> >>> > version.  This is conflicting with the default blog theme, and
> leading
> >>> > to
> >>> > some results which are not very pretty.
> >>> >
> >>> > In the old iteration of the notebook plugin, I used about half a
> dozen
> >>> > regular expression replace statements to modify the stylesheet &
> >>> > content and
> >>> > make it play well with the blog style.  My hope was that the new
> >>> > nbconvert
> >>> > would be flexible enough to obviate the need for this sort of
> >>> > text-mangling;
> >>> > in reality the required text-mangling in the new version is much more
> >>> > extensive.  I've been working at it for several hours, and still
> don't
> >>> > have
> >>> > a suitable solution that leads to nicely-formatted notebooks within
> >>> > blog
> >>> > posts.
> >>> >
> >>> > For those of you familiar with the new nbconvert: what is the reason
> >>> > for the
> >>> > changes in the default CSS styles between nbconvert 1 and 2?  Is
> there
> >>> > a
> >>> > good way to recover the old style sheet within the new codebase?
> >>> >
> >>> > Thanks,
> >>> >    Jake
> >>> >
> >>> > _______________________________________________
> >>> > IPython-dev mailing list
> >>> > IPython-dev at scipy.org
> >>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Brian E. Granger
> >>> Cal Poly State University, San Luis Obispo
> >>> bgranger at calpoly.edu and ellisonbg at gmail.com
> >>> _______________________________________________
> >>> IPython-dev mailing list
> >>> IPython-dev at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/a8b931ac/attachment.html>

From ellisonbg at gmail.com  Thu Jul  4 16:37:35 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 4 Jul 2013 13:37:35 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
Message-ID: <CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>

One other usage case - if someone wants to integrate ipython assets
into their web page at a finer grained scale - imagine someone wanting
to put a single cell at various locations on their non-ipython using
page.  In that case, each IPython containing div would have to have
the .ipython class?  I don't think that is a problem, but I just want
to make sure we are covering the different usage cases...

Cheers,

Brian

On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
> There are two ways of applying style to descendants - immediate (with `>`)
> and any level (with space, or as you posted).
>
> So if we just put all of our style in a `.ipython {...}` block, then our
> classes in a single `.ipython` div, our style would not apply outside that
> context.
>
> Illustration: http://nbviewer.ipython.org/5929801
>
>
>
>
> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> Does the less syntax for nesting:
>>
>> .ipython {
>>
>>   .cell {}
>>
>> }
>>
>> map to only the immediate children or all descendents?  If it only
>> works for immediate children, our css stylesheets would have to become
>> horribly nested.
>>
>> Cheers,
>>
>> Brian
>>
>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>> > Matthias is right - we don't actually have to prefix every name, we can
>> > just
>> > scope our CSS based on the outer ipython container.
>> >
>> >
>> > On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>> > <bussonniermatthias at gmail.com> wrote:
>> >>
>> >> Short from my phone:
>> >>
>> >> Could we add an .ipython class to body,
>> >> And wrap the all less import in a .ipython{}
>> >>
>> >> CSS will then only apply to elements that are included in a div.ipython
>> >> ?
>> >>
>> >> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>> >>
>> >>> There are a couple of factors going on:
>> >>>
>> >>> * We are now minimizing the stylesheet which obscures everything.
>> >>> * We are transitioning to using bootstrap.  You may have direct
>> >>> conflicts with bootstrap classes as well as ours
>> >>> * Our css classes are horribly named - generic names like "cell" or
>> >>> "selected".  To address this we are planning on renaming our css
>> >>> classes using the following convention:
>> >>>
>> >>> ipy-cell-selected
>> >>> ipy-notebook-foo
>> >>>
>> >>> We probably won't have time to do all of our classes before 1.0, but
>> >>> we can prioritize the ones you are having problems with.  Can you open
>> >>> an issue for this and provide us with a list of the ones you are
>> >>> running into?
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Brian
>> >>>
>> >>>
>> >>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>> >>> <jakevdp at cs.washington.edu> wrote:
>> >>> > Hi,
>> >>> > I've been working on adapting the recently-merged nbconvert refactor
>> >>> > to
>> >>> > work
>> >>> > with my Pelican blogging plugin, and am having a really difficult
>> >>> > time.
>> >>> > In
>> >>> > particular, the header content produced by the new nbconvert (via
>> >>> > CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to
>> >>> > the
>> >>> > old
>> >>> > version.  This is conflicting with the default blog theme, and
>> >>> > leading
>> >>> > to
>> >>> > some results which are not very pretty.
>> >>> >
>> >>> > In the old iteration of the notebook plugin, I used about half a
>> >>> > dozen
>> >>> > regular expression replace statements to modify the stylesheet &
>> >>> > content and
>> >>> > make it play well with the blog style.  My hope was that the new
>> >>> > nbconvert
>> >>> > would be flexible enough to obviate the need for this sort of
>> >>> > text-mangling;
>> >>> > in reality the required text-mangling in the new version is much
>> >>> > more
>> >>> > extensive.  I've been working at it for several hours, and still
>> >>> > don't
>> >>> > have
>> >>> > a suitable solution that leads to nicely-formatted notebooks within
>> >>> > blog
>> >>> > posts.
>> >>> >
>> >>> > For those of you familiar with the new nbconvert: what is the reason
>> >>> > for the
>> >>> > changes in the default CSS styles between nbconvert 1 and 2?  Is
>> >>> > there
>> >>> > a
>> >>> > good way to recover the old style sheet within the new codebase?
>> >>> >
>> >>> > Thanks,
>> >>> >    Jake
>> >>> >
>> >>> > _______________________________________________
>> >>> > IPython-dev mailing list
>> >>> > IPython-dev at scipy.org
>> >>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Brian E. Granger
>> >>> Cal Poly State University, San Luis Obispo
>> >>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >>> _______________________________________________
>> >>> IPython-dev mailing list
>> >>> IPython-dev at scipy.org
>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From benjaminrk at gmail.com  Thu Jul  4 16:42:01 2013
From: benjaminrk at gmail.com (Min RK)
Date: Thu, 4 Jul 2013 13:42:01 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
Message-ID: <A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>

If we did it that way, yes.

We are going to have to explore a few options, it's not obvious what's the best choice for various contexts.  It will help to have the various sticky points affecting Jake as data points.

-MinRK

On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:

> One other usage case - if someone wants to integrate ipython assets
> into their web page at a finer grained scale - imagine someone wanting
> to put a single cell at various locations on their non-ipython using
> page.  In that case, each IPython containing div would have to have
> the .ipython class?  I don't think that is a problem, but I just want
> to make sure we are covering the different usage cases...
> 
> Cheers,
> 
> Brian
> 
> On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
>> There are two ways of applying style to descendants - immediate (with `>`)
>> and any level (with space, or as you posted).
>> 
>> So if we just put all of our style in a `.ipython {...}` block, then our
>> classes in a single `.ipython` div, our style would not apply outside that
>> context.
>> 
>> Illustration: http://nbviewer.ipython.org/5929801
>> 
>> 
>> 
>> 
>> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>> 
>>> Does the less syntax for nesting:
>>> 
>>> .ipython {
>>> 
>>>  .cell {}
>>> 
>>> }
>>> 
>>> map to only the immediate children or all descendents?  If it only
>>> works for immediate children, our css stylesheets would have to become
>>> horribly nested.
>>> 
>>> Cheers,
>>> 
>>> Brian
>>> 
>>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>> Matthias is right - we don't actually have to prefix every name, we can
>>>> just
>>>> scope our CSS based on the outer ipython container.
>>>> 
>>>> 
>>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>>>> <bussonniermatthias at gmail.com> wrote:
>>>>> 
>>>>> Short from my phone:
>>>>> 
>>>>> Could we add an .ipython class to body,
>>>>> And wrap the all less import in a .ipython{}
>>>>> 
>>>>> CSS will then only apply to elements that are included in a div.ipython
>>>>> ?
>>>>> 
>>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>>>>> 
>>>>>> There are a couple of factors going on:
>>>>>> 
>>>>>> * We are now minimizing the stylesheet which obscures everything.
>>>>>> * We are transitioning to using bootstrap.  You may have direct
>>>>>> conflicts with bootstrap classes as well as ours
>>>>>> * Our css classes are horribly named - generic names like "cell" or
>>>>>> "selected".  To address this we are planning on renaming our css
>>>>>> classes using the following convention:
>>>>>> 
>>>>>> ipy-cell-selected
>>>>>> ipy-notebook-foo
>>>>>> 
>>>>>> We probably won't have time to do all of our classes before 1.0, but
>>>>>> we can prioritize the ones you are having problems with.  Can you open
>>>>>> an issue for this and provide us with a list of the ones you are
>>>>>> running into?
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> Brian
>>>>>> 
>>>>>> 
>>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>>>>>> <jakevdp at cs.washington.edu> wrote:
>>>>>>> Hi,
>>>>>>> I've been working on adapting the recently-merged nbconvert refactor
>>>>>>> to
>>>>>>> work
>>>>>>> with my Pelican blogging plugin, and am having a really difficult
>>>>>>> time.
>>>>>>> In
>>>>>>> particular, the header content produced by the new nbconvert (via
>>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to
>>>>>>> the
>>>>>>> old
>>>>>>> version.  This is conflicting with the default blog theme, and
>>>>>>> leading
>>>>>>> to
>>>>>>> some results which are not very pretty.
>>>>>>> 
>>>>>>> In the old iteration of the notebook plugin, I used about half a
>>>>>>> dozen
>>>>>>> regular expression replace statements to modify the stylesheet &
>>>>>>> content and
>>>>>>> make it play well with the blog style.  My hope was that the new
>>>>>>> nbconvert
>>>>>>> would be flexible enough to obviate the need for this sort of
>>>>>>> text-mangling;
>>>>>>> in reality the required text-mangling in the new version is much
>>>>>>> more
>>>>>>> extensive.  I've been working at it for several hours, and still
>>>>>>> don't
>>>>>>> have
>>>>>>> a suitable solution that leads to nicely-formatted notebooks within
>>>>>>> blog
>>>>>>> posts.
>>>>>>> 
>>>>>>> For those of you familiar with the new nbconvert: what is the reason
>>>>>>> for the
>>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
>>>>>>> there
>>>>>>> a
>>>>>>> good way to recover the old style sheet within the new codebase?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>>   Jake
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> IPython-dev mailing list
>>>>>>> IPython-dev at scipy.org
>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Brian E. Granger
>>>>>> Cal Poly State University, San Luis Obispo
>>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> 
>>> 
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> 
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From jakevdp at cs.washington.edu  Thu Jul  4 17:23:54 2013
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Thu, 4 Jul 2013 14:23:54 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
Message-ID: <CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>

On Thu, Jul 4, 2013 at 1:42 PM, Min RK <benjaminrk at gmail.com> wrote:

> We are going to have to explore a few options, it's not obvious what's the
> best choice for various contexts.  It will help to have the various sticky
> points affecting Jake as data points.
>

I like the idea of wrapping everything so it's self-contained. My "sticky
points" are basically all due to this: either places where the blog theme
overrides the notebook and makes it look ugly, or places where the notebook
css overrides the blog theme and makes it look ugly.  Most of my hacks to
the CSS have involved getting the template to the point of being
essentially what you're describing here.  Because it's not designed that
way from the ground up, however, some of the styles aren't carrying down to
multiply nested elements.

I think the best solution would have the following:

- use non-generic CSS class names (e.g. "ipynb-cell" rather than "cell",
and "ipynb-highlight" rather than "highlight"),
or nest any such generic names within an "ipynb" namespace.
- make no global CSS modifications (e.g. global-level header formatting,
page formatting, body formatting)
- design the CSS so that notebook sections in a div, such as <div
class="ipynb"></div>, so that multiple notebook snippets can easily be
included in a page which otherwise has arbitrary html/css styling.

I don't have a great deal of experience with CSS, but from my
understanding, this sort of approach should be possible.
   Jake


>
> -MinRK
>
> On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:
>
> > One other usage case - if someone wants to integrate ipython assets
> > into their web page at a finer grained scale - imagine someone wanting
> > to put a single cell at various locations on their non-ipython using
> > page.  In that case, each IPython containing div would have to have
> > the .ipython class?  I don't think that is a problem, but I just want
> > to make sure we are covering the different usage cases...
> >
> > Cheers,
> >
> > Brian
> >
> > On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
> >> There are two ways of applying style to descendants - immediate (with
> `>`)
> >> and any level (with space, or as you posted).
> >>
> >> So if we just put all of our style in a `.ipython {...}` block, then our
> >> classes in a single `.ipython` div, our style would not apply outside
> that
> >> context.
> >>
> >> Illustration: http://nbviewer.ipython.org/5929801
> >>
> >>
> >>
> >>
> >> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
> >>>
> >>> Does the less syntax for nesting:
> >>>
> >>> .ipython {
> >>>
> >>>  .cell {}
> >>>
> >>> }
> >>>
> >>> map to only the immediate children or all descendents?  If it only
> >>> works for immediate children, our css stylesheets would have to become
> >>> horribly nested.
> >>>
> >>> Cheers,
> >>>
> >>> Brian
> >>>
> >>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
> >>>> Matthias is right - we don't actually have to prefix every name, we
> can
> >>>> just
> >>>> scope our CSS based on the outer ipython container.
> >>>>
> >>>>
> >>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
> >>>> <bussonniermatthias at gmail.com> wrote:
> >>>>>
> >>>>> Short from my phone:
> >>>>>
> >>>>> Could we add an .ipython class to body,
> >>>>> And wrap the all less import in a .ipython{}
> >>>>>
> >>>>> CSS will then only apply to elements that are included in a
> div.ipython
> >>>>> ?
> >>>>>
> >>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
> >>>>>
> >>>>>> There are a couple of factors going on:
> >>>>>>
> >>>>>> * We are now minimizing the stylesheet which obscures everything.
> >>>>>> * We are transitioning to using bootstrap.  You may have direct
> >>>>>> conflicts with bootstrap classes as well as ours
> >>>>>> * Our css classes are horribly named - generic names like "cell" or
> >>>>>> "selected".  To address this we are planning on renaming our css
> >>>>>> classes using the following convention:
> >>>>>>
> >>>>>> ipy-cell-selected
> >>>>>> ipy-notebook-foo
> >>>>>>
> >>>>>> We probably won't have time to do all of our classes before 1.0, but
> >>>>>> we can prioritize the ones you are having problems with.  Can you
> open
> >>>>>> an issue for this and provide us with a list of the ones you are
> >>>>>> running into?
> >>>>>>
> >>>>>> Cheers,
> >>>>>>
> >>>>>> Brian
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
> >>>>>> <jakevdp at cs.washington.edu> wrote:
> >>>>>>> Hi,
> >>>>>>> I've been working on adapting the recently-merged nbconvert
> refactor
> >>>>>>> to
> >>>>>>> work
> >>>>>>> with my Pelican blogging plugin, and am having a really difficult
> >>>>>>> time.
> >>>>>>> In
> >>>>>>> particular, the header content produced by the new nbconvert (via
> >>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared to
> >>>>>>> the
> >>>>>>> old
> >>>>>>> version.  This is conflicting with the default blog theme, and
> >>>>>>> leading
> >>>>>>> to
> >>>>>>> some results which are not very pretty.
> >>>>>>>
> >>>>>>> In the old iteration of the notebook plugin, I used about half a
> >>>>>>> dozen
> >>>>>>> regular expression replace statements to modify the stylesheet &
> >>>>>>> content and
> >>>>>>> make it play well with the blog style.  My hope was that the new
> >>>>>>> nbconvert
> >>>>>>> would be flexible enough to obviate the need for this sort of
> >>>>>>> text-mangling;
> >>>>>>> in reality the required text-mangling in the new version is much
> >>>>>>> more
> >>>>>>> extensive.  I've been working at it for several hours, and still
> >>>>>>> don't
> >>>>>>> have
> >>>>>>> a suitable solution that leads to nicely-formatted notebooks within
> >>>>>>> blog
> >>>>>>> posts.
> >>>>>>>
> >>>>>>> For those of you familiar with the new nbconvert: what is the
> reason
> >>>>>>> for the
> >>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
> >>>>>>> there
> >>>>>>> a
> >>>>>>> good way to recover the old style sheet within the new codebase?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>   Jake
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> IPython-dev mailing list
> >>>>>>> IPython-dev at scipy.org
> >>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Brian E. Granger
> >>>>>> Cal Poly State University, San Luis Obispo
> >>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
> >>>>>> _______________________________________________
> >>>>>> IPython-dev mailing list
> >>>>>> IPython-dev at scipy.org
> >>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> IPython-dev mailing list
> >>>>> IPython-dev at scipy.org
> >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> IPython-dev mailing list
> >>>> IPython-dev at scipy.org
> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>>
> >>>
> >>>
> >>> --
> >>> Brian E. Granger
> >>> Cal Poly State University, San Luis Obispo
> >>> bgranger at calpoly.edu and ellisonbg at gmail.com
> >>> _______________________________________________
> >>> IPython-dev mailing list
> >>> IPython-dev at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> > --
> > Brian E. Granger
> > Cal Poly State University, San Luis Obispo
> > bgranger at calpoly.edu and ellisonbg at gmail.com
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/a87e8f0a/attachment.html>

From benjaminrk at gmail.com  Thu Jul  4 18:47:50 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 4 Jul 2013 15:47:50 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
	<CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
Message-ID: <CAHNn8BUbLD62XXEDozHaAtkm3q02ZiF9dzKCqPgbXwy=6_fcfQ@mail.gmail.com>

One problem is probably the 'reset' styles.  While it makes sense for the
webapp to have just one `style.min.css`, we should probably generate at
least one or two more css files, so that others can use just the IPython
style without inheriting reset and (maybe) bootstrap.


On Thu, Jul 4, 2013 at 2:23 PM, Jacob Vanderplas
<jakevdp at cs.washington.edu>wrote:

> On Thu, Jul 4, 2013 at 1:42 PM, Min RK <benjaminrk at gmail.com> wrote:
>
>> We are going to have to explore a few options, it's not obvious what's
>> the best choice for various contexts.  It will help to have the various
>> sticky points affecting Jake as data points.
>>
>
> I like the idea of wrapping everything so it's self-contained. My "sticky
> points" are basically all due to this: either places where the blog theme
> overrides the notebook and makes it look ugly, or places where the notebook
> css overrides the blog theme and makes it look ugly.  Most of my hacks to
> the CSS have involved getting the template to the point of being
> essentially what you're describing here.  Because it's not designed that
> way from the ground up, however, some of the styles aren't carrying down to
> multiply nested elements.
>
> I think the best solution would have the following:
>
> - use non-generic CSS class names (e.g. "ipynb-cell" rather than "cell",
> and "ipynb-highlight" rather than "highlight"),
> or nest any such generic names within an "ipynb" namespace.
> - make no global CSS modifications (e.g. global-level header formatting,
> page formatting, body formatting)
> - design the CSS so that notebook sections in a div, such as <div
> class="ipynb"></div>, so that multiple notebook snippets can easily be
> included in a page which otherwise has arbitrary html/css styling.
>
> I don't have a great deal of experience with CSS, but from my
> understanding, this sort of approach should be possible.
>    Jake
>
>
>>
>> -MinRK
>>
>> On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> > One other usage case - if someone wants to integrate ipython assets
>> > into their web page at a finer grained scale - imagine someone wanting
>> > to put a single cell at various locations on their non-ipython using
>> > page.  In that case, each IPython containing div would have to have
>> > the .ipython class?  I don't think that is a problem, but I just want
>> > to make sure we are covering the different usage cases...
>> >
>> > Cheers,
>> >
>> > Brian
>> >
>> > On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
>> >> There are two ways of applying style to descendants - immediate (with
>> `>`)
>> >> and any level (with space, or as you posted).
>> >>
>> >> So if we just put all of our style in a `.ipython {...}` block, then
>> our
>> >> classes in a single `.ipython` div, our style would not apply outside
>> that
>> >> context.
>> >>
>> >> Illustration: http://nbviewer.ipython.org/5929801
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com>
>> wrote:
>> >>>
>> >>> Does the less syntax for nesting:
>> >>>
>> >>> .ipython {
>> >>>
>> >>>  .cell {}
>> >>>
>> >>> }
>> >>>
>> >>> map to only the immediate children or all descendents?  If it only
>> >>> works for immediate children, our css stylesheets would have to become
>> >>> horribly nested.
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Brian
>> >>>
>> >>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>> >>>> Matthias is right - we don't actually have to prefix every name, we
>> can
>> >>>> just
>> >>>> scope our CSS based on the outer ipython container.
>> >>>>
>> >>>>
>> >>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>> >>>> <bussonniermatthias at gmail.com> wrote:
>> >>>>>
>> >>>>> Short from my phone:
>> >>>>>
>> >>>>> Could we add an .ipython class to body,
>> >>>>> And wrap the all less import in a .ipython{}
>> >>>>>
>> >>>>> CSS will then only apply to elements that are included in a
>> div.ipython
>> >>>>> ?
>> >>>>>
>> >>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>> >>>>>
>> >>>>>> There are a couple of factors going on:
>> >>>>>>
>> >>>>>> * We are now minimizing the stylesheet which obscures everything.
>> >>>>>> * We are transitioning to using bootstrap.  You may have direct
>> >>>>>> conflicts with bootstrap classes as well as ours
>> >>>>>> * Our css classes are horribly named - generic names like "cell" or
>> >>>>>> "selected".  To address this we are planning on renaming our css
>> >>>>>> classes using the following convention:
>> >>>>>>
>> >>>>>> ipy-cell-selected
>> >>>>>> ipy-notebook-foo
>> >>>>>>
>> >>>>>> We probably won't have time to do all of our classes before 1.0,
>> but
>> >>>>>> we can prioritize the ones you are having problems with.  Can you
>> open
>> >>>>>> an issue for this and provide us with a list of the ones you are
>> >>>>>> running into?
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>>
>> >>>>>> Brian
>> >>>>>>
>> >>>>>>
>> >>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>> >>>>>> <jakevdp at cs.washington.edu> wrote:
>> >>>>>>> Hi,
>> >>>>>>> I've been working on adapting the recently-merged nbconvert
>> refactor
>> >>>>>>> to
>> >>>>>>> work
>> >>>>>>> with my Pelican blogging plugin, and am having a really difficult
>> >>>>>>> time.
>> >>>>>>> In
>> >>>>>>> particular, the header content produced by the new nbconvert (via
>> >>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared
>> to
>> >>>>>>> the
>> >>>>>>> old
>> >>>>>>> version.  This is conflicting with the default blog theme, and
>> >>>>>>> leading
>> >>>>>>> to
>> >>>>>>> some results which are not very pretty.
>> >>>>>>>
>> >>>>>>> In the old iteration of the notebook plugin, I used about half a
>> >>>>>>> dozen
>> >>>>>>> regular expression replace statements to modify the stylesheet &
>> >>>>>>> content and
>> >>>>>>> make it play well with the blog style.  My hope was that the new
>> >>>>>>> nbconvert
>> >>>>>>> would be flexible enough to obviate the need for this sort of
>> >>>>>>> text-mangling;
>> >>>>>>> in reality the required text-mangling in the new version is much
>> >>>>>>> more
>> >>>>>>> extensive.  I've been working at it for several hours, and still
>> >>>>>>> don't
>> >>>>>>> have
>> >>>>>>> a suitable solution that leads to nicely-formatted notebooks
>> within
>> >>>>>>> blog
>> >>>>>>> posts.
>> >>>>>>>
>> >>>>>>> For those of you familiar with the new nbconvert: what is the
>> reason
>> >>>>>>> for the
>> >>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
>> >>>>>>> there
>> >>>>>>> a
>> >>>>>>> good way to recover the old style sheet within the new codebase?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>   Jake
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> IPython-dev mailing list
>> >>>>>>> IPython-dev at scipy.org
>> >>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Brian E. Granger
>> >>>>>> Cal Poly State University, San Luis Obispo
>> >>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >>>>>> _______________________________________________
>> >>>>>> IPython-dev mailing list
>> >>>>>> IPython-dev at scipy.org
>> >>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> IPython-dev mailing list
>> >>>>> IPython-dev at scipy.org
>> >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> IPython-dev mailing list
>> >>>> IPython-dev at scipy.org
>> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Brian E. Granger
>> >>> Cal Poly State University, San Luis Obispo
>> >>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >>> _______________________________________________
>> >>> IPython-dev mailing list
>> >>> IPython-dev at scipy.org
>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> > --
>> > Brian E. Granger
>> > Cal Poly State University, San Luis Obispo
>> > bgranger at calpoly.edu and ellisonbg at gmail.com
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130704/b0265ac2/attachment.html>

From ellisonbg at gmail.com  Fri Jul  5 01:19:43 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 4 Jul 2013 22:19:43 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAHNn8BUbLD62XXEDozHaAtkm3q02ZiF9dzKCqPgbXwy=6_fcfQ@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
	<CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
	<CAHNn8BUbLD62XXEDozHaAtkm3q02ZiF9dzKCqPgbXwy=6_fcfQ@mail.gmail.com>
Message-ID: <CAH4pYpSmwy1_FNYsT4UAHU5dfAgQeLCVEZWnPVAtY32xC5EZNQ@mail.gmail.com>

That bring up another issue.  We are currently using both the
boilerplate style reset as well as bootstrap which does its own reset.
 We can probably get rid of the non-bootstrap reset stuff.

I think it will be very difficult to split up style.min.css because
our own CSS uses bootstrap variables.  That is one of the downsides of
using less - it sort of all has to be built at the same time...



On Thu, Jul 4, 2013 at 3:47 PM, MinRK <benjaminrk at gmail.com> wrote:
> One problem is probably the 'reset' styles.  While it makes sense for the
> webapp to have just one `style.min.css`, we should probably generate at
> least one or two more css files, so that others can use just the IPython
> style without inheriting reset and (maybe) bootstrap.
>
>
> On Thu, Jul 4, 2013 at 2:23 PM, Jacob Vanderplas <jakevdp at cs.washington.edu>
> wrote:
>>
>> On Thu, Jul 4, 2013 at 1:42 PM, Min RK <benjaminrk at gmail.com> wrote:
>>>
>>> We are going to have to explore a few options, it's not obvious what's
>>> the best choice for various contexts.  It will help to have the various
>>> sticky points affecting Jake as data points.
>>
>>
>> I like the idea of wrapping everything so it's self-contained. My "sticky
>> points" are basically all due to this: either places where the blog theme
>> overrides the notebook and makes it look ugly, or places where the notebook
>> css overrides the blog theme and makes it look ugly.  Most of my hacks to
>> the CSS have involved getting the template to the point of being essentially
>> what you're describing here.  Because it's not designed that way from the
>> ground up, however, some of the styles aren't carrying down to multiply
>> nested elements.
>>
>> I think the best solution would have the following:
>>
>> - use non-generic CSS class names (e.g. "ipynb-cell" rather than "cell",
>> and "ipynb-highlight" rather than "highlight"),
>> or nest any such generic names within an "ipynb" namespace.
>> - make no global CSS modifications (e.g. global-level header formatting,
>> page formatting, body formatting)
>> - design the CSS so that notebook sections in a div, such as <div
>> class="ipynb"></div>, so that multiple notebook snippets can easily be
>> included in a page which otherwise has arbitrary html/css styling.
>>
>> I don't have a great deal of experience with CSS, but from my
>> understanding, this sort of approach should be possible.
>>    Jake
>>
>>>
>>>
>>> -MinRK
>>>
>>> On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:
>>>
>>> > One other usage case - if someone wants to integrate ipython assets
>>> > into their web page at a finer grained scale - imagine someone wanting
>>> > to put a single cell at various locations on their non-ipython using
>>> > page.  In that case, each IPython containing div would have to have
>>> > the .ipython class?  I don't think that is a problem, but I just want
>>> > to make sure we are covering the different usage cases...
>>> >
>>> > Cheers,
>>> >
>>> > Brian
>>> >
>>> > On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
>>> >> There are two ways of applying style to descendants - immediate (with
>>> >> `>`)
>>> >> and any level (with space, or as you posted).
>>> >>
>>> >> So if we just put all of our style in a `.ipython {...}` block, then
>>> >> our
>>> >> classes in a single `.ipython` div, our style would not apply outside
>>> >> that
>>> >> context.
>>> >>
>>> >> Illustration: http://nbviewer.ipython.org/5929801
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> Does the less syntax for nesting:
>>> >>>
>>> >>> .ipython {
>>> >>>
>>> >>>  .cell {}
>>> >>>
>>> >>> }
>>> >>>
>>> >>> map to only the immediate children or all descendents?  If it only
>>> >>> works for immediate children, our css stylesheets would have to
>>> >>> become
>>> >>> horribly nested.
>>> >>>
>>> >>> Cheers,
>>> >>>
>>> >>> Brian
>>> >>>
>>> >>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>>> >>>> Matthias is right - we don't actually have to prefix every name, we
>>> >>>> can
>>> >>>> just
>>> >>>> scope our CSS based on the outer ipython container.
>>> >>>>
>>> >>>>
>>> >>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>>> >>>> <bussonniermatthias at gmail.com> wrote:
>>> >>>>>
>>> >>>>> Short from my phone:
>>> >>>>>
>>> >>>>> Could we add an .ipython class to body,
>>> >>>>> And wrap the all less import in a .ipython{}
>>> >>>>>
>>> >>>>> CSS will then only apply to elements that are included in a
>>> >>>>> div.ipython
>>> >>>>> ?
>>> >>>>>
>>> >>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>>> >>>>>
>>> >>>>>> There are a couple of factors going on:
>>> >>>>>>
>>> >>>>>> * We are now minimizing the stylesheet which obscures everything.
>>> >>>>>> * We are transitioning to using bootstrap.  You may have direct
>>> >>>>>> conflicts with bootstrap classes as well as ours
>>> >>>>>> * Our css classes are horribly named - generic names like "cell"
>>> >>>>>> or
>>> >>>>>> "selected".  To address this we are planning on renaming our css
>>> >>>>>> classes using the following convention:
>>> >>>>>>
>>> >>>>>> ipy-cell-selected
>>> >>>>>> ipy-notebook-foo
>>> >>>>>>
>>> >>>>>> We probably won't have time to do all of our classes before 1.0,
>>> >>>>>> but
>>> >>>>>> we can prioritize the ones you are having problems with.  Can you
>>> >>>>>> open
>>> >>>>>> an issue for this and provide us with a list of the ones you are
>>> >>>>>> running into?
>>> >>>>>>
>>> >>>>>> Cheers,
>>> >>>>>>
>>> >>>>>> Brian
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>>> >>>>>> <jakevdp at cs.washington.edu> wrote:
>>> >>>>>>> Hi,
>>> >>>>>>> I've been working on adapting the recently-merged nbconvert
>>> >>>>>>> refactor
>>> >>>>>>> to
>>> >>>>>>> work
>>> >>>>>>> with my Pelican blogging plugin, and am having a really difficult
>>> >>>>>>> time.
>>> >>>>>>> In
>>> >>>>>>> particular, the header content produced by the new nbconvert (via
>>> >>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared
>>> >>>>>>> to
>>> >>>>>>> the
>>> >>>>>>> old
>>> >>>>>>> version.  This is conflicting with the default blog theme, and
>>> >>>>>>> leading
>>> >>>>>>> to
>>> >>>>>>> some results which are not very pretty.
>>> >>>>>>>
>>> >>>>>>> In the old iteration of the notebook plugin, I used about half a
>>> >>>>>>> dozen
>>> >>>>>>> regular expression replace statements to modify the stylesheet &
>>> >>>>>>> content and
>>> >>>>>>> make it play well with the blog style.  My hope was that the new
>>> >>>>>>> nbconvert
>>> >>>>>>> would be flexible enough to obviate the need for this sort of
>>> >>>>>>> text-mangling;
>>> >>>>>>> in reality the required text-mangling in the new version is much
>>> >>>>>>> more
>>> >>>>>>> extensive.  I've been working at it for several hours, and still
>>> >>>>>>> don't
>>> >>>>>>> have
>>> >>>>>>> a suitable solution that leads to nicely-formatted notebooks
>>> >>>>>>> within
>>> >>>>>>> blog
>>> >>>>>>> posts.
>>> >>>>>>>
>>> >>>>>>> For those of you familiar with the new nbconvert: what is the
>>> >>>>>>> reason
>>> >>>>>>> for the
>>> >>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
>>> >>>>>>> there
>>> >>>>>>> a
>>> >>>>>>> good way to recover the old style sheet within the new codebase?
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>>   Jake
>>> >>>>>>>
>>> >>>>>>> _______________________________________________
>>> >>>>>>> IPython-dev mailing list
>>> >>>>>>> IPython-dev at scipy.org
>>> >>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Brian E. Granger
>>> >>>>>> Cal Poly State University, San Luis Obispo
>>> >>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> >>>>>> _______________________________________________
>>> >>>>>> IPython-dev mailing list
>>> >>>>>> IPython-dev at scipy.org
>>> >>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >>>>>
>>> >>>>>
>>> >>>>> _______________________________________________
>>> >>>>> IPython-dev mailing list
>>> >>>>> IPython-dev at scipy.org
>>> >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >>>>
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> IPython-dev mailing list
>>> >>>> IPython-dev at scipy.org
>>> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Brian E. Granger
>>> >>> Cal Poly State University, San Luis Obispo
>>> >>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> >>> _______________________________________________
>>> >>> IPython-dev mailing list
>>> >>> IPython-dev at scipy.org
>>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> IPython-dev mailing list
>>> >> IPython-dev at scipy.org
>>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> >
>>> >
>>> >
>>> > --
>>> > Brian E. Granger
>>> > Cal Poly State University, San Luis Obispo
>>> > bgranger at calpoly.edu and ellisonbg at gmail.com
>>> > _______________________________________________
>>> > IPython-dev mailing list
>>> > IPython-dev at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Fri Jul  5 01:22:26 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 4 Jul 2013 22:22:26 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
	<CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
Message-ID: <CAH4pYpSLyUo6czfisnTCH+EkqfomuF_54cg0iR9WX_c6VHrBBQ@mail.gmail.com>

> I like the idea of wrapping everything so it's self-contained. My "sticky
> points" are basically all due to this: either places where the blog theme
> overrides the notebook and makes it look ugly, or places where the notebook
> css overrides the blog theme and makes it look ugly.  Most of my hacks to
> the CSS have involved getting the template to the point of being essentially
> what you're describing here.  Because it's not designed that way from the
> ground up, however, some of the styles aren't carrying down to multiply
> nested elements.
>
> I think the best solution would have the following:
>
> - use non-generic CSS class names (e.g. "ipynb-cell" rather than "cell", and
> "ipynb-highlight" rather than "highlight"),
> or nest any such generic names within an "ipynb" namespace.

The nesting will prevent ipython's styles from affecting your own.
But it would still be possible for your own styles to affect ipython
managed elements.  css has a notion of priority.  If you define
styling for a cell class that has higher priority than our nested
rules, it will win out.  The only way to avoid that is to use truly
different names for css classes.

> - make no global CSS modifications (e.g. global-level header formatting,
> page formatting, body formatting)

Well, except for the fact that we use bootstrap, which does these
things...I don't see any way around that.

> - design the CSS so that notebook sections in a div, such as <div
> class="ipynb"></div>, so that multiple notebook snippets can easily be
> included in a page which otherwise has arbitrary html/css styling.

Yes, this should definitely work.

> I don't have a great deal of experience with CSS, but from my understanding,
> this sort of approach should be possible.
>    Jake
>
>>
>>
>> -MinRK
>>
>> On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> > One other usage case - if someone wants to integrate ipython assets
>> > into their web page at a finer grained scale - imagine someone wanting
>> > to put a single cell at various locations on their non-ipython using
>> > page.  In that case, each IPython containing div would have to have
>> > the .ipython class?  I don't think that is a problem, but I just want
>> > to make sure we are covering the different usage cases...
>> >
>> > Cheers,
>> >
>> > Brian
>> >
>> > On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
>> >> There are two ways of applying style to descendants - immediate (with
>> >> `>`)
>> >> and any level (with space, or as you posted).
>> >>
>> >> So if we just put all of our style in a `.ipython {...}` block, then
>> >> our
>> >> classes in a single `.ipython` div, our style would not apply outside
>> >> that
>> >> context.
>> >>
>> >> Illustration: http://nbviewer.ipython.org/5929801
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com>
>> >> wrote:
>> >>>
>> >>> Does the less syntax for nesting:
>> >>>
>> >>> .ipython {
>> >>>
>> >>>  .cell {}
>> >>>
>> >>> }
>> >>>
>> >>> map to only the immediate children or all descendents?  If it only
>> >>> works for immediate children, our css stylesheets would have to become
>> >>> horribly nested.
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Brian
>> >>>
>> >>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>> >>>> Matthias is right - we don't actually have to prefix every name, we
>> >>>> can
>> >>>> just
>> >>>> scope our CSS based on the outer ipython container.
>> >>>>
>> >>>>
>> >>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>> >>>> <bussonniermatthias at gmail.com> wrote:
>> >>>>>
>> >>>>> Short from my phone:
>> >>>>>
>> >>>>> Could we add an .ipython class to body,
>> >>>>> And wrap the all less import in a .ipython{}
>> >>>>>
>> >>>>> CSS will then only apply to elements that are included in a
>> >>>>> div.ipython
>> >>>>> ?
>> >>>>>
>> >>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>> >>>>>
>> >>>>>> There are a couple of factors going on:
>> >>>>>>
>> >>>>>> * We are now minimizing the stylesheet which obscures everything.
>> >>>>>> * We are transitioning to using bootstrap.  You may have direct
>> >>>>>> conflicts with bootstrap classes as well as ours
>> >>>>>> * Our css classes are horribly named - generic names like "cell" or
>> >>>>>> "selected".  To address this we are planning on renaming our css
>> >>>>>> classes using the following convention:
>> >>>>>>
>> >>>>>> ipy-cell-selected
>> >>>>>> ipy-notebook-foo
>> >>>>>>
>> >>>>>> We probably won't have time to do all of our classes before 1.0,
>> >>>>>> but
>> >>>>>> we can prioritize the ones you are having problems with.  Can you
>> >>>>>> open
>> >>>>>> an issue for this and provide us with a list of the ones you are
>> >>>>>> running into?
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>>
>> >>>>>> Brian
>> >>>>>>
>> >>>>>>
>> >>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>> >>>>>> <jakevdp at cs.washington.edu> wrote:
>> >>>>>>> Hi,
>> >>>>>>> I've been working on adapting the recently-merged nbconvert
>> >>>>>>> refactor
>> >>>>>>> to
>> >>>>>>> work
>> >>>>>>> with my Pelican blogging plugin, and am having a really difficult
>> >>>>>>> time.
>> >>>>>>> In
>> >>>>>>> particular, the header content produced by the new nbconvert (via
>> >>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared
>> >>>>>>> to
>> >>>>>>> the
>> >>>>>>> old
>> >>>>>>> version.  This is conflicting with the default blog theme, and
>> >>>>>>> leading
>> >>>>>>> to
>> >>>>>>> some results which are not very pretty.
>> >>>>>>>
>> >>>>>>> In the old iteration of the notebook plugin, I used about half a
>> >>>>>>> dozen
>> >>>>>>> regular expression replace statements to modify the stylesheet &
>> >>>>>>> content and
>> >>>>>>> make it play well with the blog style.  My hope was that the new
>> >>>>>>> nbconvert
>> >>>>>>> would be flexible enough to obviate the need for this sort of
>> >>>>>>> text-mangling;
>> >>>>>>> in reality the required text-mangling in the new version is much
>> >>>>>>> more
>> >>>>>>> extensive.  I've been working at it for several hours, and still
>> >>>>>>> don't
>> >>>>>>> have
>> >>>>>>> a suitable solution that leads to nicely-formatted notebooks
>> >>>>>>> within
>> >>>>>>> blog
>> >>>>>>> posts.
>> >>>>>>>
>> >>>>>>> For those of you familiar with the new nbconvert: what is the
>> >>>>>>> reason
>> >>>>>>> for the
>> >>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
>> >>>>>>> there
>> >>>>>>> a
>> >>>>>>> good way to recover the old style sheet within the new codebase?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>   Jake
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> IPython-dev mailing list
>> >>>>>>> IPython-dev at scipy.org
>> >>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Brian E. Granger
>> >>>>>> Cal Poly State University, San Luis Obispo
>> >>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >>>>>> _______________________________________________
>> >>>>>> IPython-dev mailing list
>> >>>>>> IPython-dev at scipy.org
>> >>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> IPython-dev mailing list
>> >>>>> IPython-dev at scipy.org
>> >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> IPython-dev mailing list
>> >>>> IPython-dev at scipy.org
>> >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Brian E. Granger
>> >>> Cal Poly State University, San Luis Obispo
>> >>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >>> _______________________________________________
>> >>> IPython-dev mailing list
>> >>> IPython-dev at scipy.org
>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> > --
>> > Brian E. Granger
>> > Cal Poly State University, San Luis Obispo
>> > bgranger at calpoly.edu and ellisonbg at gmail.com
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From benjaminrk at gmail.com  Fri Jul  5 02:33:15 2013
From: benjaminrk at gmail.com (Min RK)
Date: Thu, 4 Jul 2013 23:33:15 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAH4pYpSmwy1_FNYsT4UAHU5dfAgQeLCVEZWnPVAtY32xC5EZNQ@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
	<CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
	<CAHNn8BUbLD62XXEDozHaAtkm3q02ZiF9dzKCqPgbXwy=6_fcfQ@mail.gmail.com>
	<CAH4pYpSmwy1_FNYsT4UAHU5dfAgQeLCVEZWnPVAtY32xC5EZNQ@mail.gmail.com>
Message-ID: <283D2D64-D9CE-41FE-A1F2-D7A89830508C@gmail.com>

I'm not sure that's actually true. I will experiment with generating subset CSS files tomorrow.

-MinRK

On Jul 4, 2013, at 22:19, Brian Granger <ellisonbg at gmail.com> wrote:

> That bring up another issue.  We are currently using both the
> boilerplate style reset as well as bootstrap which does its own reset.
> We can probably get rid of the non-bootstrap reset stuff.
> 
> I think it will be very difficult to split up style.min.css because
> our own CSS uses bootstrap variables.  That is one of the downsides of
> using less - it sort of all has to be built at the same time...
> 
> 
> 
> On Thu, Jul 4, 2013 at 3:47 PM, MinRK <benjaminrk at gmail.com> wrote:
>> One problem is probably the 'reset' styles.  While it makes sense for the
>> webapp to have just one `style.min.css`, we should probably generate at
>> least one or two more css files, so that others can use just the IPython
>> style without inheriting reset and (maybe) bootstrap.
>> 
>> 
>> On Thu, Jul 4, 2013 at 2:23 PM, Jacob Vanderplas <jakevdp at cs.washington.edu>
>> wrote:
>>> 
>>> On Thu, Jul 4, 2013 at 1:42 PM, Min RK <benjaminrk at gmail.com> wrote:
>>>> 
>>>> We are going to have to explore a few options, it's not obvious what's
>>>> the best choice for various contexts.  It will help to have the various
>>>> sticky points affecting Jake as data points.
>>> 
>>> 
>>> I like the idea of wrapping everything so it's self-contained. My "sticky
>>> points" are basically all due to this: either places where the blog theme
>>> overrides the notebook and makes it look ugly, or places where the notebook
>>> css overrides the blog theme and makes it look ugly.  Most of my hacks to
>>> the CSS have involved getting the template to the point of being essentially
>>> what you're describing here.  Because it's not designed that way from the
>>> ground up, however, some of the styles aren't carrying down to multiply
>>> nested elements.
>>> 
>>> I think the best solution would have the following:
>>> 
>>> - use non-generic CSS class names (e.g. "ipynb-cell" rather than "cell",
>>> and "ipynb-highlight" rather than "highlight"),
>>> or nest any such generic names within an "ipynb" namespace.
>>> - make no global CSS modifications (e.g. global-level header formatting,
>>> page formatting, body formatting)
>>> - design the CSS so that notebook sections in a div, such as <div
>>> class="ipynb"></div>, so that multiple notebook snippets can easily be
>>> included in a page which otherwise has arbitrary html/css styling.
>>> 
>>> I don't have a great deal of experience with CSS, but from my
>>> understanding, this sort of approach should be possible.
>>>   Jake
>>> 
>>>> 
>>>> 
>>>> -MinRK
>>>> 
>>>> On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:
>>>> 
>>>>> One other usage case - if someone wants to integrate ipython assets
>>>>> into their web page at a finer grained scale - imagine someone wanting
>>>>> to put a single cell at various locations on their non-ipython using
>>>>> page.  In that case, each IPython containing div would have to have
>>>>> the .ipython class?  I don't think that is a problem, but I just want
>>>>> to make sure we are covering the different usage cases...
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Brian
>>>>> 
>>>>> On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>>>> There are two ways of applying style to descendants - immediate (with
>>>>>> `>`)
>>>>>> and any level (with space, or as you posted).
>>>>>> 
>>>>>> So if we just put all of our style in a `.ipython {...}` block, then
>>>>>> our
>>>>>> classes in a single `.ipython` div, our style would not apply outside
>>>>>> that
>>>>>> context.
>>>>>> 
>>>>>> Illustration: http://nbviewer.ipython.org/5929801
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Does the less syntax for nesting:
>>>>>>> 
>>>>>>> .ipython {
>>>>>>> 
>>>>>>> .cell {}
>>>>>>> 
>>>>>>> }
>>>>>>> 
>>>>>>> map to only the immediate children or all descendents?  If it only
>>>>>>> works for immediate children, our css stylesheets would have to
>>>>>>> become
>>>>>>> horribly nested.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> Brian
>>>>>>> 
>>>>>>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>>>>>> Matthias is right - we don't actually have to prefix every name, we
>>>>>>>> can
>>>>>>>> just
>>>>>>>> scope our CSS based on the outer ipython container.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>>>>>>>> <bussonniermatthias at gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Short from my phone:
>>>>>>>>> 
>>>>>>>>> Could we add an .ipython class to body,
>>>>>>>>> And wrap the all less import in a .ipython{}
>>>>>>>>> 
>>>>>>>>> CSS will then only apply to elements that are included in a
>>>>>>>>> div.ipython
>>>>>>>>> ?
>>>>>>>>> 
>>>>>>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>>>>>>>>> 
>>>>>>>>>> There are a couple of factors going on:
>>>>>>>>>> 
>>>>>>>>>> * We are now minimizing the stylesheet which obscures everything.
>>>>>>>>>> * We are transitioning to using bootstrap.  You may have direct
>>>>>>>>>> conflicts with bootstrap classes as well as ours
>>>>>>>>>> * Our css classes are horribly named - generic names like "cell"
>>>>>>>>>> or
>>>>>>>>>> "selected".  To address this we are planning on renaming our css
>>>>>>>>>> classes using the following convention:
>>>>>>>>>> 
>>>>>>>>>> ipy-cell-selected
>>>>>>>>>> ipy-notebook-foo
>>>>>>>>>> 
>>>>>>>>>> We probably won't have time to do all of our classes before 1.0,
>>>>>>>>>> but
>>>>>>>>>> we can prioritize the ones you are having problems with.  Can you
>>>>>>>>>> open
>>>>>>>>>> an issue for this and provide us with a list of the ones you are
>>>>>>>>>> running into?
>>>>>>>>>> 
>>>>>>>>>> Cheers,
>>>>>>>>>> 
>>>>>>>>>> Brian
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>>>>>>>>>> <jakevdp at cs.washington.edu> wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>> I've been working on adapting the recently-merged nbconvert
>>>>>>>>>>> refactor
>>>>>>>>>>> to
>>>>>>>>>>> work
>>>>>>>>>>> with my Pelican blogging plugin, and am having a really difficult
>>>>>>>>>>> time.
>>>>>>>>>>> In
>>>>>>>>>>> particular, the header content produced by the new nbconvert (via
>>>>>>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared
>>>>>>>>>>> to
>>>>>>>>>>> the
>>>>>>>>>>> old
>>>>>>>>>>> version.  This is conflicting with the default blog theme, and
>>>>>>>>>>> leading
>>>>>>>>>>> to
>>>>>>>>>>> some results which are not very pretty.
>>>>>>>>>>> 
>>>>>>>>>>> In the old iteration of the notebook plugin, I used about half a
>>>>>>>>>>> dozen
>>>>>>>>>>> regular expression replace statements to modify the stylesheet &
>>>>>>>>>>> content and
>>>>>>>>>>> make it play well with the blog style.  My hope was that the new
>>>>>>>>>>> nbconvert
>>>>>>>>>>> would be flexible enough to obviate the need for this sort of
>>>>>>>>>>> text-mangling;
>>>>>>>>>>> in reality the required text-mangling in the new version is much
>>>>>>>>>>> more
>>>>>>>>>>> extensive.  I've been working at it for several hours, and still
>>>>>>>>>>> don't
>>>>>>>>>>> have
>>>>>>>>>>> a suitable solution that leads to nicely-formatted notebooks
>>>>>>>>>>> within
>>>>>>>>>>> blog
>>>>>>>>>>> posts.
>>>>>>>>>>> 
>>>>>>>>>>> For those of you familiar with the new nbconvert: what is the
>>>>>>>>>>> reason
>>>>>>>>>>> for the
>>>>>>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
>>>>>>>>>>> there
>>>>>>>>>>> a
>>>>>>>>>>> good way to recover the old style sheet within the new codebase?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>>  Jake
>>>>>>>>>>> 
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> IPython-dev mailing list
>>>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Brian E. Granger
>>>>>>>>>> Cal Poly State University, San Luis Obispo
>>>>>>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>>>>>>> _______________________________________________
>>>>>>>>>> IPython-dev mailing list
>>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> IPython-dev mailing list
>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> IPython-dev mailing list
>>>>>>>> IPython-dev at scipy.org
>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Brian E. Granger
>>>>>>> Cal Poly State University, San Luis Obispo
>>>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>>>> _______________________________________________
>>>>>>> IPython-dev mailing list
>>>>>>> IPython-dev at scipy.org
>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Brian E. Granger
>>>>> Cal Poly State University, San Luis Obispo
>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>> 
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Fri Jul  5 02:38:54 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 4 Jul 2013 23:38:54 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <283D2D64-D9CE-41FE-A1F2-D7A89830508C@gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
	<CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
	<CAHNn8BUbLD62XXEDozHaAtkm3q02ZiF9dzKCqPgbXwy=6_fcfQ@mail.gmail.com>
	<CAH4pYpSmwy1_FNYsT4UAHU5dfAgQeLCVEZWnPVAtY32xC5EZNQ@mail.gmail.com>
	<283D2D64-D9CE-41FE-A1F2-D7A89830508C@gmail.com>
Message-ID: <CAH4pYpQF-VKpfyCFLNAtUCxQFasogtHDeOzZE_9FBZ_orbD6fA@mail.gmail.com>

OK, great.

On Thu, Jul 4, 2013 at 11:33 PM, Min RK <benjaminrk at gmail.com> wrote:
> I'm not sure that's actually true. I will experiment with generating subset CSS files tomorrow.
>
> -MinRK
>
> On Jul 4, 2013, at 22:19, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> That bring up another issue.  We are currently using both the
>> boilerplate style reset as well as bootstrap which does its own reset.
>> We can probably get rid of the non-bootstrap reset stuff.
>>
>> I think it will be very difficult to split up style.min.css because
>> our own CSS uses bootstrap variables.  That is one of the downsides of
>> using less - it sort of all has to be built at the same time...
>>
>>
>>
>> On Thu, Jul 4, 2013 at 3:47 PM, MinRK <benjaminrk at gmail.com> wrote:
>>> One problem is probably the 'reset' styles.  While it makes sense for the
>>> webapp to have just one `style.min.css`, we should probably generate at
>>> least one or two more css files, so that others can use just the IPython
>>> style without inheriting reset and (maybe) bootstrap.
>>>
>>>
>>> On Thu, Jul 4, 2013 at 2:23 PM, Jacob Vanderplas <jakevdp at cs.washington.edu>
>>> wrote:
>>>>
>>>> On Thu, Jul 4, 2013 at 1:42 PM, Min RK <benjaminrk at gmail.com> wrote:
>>>>>
>>>>> We are going to have to explore a few options, it's not obvious what's
>>>>> the best choice for various contexts.  It will help to have the various
>>>>> sticky points affecting Jake as data points.
>>>>
>>>>
>>>> I like the idea of wrapping everything so it's self-contained. My "sticky
>>>> points" are basically all due to this: either places where the blog theme
>>>> overrides the notebook and makes it look ugly, or places where the notebook
>>>> css overrides the blog theme and makes it look ugly.  Most of my hacks to
>>>> the CSS have involved getting the template to the point of being essentially
>>>> what you're describing here.  Because it's not designed that way from the
>>>> ground up, however, some of the styles aren't carrying down to multiply
>>>> nested elements.
>>>>
>>>> I think the best solution would have the following:
>>>>
>>>> - use non-generic CSS class names (e.g. "ipynb-cell" rather than "cell",
>>>> and "ipynb-highlight" rather than "highlight"),
>>>> or nest any such generic names within an "ipynb" namespace.
>>>> - make no global CSS modifications (e.g. global-level header formatting,
>>>> page formatting, body formatting)
>>>> - design the CSS so that notebook sections in a div, such as <div
>>>> class="ipynb"></div>, so that multiple notebook snippets can easily be
>>>> included in a page which otherwise has arbitrary html/css styling.
>>>>
>>>> I don't have a great deal of experience with CSS, but from my
>>>> understanding, this sort of approach should be possible.
>>>>   Jake
>>>>
>>>>>
>>>>>
>>>>> -MinRK
>>>>>
>>>>> On Jul 4, 2013, at 13:37, Brian Granger <ellisonbg at gmail.com> wrote:
>>>>>
>>>>>> One other usage case - if someone wants to integrate ipython assets
>>>>>> into their web page at a finer grained scale - imagine someone wanting
>>>>>> to put a single cell at various locations on their non-ipython using
>>>>>> page.  In that case, each IPython containing div would have to have
>>>>>> the .ipython class?  I don't think that is a problem, but I just want
>>>>>> to make sure we are covering the different usage cases...
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Brian
>>>>>>
>>>>>> On Thu, Jul 4, 2013 at 12:38 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>>>>> There are two ways of applying style to descendants - immediate (with
>>>>>>> `>`)
>>>>>>> and any level (with space, or as you posted).
>>>>>>>
>>>>>>> So if we just put all of our style in a `.ipython {...}` block, then
>>>>>>> our
>>>>>>> classes in a single `.ipython` div, our style would not apply outside
>>>>>>> that
>>>>>>> context.
>>>>>>>
>>>>>>> Illustration: http://nbviewer.ipython.org/5929801
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 4, 2013 at 12:25 PM, Brian Granger <ellisonbg at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Does the less syntax for nesting:
>>>>>>>>
>>>>>>>> .ipython {
>>>>>>>>
>>>>>>>> .cell {}
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> map to only the immediate children or all descendents?  If it only
>>>>>>>> works for immediate children, our css stylesheets would have to
>>>>>>>> become
>>>>>>>> horribly nested.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Brian
>>>>>>>>
>>>>>>>> On Thu, Jul 4, 2013 at 12:11 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>>>>>>> Matthias is right - we don't actually have to prefix every name, we
>>>>>>>>> can
>>>>>>>>> just
>>>>>>>>> scope our CSS based on the outer ipython container.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jul 4, 2013 at 12:04 PM, Matthias Bussonnier
>>>>>>>>> <bussonniermatthias at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Short from my phone:
>>>>>>>>>>
>>>>>>>>>> Could we add an .ipython class to body,
>>>>>>>>>> And wrap the all less import in a .ipython{}
>>>>>>>>>>
>>>>>>>>>> CSS will then only apply to elements that are included in a
>>>>>>>>>> div.ipython
>>>>>>>>>> ?
>>>>>>>>>>
>>>>>>>>>> Le jeudi 4 juillet 2013, Brian Granger a ?crit :
>>>>>>>>>>
>>>>>>>>>>> There are a couple of factors going on:
>>>>>>>>>>>
>>>>>>>>>>> * We are now minimizing the stylesheet which obscures everything.
>>>>>>>>>>> * We are transitioning to using bootstrap.  You may have direct
>>>>>>>>>>> conflicts with bootstrap classes as well as ours
>>>>>>>>>>> * Our css classes are horribly named - generic names like "cell"
>>>>>>>>>>> or
>>>>>>>>>>> "selected".  To address this we are planning on renaming our css
>>>>>>>>>>> classes using the following convention:
>>>>>>>>>>>
>>>>>>>>>>> ipy-cell-selected
>>>>>>>>>>> ipy-notebook-foo
>>>>>>>>>>>
>>>>>>>>>>> We probably won't have time to do all of our classes before 1.0,
>>>>>>>>>>> but
>>>>>>>>>>> we can prioritize the ones you are having problems with.  Can you
>>>>>>>>>>> open
>>>>>>>>>>> an issue for this and provide us with a list of the ones you are
>>>>>>>>>>> running into?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Brian
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 4, 2013 at 9:10 AM, Jacob Vanderplas
>>>>>>>>>>> <jakevdp at cs.washington.edu> wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I've been working on adapting the recently-merged nbconvert
>>>>>>>>>>>> refactor
>>>>>>>>>>>> to
>>>>>>>>>>>> work
>>>>>>>>>>>> with my Pelican blogging plugin, and am having a really difficult
>>>>>>>>>>>> time.
>>>>>>>>>>>> In
>>>>>>>>>>>> particular, the header content produced by the new nbconvert (via
>>>>>>>>>>>> CSSHTMLHeaderTransformer) contains a lot of extra stuff compared
>>>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> old
>>>>>>>>>>>> version.  This is conflicting with the default blog theme, and
>>>>>>>>>>>> leading
>>>>>>>>>>>> to
>>>>>>>>>>>> some results which are not very pretty.
>>>>>>>>>>>>
>>>>>>>>>>>> In the old iteration of the notebook plugin, I used about half a
>>>>>>>>>>>> dozen
>>>>>>>>>>>> regular expression replace statements to modify the stylesheet &
>>>>>>>>>>>> content and
>>>>>>>>>>>> make it play well with the blog style.  My hope was that the new
>>>>>>>>>>>> nbconvert
>>>>>>>>>>>> would be flexible enough to obviate the need for this sort of
>>>>>>>>>>>> text-mangling;
>>>>>>>>>>>> in reality the required text-mangling in the new version is much
>>>>>>>>>>>> more
>>>>>>>>>>>> extensive.  I've been working at it for several hours, and still
>>>>>>>>>>>> don't
>>>>>>>>>>>> have
>>>>>>>>>>>> a suitable solution that leads to nicely-formatted notebooks
>>>>>>>>>>>> within
>>>>>>>>>>>> blog
>>>>>>>>>>>> posts.
>>>>>>>>>>>>
>>>>>>>>>>>> For those of you familiar with the new nbconvert: what is the
>>>>>>>>>>>> reason
>>>>>>>>>>>> for the
>>>>>>>>>>>> changes in the default CSS styles between nbconvert 1 and 2?  Is
>>>>>>>>>>>> there
>>>>>>>>>>>> a
>>>>>>>>>>>> good way to recover the old style sheet within the new codebase?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>  Jake
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> IPython-dev mailing list
>>>>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Brian E. Granger
>>>>>>>>>>> Cal Poly State University, San Luis Obispo
>>>>>>>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> IPython-dev mailing list
>>>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> IPython-dev mailing list
>>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> IPython-dev mailing list
>>>>>>>>> IPython-dev at scipy.org
>>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Brian E. Granger
>>>>>>>> Cal Poly State University, San Luis Obispo
>>>>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>>>>> _______________________________________________
>>>>>>>> IPython-dev mailing list
>>>>>>>> IPython-dev at scipy.org
>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> IPython-dev mailing list
>>>>>>> IPython-dev at scipy.org
>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Brian E. Granger
>>>>>> Cal Poly State University, San Luis Obispo
>>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Fri Jul  5 02:58:43 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 04 Jul 2013 23:58:43 -0700
Subject: [IPython-dev] nbconvert: trouble with the new style sheet
In-Reply-To: <CAH4pYpSLyUo6czfisnTCH+EkqfomuF_54cg0iR9WX_c6VHrBBQ@mail.gmail.com>
References: <CACpqBg265CxVfLBjzamLu6EHPGfG3ejPaR7iPMDXx0Bc3tGGSQ@mail.gmail.com>
	<CAH4pYpRjV=C+chtyx4wtM-c0hNN_uM_6JN316asve-q96S_zCA@mail.gmail.com>
	<CANJQusWOFZ76B9zUpetXOjN5TCH=VuDuAMdANUN3GG+z9D4AgA@mail.gmail.com>
	<CAHNn8BUJ3Lmju1TT4rGjF3dGaCH9UZ9k6D4YP4aMVtVQ98K6ZA@mail.gmail.com>
	<CAH4pYpRgCY-yqapYiJz_borcLM20RcdZB4CXgC=gSEwWTALDYg@mail.gmail.com>
	<CAHNn8BVAgxakE9N1F-o5eDjwtmEVAW6o7CFomrnDDKMMHhD3cg@mail.gmail.com>
	<CAH4pYpRFe9pFpe-3MP0Lap2LhGZVjH+NOLmxBbcmoFnkBLzDxg@mail.gmail.com>
	<A0F9FC0B-872A-491D-A3D8-9B6B3BB48C37@gmail.com>
	<CACpqBg1ANzJPMGzbOvfLPAX5udT04X8DYDU8fLcdjmnhxTtb7A@mail.gmail.com>
	<CAH4pYpSLyUo6czfisnTCH+EkqfomuF_54cg0iR9WX_c6VHrBBQ@mail.gmail.com>
Message-ID: <51D66EA3.6020102@creativetrax.com>

On 7/4/13 10:22 PM, Brian Granger wrote:
> The nesting will prevent ipython's styles from affecting your own.
> But it would still be possible for your own styles to affect ipython
> managed elements.  css has a notion of priority.  If you define
> styling for a cell class that has higher priority than our nested
> rules, it will win out.  The only way to avoid that is to use truly
> different names for css classes.

We have a fair amount of experience with these sorts of problems with 
the Sage cell server (which is designed to be embedded into other 
pages).  It's more than class names.  Things like default spacing, 
margins, padding, as well as styling on various form elements and things 
can mess up the sage cell.  We provide a sagecell_embed.css [1] file 
that basically has all of our css rules declared !important to try to 
minimize issues, but we still see problems from time to time.

I'm really looking forward to the web component stuff and shadow doms 
taking off in the near future:

https://github.com/sagemath/sagecell/issues/405

They will allow us to have a web component that isn't affected by CSS 
cascading into the component.

We also specifically don't use bootstrap because it doesn't embed well 
into random webpages.  Instead, Ira Hanson wrote a small responsive 
layout system when we needed it.  I realize you guys have a different 
use case with the IPython notebook; just pointing out that embedding 
into random webpages has a lot of different requirements than designing 
your own web app, as I'm sure you could guess.

-Jason

[1] http://sagecell.sagemath.org/static/sagecell_embed.css -- this is 
generated from our normal 
http://sagecell.sagemath.org/static/sagecell.css by appending !important 
to each rule.  Since we scope things inside the .sagecell class (or 
prepend class names with sagecell_), and wrap everything in a .sagecell 
div, we aren't messing with things outside of the sage cell.


From gmbecker at ucdavis.edu  Fri Jul  5 15:28:55 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Fri, 5 Jul 2013 12:28:55 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
Message-ID: <CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>

 Brian,

Thank you for taking the time to watch my video and think about the ideas
I'm presenting. It is appreciated.

I am cognizant of the added complexity I am talking about. I disagree a bit
with the size/damage being attributed to it, but the main place we seem to
be on different pages is regarding what it buys us.

The alt cells I'm talking about are not a cool rendering/interaction trick
for linear notebooks. If they were I would absolutely agree that the
nesting is overkill and not worth its cost.

Linear/sequental notebooks and other dynamic document systems reproduce
computational *results*. The documents my advisors (Duncan Temple Lang and
Deborah Nolan) and I are working towards aim to describe and reproduce *the
research itself*.

I once asked a room full of quants to raise their hands if their standard
operating procedure was to read in the data, maybe clean it a bit, fit a
single model, write up the results and be done. Not only did no one raise
their hand, but the mere suggestion that that could be how it worked got a
substantial laugh from the audience. Even though this is not how the work
is done, however, it is the narrative encoded into a linear notebook.

Scientists and data analysts are already doing branching in their work, but
they don't have any good tools to record or describe it that way. So they
comment out large blocks of code in linear scripts, or they save old
attempts and alternate approaches in separate files, or they (hopefully
not) simply delete or overwrite old parameter configurations with new ones
when they decide the old one wasn't right. Not because they think these are
good ideas, but because its all they have available and they are
scientists/analysts.

Their job is to analyze data, extract insight and share their work in a
useful manner; our job is to conceive and implement the tools they need to
do that. They are good at their job, but we are struggling with ours.

In my opinion, the single most important feature on display in my video is
not that the alternative cells are rendered side-by-side, or that they can
be executed and the software knows what that means in terms of executing
their content; it is that the IPython notebook has become an authoring tool
which analysts can use to easily create documents which describe what they
actually did in a way that is reproducible, distributable, and most
importantly, useful to the analyst.

Suddenly they don't need to comment out or overwrite code when they take a
different approach. Suddenly they can distribute a document that actually
describes what they did, instead of just what they found. Suddenly referees
don't need to wonder or ask about whether an alternative analysis method
was investigated. Suddenly professors can show statistics students what
statisticians actually *do*, instead of how final results are generated.

You may read all this and think "that is great but way out of scope for
what we want to achieve with IPython notebook". That is your right. And if
the IPython team feels that way there is, of course, nothing I can do other
than what I have done: explore these ideas on my own. I just want to be
sure we're all talking about the same thing before that decision gets made.

Detailed responses below.


On Wed, Jul 3, 2013 at 6:59 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Gabriel,
>
> I watched your video and there are some nice ideas here.  We are not
> headed in this direction in terms of *implementation* but I think you
> will find that similar *capabilities* will show up in the notebook
> over time.  A few comments about the implementation aspects:
>
> First, the benefits of having a notebook be a linear sequence of cells
> are massive:
>
> * Simple, simple simple - this makes it very easy to reason about
> notebooks in code.  Nesting leads to complexity that is not worth the
> cost.
>

It does lead to complexity. Whether it is worth the cost depends on what we
gain. In my opinion I think we gain a massive amount, as I tried to
describe above.


> * You can get most of the benefits of nesting without the complexity.
> As Min mentioned, there is an implied hirrarchy in the heading cells.
> We plan on using that to allow group level actions - show/hide, run
> group, move, cut/copy/paste, etc.
>

With respect, I don't know that you will get the actual benefits I am
aiming for this way. Branches (and thus the code contained in them) in this
context are mutually exclusive.  A document which has branching that leads
back into a single computation, such as the example in my
video/screenshots, *no longer makes sense to think about or execute as a
sequential set of code blocks*.

It is not that we *can* view it as a branching structure and only execute
one branch at a time, but that we *must*. Again, people do this already via
deletion or commenting because all the code for all the branches cannot
live in the same block of code. I seek to give them a tool to do it that is
not damaging to the record or reproducibilty of what they did.

If you are familiar with R code and look carefully at the code for the data
cleaning branches in my example, you will see that if we put all of that
code into a sequential notebook, with heading cells to differentiate the
branches, and ran the code start to finish it would generate a plot not
created by *any* of the 18 paths through the structured document. In other
words, it generates the wrong plot regardless of what the "right" set of
choices at the branching points is in that execution context.


> * It is not difficult to think about building a proper diff tool for
> notebooks.  With nested cells this becomes horrific.
>

This does become much more difficult, but there are still things which can
be done. Archambault and others have done work on differencing graphs under
the assumption that one of them modified to create the other ( see
here<http://hal.inria.fr/docs/00/51/41/50/PDF/diffMapExperiment.pdf>and
here<http://www.cs.ubc.ca/nest/imager/tr/2009/Archambault_StructDiff_GI/structDiffPres.pdf>for
quickly retrieved examples)

Thus we can combine graph differencing with differencing of the actual
code, e.g. a combination of difference maps  and normal code diffing.

This would require unique identifiers for all notebook cells, but from
something said previously in this thread I gather that is likely to happen
anyway.


> * Hierarchy puts a significant cognitive load on users
>

Maybe. Again, this is something people are already implicitly doing. It is
possible that users would have trouble thinking about what they are doing
more explicitly in these terms, but its also entirely possible that this
will actually simplify their thinking process.

I admittedly think about these types of documents a lot, but I was
pleasantly surprised at how easy it was to construct and interact with the
example I made for the video once the UI hooks were in place. Constructing
such documents by hand, which I have done a bit, IS quite unpleasant, but
that is one of the issues I seek to address.

Nested branching and other more complex structures are more difficult to
think about, but users would only create those if they had a really good
reason to do so (ie if they were already thinking in those terms) and just
a couple levels of branching buy us an enormous amount of fidelity when
describing data analyses.



>
> Because of these things we don't have any plans on changing the
> notebook document format or notebook UI to allow nested cells.
>
>
As I said above, it is absolutely the right of the IPython core team to
make that call. I just want to make sure we are all talking about the same
costs and the same gains when the cost-benefit analysis to make that
decision is performed.

I intend to continue pursuing these concepts. I would be thrilled to
collaborate with the IPython team if you guys decide it is something you
are interested in, but I will understand and carry on myself if you do not.

Thanks for reading and for the discussion. Its great to talk to people with
differing opinions about this instead of sitting alone in my thesis-filled
bubble stewing :).

~G


Second, while it is tempting to generalize the notion of input to
> include widgety things, it is more appropriate to put these things in
> the output:
>
> * Our output architecture has the notion of multiple representations.
> This allows us to build rich widget as you have done, but to still
> provide static representations (png, jpg, latex).
> * Having the multiple representations of output allows us to build the
> rich widgets, but maintain a clear path for converting notebooks to
> static formats (pdf, html, word, powerpoint).
> * Insisting that input cells are pure code allows you to reason in a
> clear manner about how a notebook works = code runs and leads to
> output.  That reasoning can be applied in an automate manner by
> running notebooks in batch mode, or building a test system based on
> them.
> * Putting widgets in the input area forces you to do regular
> expression matching to replace those variables in the code.  This
> limits you to an extremely simple event model where the only possible
> event you can know about is substitute the regular expression and run
> all the code.  What if you want different UI controls in the browser
> to trigger different bits of code in the kernels when different fine
> grained events happen.  Making the UI controls live on the Python and
> JS side allows us to build this in a natural way.
>
> Th alt-cells you show bring up the issue of providence.  We have some
> very initial thoughts about that, but it is way out of scope for the
> project right now - we have a plates 10x overfull already.  We will
> get there though eventually.
>
> Thanks for sharing your ideas.
>
> PS - for a bit more background about the context of our saying "no" to
> this feature request, see this blog post:
>
> http://brianegranger.com/?p=249
>
> I also gave a talk about this at SciPy and will be posting my slides soon.
>
> Cheers,
>
> Brian
>
> On Wed, Jul 3, 2013 at 6:04 PM, Gabriel Becker <gmbecker at ucdavis.edu>
> wrote:
> > Matthias,
> >
> > Thanks for your detailed response.
> >
> >
> > On Wed, Jul 3, 2013 at 1:25 AM, Matthias BUSSONNIER
> > <bussonniermatthias at gmail.com> wrote:
> >>
> >> Gabriel,
> >>
> >> You screen shot are interesting,
> >> At some point I played with gridster[1]
> >>
> >> and was more or less able to get cell to rearranges, but didn't keep the
> >> code.
> >> You might be interested.
> >>
> >> Keep in mind that the notebook browser-interface we ship is only one
> >> possible
> >> frontend that can interpret ipynb files, nothing prevent you to write a
> >> different frontend that display the notebook in a different format.
> >>
> >> This added to the fact that each cell can support arbitrary metadata,
> you
> >> should be able to arrange preexisting in structure that work together.
> It
> >> might
> >> be a little difficult to do it right now as our javascript is not yet
> >> modular
> >> enough to be easily reused, but we are moving toward it.
> >
> >
> > Respectfully, rolling my own frontend for ipynb files given all the work
> the
> > IPython team has done on the excellent notebook browser interface would
> be
> > an enormous and extremely wasteful duplication of effort. I don't think
> its
> > the right way to pursue these features.
> >
> > Furthermore, if I were going to write an application offering the types
> of
> > features I am talking about from scratch, there wouldn't be any good
> reason
> > to base it on the unaltered ipynb format, as they don't easily support
> the
> > structure required by those features without the additional cell types I
> > implemented in my forked version.
> >
> >>
> >> Right now I thing storing the notebook as a directed graph is
> problematic
> >> in a
> >> few way,
> >
> >
> > I'm not talking about storing the notebook as an actual directed graph
> data
> > structure. There would be benefits to that but its not necessary and it
> > isn't want I did in my forked version.
> >
> > The ability to have nested cells (cells which contain other cells) gets
> us
> > everything we need structure wise, and is the basis of everything seen in
> > both the video (other than interactive code cell stuff) and screenshots I
> > posted. The ipynb file for the notebook pictured in the screenshot looks
> > exactly like a normal ipynb file except that in the json there are cell
> > declarations which have a cells field which contains the json
> descriptions
> > of the cells contained in that cell.
> >
> >
> >>
> >> the first being that it is incompatible with the fact that people want
> >> to be able to run notebook in a headless manner, which if you add
> explicit
> >> choice is not possible.
> >
> >
> > This isn't the case. The json saved versions of notebooks with branching
> > remember which version was most recently run. When an altset cell is
> > executed, it runs only the most recently run (or currently "selected",
> > though that means something else internally) branch. Thus by doing the
> naive
> > thing and looping through all top level cells and executing them, the
> > currently chosen path through the notebook can easily be run in a
> headless
> > environment and give the correct results.
> >
> >>
> >> This also contradict the fact that the notebook capture
> >> both the input and the output of the computation.
> >
> >
> > I don't really understand what you mean by this. In the JSON
> representation
> > of an executed code cell, the input field contains the code, but not any
> > values of variables used by the code, nor any indication of code which
> was
> > run before executing the code cell.
> >
> > Changing and rerunning an earlier code cell without re-executing the
> cell in
> > question can easily invalidate the output stored in the JSON, even
> without
> > the concept of branching or choice.
> >
> >
> >>
> >> As you showed there is
> >> actually 18 different combinations of data analysis, and they are not
> all
> >> stored in the notebook.
> >
> >
> > The notebook knows and records which choices were made. There are 18
> > different combinations of data analysis but only one was chosen by
> analyst
> > as generating the final/most recent result.
> >
> > In the case of "publishing" about an analysis the notebook stores the
> path
> > most chosen by the analyst, while retaining information about what else
> he
> > or she did during the decision process.
> >
> > In the case of instruction, imagine how much easier it would be to teach
> > data analysis if the students could actually see what data analysts do,
> > instead of simply the final method they choose in a particular analysis.
> >
> >
> >>
> >>
> >> I really thing this is an interesting project, and reusing only our
> >> metadata in
> >> the notebook, you should be able to  simulate it (store the dag in
> >> notebook
> >> level metadata, and cell id in cell metadata) then reconstruct the graph
> >> when
> >> needed. Keep in mind that at some point we might/will add cell id to the
> >> notebook.
> >>
> >> To sum up, I don't think the current JS client is in it's current state
> >> the
> >> place to implement such an idea. The Dag for cell order might be an idea
> >> for
> >> future notebook format but need to be well thought, and wait for cell
> IDs.
> >
> >
> > I apologize for not being clear. As I said in a response above, the
> directed
> > graph idea was intended to be conceptual for thinking about the
> documents,
> > not structural for actually storing them.
> >
> > What I actually did was simply allow cell nesting and change indexing so
> > that it is with respect to the parent/container (cell or notebook)
> instead
> > of always with respect to the notebook. This required some machinery
> changes
> > but not too many and it is an extension in the mathematical sense in that
> > indexing will behave identically to the old system for notebooks without
> any
> > nesting while now meaningfully functioning for notebooks with nesting.
> >
> > ~G
> >>
> >>
> >> --
> >> Matthias
> >>
> >>
> >>
> >> [1] http://gridster.net/
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >
> >
> >
> > --
> > Gabriel Becker
> > Graduate Student
> > Statistics Department
> > University of California, Davis
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130705/6881bad7/attachment.html>

From takowl at gmail.com  Fri Jul  5 17:09:15 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 5 Jul 2013 22:09:15 +0100
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
Message-ID: <CAOvn4qjAKWk7HNXY_U9OBqPu8d9b_6Cf1akwVqYuScY2t62o4Q@mail.gmail.com>

On 5 July 2013 20:28, Gabriel Becker <gmbecker at ucdavis.edu> wrote:

> Scientists and data analysts are already doing branching in their work,
> but they don't have any good tools to record or describe it that way. So
> they comment out large blocks of code in linear scripts, or they save old
> attempts and alternate approaches in separate files, or they (hopefully
> not) simply delete or overwrite old parameter configurations with new ones
> when they decide the old one wasn't right. Not because they think these are
> good ideas, but because its all they have available and they are
> scientists/analysts.


This is a fascinating idea, and I look forward to seeing it developed. I
can certainly think of cases where I could use it in my own work, although
I don't think I'd ever preserve every dead end I tried in alternative cells
- it would be far too much clutter.

That said, I don't think this is something we should be building into
IPython at the moment. As I see it, the notebook that we build is actually
quite a low-level tool, like a word processor for code. By which I mean
that we give users a blank canvas to create their own work, rather than a
consciously designed framework into which to slot their code. For instance,
while it's clearly sensible to show cells in execution order, we do nothing
to enforce that.

But I do think that we should try our best to support other people who want
to build higher-level tools, like your branching code system, on top of
IPython. I confess I haven't read through what everyone's said above, but I
hope we can find a good way for you to do that and have it work as well as
possible with the kernel, the notebook format, nbviewer, etc.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130705/053f4e85/attachment.html>

From sychan at lbl.gov  Fri Jul  5 19:12:43 2013
From: sychan at lbl.gov (Stephen Chan)
Date: Fri, 5 Jul 2013 16:12:43 -0700
Subject: [IPython-dev] Converting ipython notebook from tornado to
	gevent?
In-Reply-To: <CAHNn8BW3WtTepTvJhDwhjyHE+m4jGpq4D_sGLViMVZSmqYSbRQ@mail.gmail.com>
References: <CA+n9YfqACs+s=5GRmFded=9UBxgWnNWuUCRBgYKbc3O9y3nOvQ@mail.gmail.com>
	<D62C85F4-E359-4865-B44E-AFBBE4328C60@gmail.com>
	<CAHNn8BW3WtTepTvJhDwhjyHE+m4jGpq4D_sGLViMVZSmqYSbRQ@mail.gmail.com>
Message-ID: <CA+n9Yfrby9QfWV2Kw2kskY3JnuK9-rSm9YSDDYERYAufFY_ZBQ@mail.gmail.com>

On Thu, Jun 27, 2013 at 3:50 PM, MinRK <benjaminrk at gmail.com> wrote:

> I hope you reconsider this goal. A true wsgi-gevent notebook webapp would
> really mean a fork of the notebook, with all of the long-term burden and
> unpleasantness that entails. Getting the existing notebook to sit somewhere
> inside a webapp that happens to use a different toolkit is probably more
> sustainable.
>
>
>
   After some discussion based on the responses, we decided that converting
away from Tornado was not worth the effort for the reasons listed.
   Thanks for the feedback,

   Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130705/1bf63412/attachment.html>

From ellisonbg at gmail.com  Fri Jul  5 19:43:39 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 5 Jul 2013 16:43:39 -0700
Subject: [IPython-dev] JavaScript docs...
Message-ID: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>

Today, I have been working a lot on the dual mode editor stuff for the
notebook.  Obviously, this means spending lots of time in the
JavaScript code.  Part of our JavaScript development workflow is
broken.  Namely the convention we are using for our JavaScript
"docstring" is completely unworkable from a development standpoint:

* Managing comments in multiline /* */ style comments is horrific
* The visual clutter of the docstrings makes it very difficult to make
your way around the code
* It is painful enough that these docstrings easily fall out of sync

When we first went this route for our JavaScript docstrings, I thought
to myself "I don't like this, but I will get used to it, just like I
did with Python docstrings."  Unfortunately, the exact opposite has
happened - my frustration has grown.  In the spirit of GitHub, we need
to change this in order to "optimize for developer happiness."

Here is what I propose: I don't care one bit that we can generate
official API docs from our JavaScript code.  If someone is really
going to use our JS code, API docs won't go very far - they will need
to look at the code eventually.  Let's come up with a standard that is
easier to write and maintain for developers...any ideas?

Cheers,

Brian

--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From asmeurer at gmail.com  Fri Jul  5 19:58:26 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Fri, 5 Jul 2013 18:58:26 -0500
Subject: [IPython-dev] JavaScript docs...
In-Reply-To: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
References: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
Message-ID: <CAKgW=6LHF_DuyeEM8gNROK+0-rDk3_nHzuUUVEvr7OpBGriiZQ@mail.gmail.com>

On Fri, Jul 5, 2013 at 6:43 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Today, I have been working a lot on the dual mode editor stuff for the
> notebook.  Obviously, this means spending lots of time in the
> JavaScript code.  Part of our JavaScript development workflow is
> broken.  Namely the convention we are using for our JavaScript
> "docstring" is completely unworkable from a development standpoint:
>
> * Managing comments in multiline /* */ style comments is horrific
> * The visual clutter of the docstrings makes it very difficult to make
> your way around the code
> * It is painful enough that these docstrings easily fall out of sync
>

Maybe this isn't the answer you are looking for, but can't the first two
points be solved by a good text editor.  Emacs is smart enough to wrap
comments using a combination of auto-fill-mode and M-q. For the second
point, you just need a good code-folding tool.

To be fair, I don't use javascript, so I can't comment deeply on this.
 Actually, I just tested the Emacs javascript-mode, and the defaults seem
to have some rough edges.

I do agree that the @param nonsense gets old pretty fast.

Aaron Meurer


> When we first went this route for our JavaScript docstrings, I thought
> to myself "I don't like this, but I will get used to it, just like I
> did with Python docstrings."  Unfortunately, the exact opposite has
> happened - my frustration has grown.  In the spirit of GitHub, we need
> to change this in order to "optimize for developer happiness."
>
> Here is what I propose: I don't care one bit that we can generate
> official API docs from our JavaScript code.  If someone is really
> going to use our JS code, API docs won't go very far - they will need
> to look at the code eventually.  Let's come up with a standard that is
> easier to write and maintain for developers...any ideas?
>
> Cheers,
>
> Brian
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130705/0997cb0d/attachment.html>

From sychan at lbl.gov  Fri Jul  5 20:09:17 2013
From: sychan at lbl.gov (Stephen Chan)
Date: Fri, 5 Jul 2013 17:09:17 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
Message-ID: <CA+n9YfoKqvaRmwZFZvRUH4=B3FUggyU9xNK1Ac32KW0SLCWe7g@mail.gmail.com>

Hi Gabriel,
   First off, thanks for posting your work. We are working
adapting/extending the IPython notebook for use as a
teaching/publication tool and one of the most important use cases is
to demonstrate alternatives/branches in a notebook that shows how
different options produce different results - documenting the
decisions that went into the final "path" through a notebook.
   For the KBase project, we think the ability to show branches and
alternatives are an important feature that we need to implement, and
we would love to work with you further on this.

On Fri, Jul 5, 2013 at 12:28 PM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
> I am cognizant of the added complexity I am talking about. I disagree a bit with the
> size/damage being attributed to it, but the main place we seem to be on different pages is regarding what it buys us.
>

   There is a broader question here: What constitutes "too complex"?
   If you using a notebook as a teaching tool, or as a way to document
your analysis, being unable to show alternatives would be considered
"too simple".
   I think the complexity here not conceptual (almost anyone doing any
kind of analysis should understand evaluating alternatives), but
rather the problem is in the user interface, and the problem of
providing an interface that works well for novice users and "read
only" users of a notebook, versus advanced users and authors who want
to express fairly reasonable ideas in their notebooks.

   That being said, I agree with everything you say in the following section:

> The alt cells I'm talking about are not a cool rendering/interaction trick for linear notebooks. If they were I would absolutely agree that the nesting is overkill and not worth its cost.
>
> Linear/sequental notebooks and other dynamic document systems reproduce computational results. The documents my advisors (Duncan Temple Lang and Deborah Nolan) and I are working towards aim to describe and reproduce the research itself.
>
> I once asked a room full of quants to raise their hands if their standard operating procedure was to read in the data, maybe clean it a bit, fit a single model, write up the results and be done. Not only did no one raise their hand, but the mere suggestion that that could be how it worked got a substantial laugh from the audience. Even though this is not how the work is done, however, it is the narrative encoded into a linear notebook.
>
> Scientists and data analysts are already doing branching in their work, but they don't have any good tools to record or describe it that way. So they comment out large blocks of code in linear scripts, or they save old attempts and alternate approaches in separate files, or they (hopefully not) simply delete or overwrite old parameter configurations with new ones when they decide the old one wasn't right. Not because they think these are good ideas, but because its all they have available and they are scientists/analysts.
>
> Their job is to analyze data, extract insight and share their work in a useful manner; our job is to conceive and implement the tools they need to do that. They are good at their job, but we are struggling with ours.
>
> In my opinion, the single most important feature on display in my video is not that the alternative cells are rendered side-by-side, or that they can be executed and the software knows what that means in terms of executing their content; it is that the IPython notebook has become an authoring tool which analysts can use to easily create documents which describe what they actually did in a way that is reproducible, distributable, and most importantly, useful to the analyst.
>
> Suddenly they don't need to comment out or overwrite code when they take a different approach. Suddenly they can distribute a document that actually describes what they did, instead of just what they found. Suddenly referees don't need to wonder or ask about whether an alternative analysis method was investigated. Suddenly professors can show statistics students what statisticians actually do, instead of how final results are generated.
>
> You may read all this and think "that is great but way out of scope for what we want to achieve with IPython notebook". That is your right. And if the IPython team feels that way there is, of course, nothing I can do other than what I have done: explore these ideas on my own. I just want to be sure we're all talking about the same thing before that decision gets made.
>

   What I would add is that as more people start using the IPython
notebook, I am fairly certain that others will want a way to represent
alternative options for executing cells and it would be good idea to
have an underlying architecture that can support this, and then think
about the UI issues going forward.

   Steve


From bussonniermatthias at gmail.com  Sat Jul  6 04:53:28 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 6 Jul 2013 10:53:28 +0200
Subject: [IPython-dev] JavaScript docs...
In-Reply-To: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
References: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
Message-ID: <638E4EC2-5D0F-4499-89C9-B770EDAE77C0@gmail.com>

Brian, 

You are probably writing more JS than I am, so whatever will suit you
I'll go with that, none the less a few comment inline

Le 6 juil. 2013 ? 01:43, Brian Granger a ?crit :

> Today, I have been working a lot on the dual mode editor stuff for the
> notebook.  Obviously, this means spending lots of time in the
> JavaScript code.  Part of our JavaScript development workflow is
> broken.  Namely the convention we are using for our JavaScript
> "docstring" is completely unworkable from a development standpoint:
> 
> * Managing comments in multiline /* */ style comments is horrific

As Aaron wrote, this should be the problem in your editor, I personally have
no problem with it, except maybe for re hard-wrapping long line that is sometime
confused. 

> * The visual clutter of the docstrings makes it very difficult to make
> your way around the code
> * It is painful enough that these docstrings easily fall out of sync

For those 2 I don't see where this differ from python. The format of the doctoring
is not particularly big, I personally have a light gray that is not disturbing. 
It is true that when you have loads of 1 line methods it take space.


> When we first went this route for our JavaScript docstrings, I thought
> to myself "I don't like this, but I will get used to it, just like I
> did with Python docstrings."  Unfortunately, the exact opposite has
> happened - my frustration has grown.  In the spirit of GitHub, we need
> to change this in order to "optimize for developer happiness."

I really don't want people to be angry at writing code, if it make you angry, 
we shoudl fix it.


> 
> Here is what I propose: I don't care one bit that we can generate
> official API docs from our JavaScript code.  If someone is really
> going to use our JS code, API docs won't go very far - they will need
> to look at the code eventually.

This is the main point where I disagree. I often use the API doc especially on user-exposed code
like cell toolbar, where it is really difficult to understand what the function does/how to use it without the docstring
especially when one need to pass a callback. and the ability to put markdown in doctoring that will be highlighted 
in doc is nice.

I agree that cell doc is almost useless : 
http://elacave.lmdb.eu/~carreau/yui/classes/IPython.Cell.html
But you can't say that the cell toolbar one is not much more friendlier than the code.
http://elacave.lmdb.eu/~carreau/yui/classes/IPython.CellToolbar.html


>  Let's come up with a standard that is
> easier to write and maintain for developers...any ideas?

would changing the comment delimiter help you like // ?
it might be possible to have comment like 

/**
//
// docstring.
//
//
*/ 

would this be better ?

we could limit docstrings to some specific functions that are not easy to use, or to figure out what they do.
This would diminish the amount of clutter you seem to feel. 

-- 
Matthias





From bussonniermatthias at gmail.com  Sat Jul  6 08:05:14 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 6 Jul 2013 14:05:14 +0200
Subject: [IPython-dev] Some new cell types for describing data analyses
	in IPy. Notebook
In-Reply-To: <CA+n9YfoKqvaRmwZFZvRUH4=B3FUggyU9xNK1Ac32KW0SLCWe7g@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CA+n9YfoKqvaRmwZFZvRUH4=B3FUggyU9xNK1Ac32KW0SLCWe7g@mail.gmail.com>
Message-ID: <E1E50173-9DFF-40EE-9A77-90315A4C6115@gmail.com>

Dear all, 


It is always a pleasure to see people making proof of concept of what
IPython can do, some have the chance to have long response, other get
through the cracks, it is often a question of chance, and I think I'm not
the only one that wished to have more time to get into theses discussion
more. 

Beeing also an academic, I do understand the frustration and the wish to have a
non-linear editing tool, I haven't been an IPython contributor for as long as
many people on the list, but I think IPython decided to take time to implement
things the correct way, and if this have to be included it should be fully
thought. 

It should be thought both in the development process, but also from a user
perspective, indeed most of the person that follow and post to this list are
advanced users which are able to manipulate advance concepts. In the other hand,
majority of people using/viewing the notebook have difficulties with the fact
that cell can be executed out of order, and have issues to distinguished
frontend from kernel and so on. This of course does not prevent from making a
non-linear tool that advanced user could use now, but this will be problematic
if we don't have the right abstraction for user to progressively migrate to it. 

My personal though on Gabe demo is that the POC is great, but has not been
though enough. The idea of DAG for the structure is really interesting and I
believe that it should be pushed further. IMHO alt-cell will only allow to
represent a subset of what could be done if the notebook was a real DAG. Like
the fact that at some point you might want to do some transform on the data, or
not. and an alt-cell with `do nothing` in one step is ugly. From the UI point
of view, I've been traumatized by labView, and I hope we'll not converge toward
something like that.

From a data structure point of view, I also strongly believe that nesting cell
is the wrong approach.  We already had a discussion about on-disk structure vs
in-memory structure and cell-id, and we might think of re discussing that at
some point. One of the key point is that we want ipynb format to be as much as
possible fixable by hand if needed. My though of that would be to store the
`cells` and `cell-order` separately as respectively a dict `cell-id:cell-data`
and a `list of cell-id` it would then be trivial to change the list-of-cell-id
to a DAG, ... or to share cell across notebooks in a more complex storage
format than file (warning, be carefull with this one, it's just an idea, but
dangerous)

To finish, I'll join with Thomas to say that this is a fascinating idea, but
that I don't think it is the right time to include. It took many prototype of
the notebook to be at current state, and we are already removing features that
we included to early by thinking that we would need it in the future. We'll do
our best to help you, and this will give us insight of what might be needed for
future versions. 

If you can of course start an IPython Enhancement Proposal[1] that describe the needs,
goal, drawback, and list different implementation proposal and/or changes proposed to
the notebook format.

Looking forward to what will come out of theses discussions. 
-- 
Matthias


[1] https://github.com/ipython/ipython/wiki/IPEPs:-IPython-Enhancement-Proposals

  





From fperez.net at gmail.com  Sun Jul  7 02:38:36 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 6 Jul 2013 23:38:36 -0700
Subject: [IPython-dev] JavaScript docs...
In-Reply-To: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
References: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
Message-ID: <CAHAreOozifGQk78dm+Lyf=2S_tft-Ycbs_sV+5Q8Jp=5D9rVGg@mail.gmail.com>

Just a couple of quick questions:

- is it possible to avoid all the * in the comments for the doc
generation machinery? Technically JS only needs the matching /* */
pair, so unless the doc tools need all the extra * on every line,
those could be skipped.

- it it OK to put the 'docstring' after the function declarations,
like in python?

If both of these were possible, we could have these JS 'docstrings'
looking like:

    var Notebook = function (selector, options) {
        /*
         A notebook contains and manages cells.

         @class Notebook

         @constructor

         @param {String} selector A jQuery selector for the notebook's
DOM element

         @param {Object} [options] A config object
        */
        var options = options || {};

instead of

    /**
     * A notebook contains and manages cells.
     *
     * @class Notebook
     * @constructor
     * @param {String} selector A jQuery selector for the notebook's DOM element
     * @param {Object} [options] A config object
     */
    var Notebook = function (selector, options) {

That seems quite a bit more python-like and readable to me...

Cheers,

f

On Fri, Jul 5, 2013 at 4:43 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Today, I have been working a lot on the dual mode editor stuff for the
> notebook.  Obviously, this means spending lots of time in the
> JavaScript code.  Part of our JavaScript development workflow is
> broken.  Namely the convention we are using for our JavaScript
> "docstring" is completely unworkable from a development standpoint:
>
> * Managing comments in multiline /* */ style comments is horrific
> * The visual clutter of the docstrings makes it very difficult to make
> your way around the code
> * It is painful enough that these docstrings easily fall out of sync
>
> When we first went this route for our JavaScript docstrings, I thought
> to myself "I don't like this, but I will get used to it, just like I
> did with Python docstrings."  Unfortunately, the exact opposite has
> happened - my frustration has grown.  In the spirit of GitHub, we need
> to change this in order to "optimize for developer happiness."
>
> Here is what I propose: I don't care one bit that we can generate
> official API docs from our JavaScript code.  If someone is really
> going to use our JS code, API docs won't go very far - they will need
> to look at the code eventually.  Let's come up with a standard that is
> easier to write and maintain for developers...any ideas?
>
> Cheers,
>
> Brian
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From bussonniermatthias at gmail.com  Sun Jul  7 04:28:58 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 7 Jul 2013 10:28:58 +0200
Subject: [IPython-dev] JavaScript docs...
In-Reply-To: <CAHAreOozifGQk78dm+Lyf=2S_tft-Ycbs_sV+5Q8Jp=5D9rVGg@mail.gmail.com>
References: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
	<CAHAreOozifGQk78dm+Lyf=2S_tft-Ycbs_sV+5Q8Jp=5D9rVGg@mail.gmail.com>
Message-ID: <504E5503-4577-4A96-9720-38FFDA3CB223@gmail.com>


Le 7 juil. 2013 ? 08:38, Fernando Perez a ?crit :

> Just a couple of quick questions:
> 
> - is it possible to avoid all the * in the comments for the doc
> generation machinery? Technically JS only needs the matching /* */
> pair, so unless the doc tools need all the extra * on every line,
> those could be skipped.

Apparently Yes it can be skipped but it still need the double asterisk on the first line
and only spaces on subsequent line would be accepted, not stuff like

/**
//
//
//
//*/

> 
> - it it OK to put the 'docstring' after the function declarations,
> like in python?

It works, but the doc will link the declaration of the function on the line following the docstring.

> That seems quite a bit more python-like and readable to me...

If you decide to go this way, so be it, but JS is not Python, and I don't think
we should try to match Python on this one.

It will still be frustrating as our editors will still try to fill the comment with leading * when writing. 
we will not get use to docs of other js project, and we will drive js-contributor crazy in not following 
language convention.

-- 
Matthias

From carl.input at gmail.com  Sun Jul  7 11:30:20 2013
From: carl.input at gmail.com (Carl Smith)
Date: Sun, 7 Jul 2013 16:30:20 +0100
Subject: [IPython-dev] Internet Freedom
Message-ID: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>

Hi all

The Internet Defence League ( http://internetdefenseleague.org ) is
signing up members to help fight for Internet freedom. Mozilla and PHP
have joined, amongst many others. This post isn't really aiming to get
the IPython community involved with the IDL, nor is it aiming to start
a debate on the issue of Internet freedom. Instead I was hoping to
explore whether IPython could and should be involved in projects of
this nature at all.

I have my own views, but would like to just put the question to the
community without bundling an opinion.

What do you think?


Carl


From ellisonbg at gmail.com  Sun Jul  7 11:55:17 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 7 Jul 2013 08:55:17 -0700
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
Message-ID: <4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>

What type of involvement are you thinking? 

Sent from my iPhone

On Jul 7, 2013, at 8:30 AM, Carl Smith <carl.input at gmail.com> wrote:

> Hi all
> 
> The Internet Defence League ( http://internetdefenseleague.org ) is
> signing up members to help fight for Internet freedom. Mozilla and PHP
> have joined, amongst many others. This post isn't really aiming to get
> the IPython community involved with the IDL, nor is it aiming to start
> a debate on the issue of Internet freedom. Instead I was hoping to
> explore whether IPython could and should be involved in projects of
> this nature at all.
> 
> I have my own views, but would like to just put the question to the
> community without bundling an opinion.
> 
> What do you think?
> 
> 
> Carl
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From sirinath at sakrio.com  Sun Jul  7 13:43:12 2013
From: sirinath at sakrio.com (Suminda Dharmasena)
Date: Sun, 7 Jul 2013 23:13:12 +0530
Subject: [IPython-dev] Python Spreadsheet & NoSQL & Simple DB Support
Message-ID: <CAAfFNYF2oYGiUNK59fr1H1OMTKUFwH=VF1dwFLaCNMwu=OUbvg@mail.gmail.com>

Hi,

Is it possible to support embed data documents like the data in embed in a
Spreadsheet or DB like manner.

Suminda
--
Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.) Lond.,
P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*: 6G ? 1st
Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*
: +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *Web
*: http://www.sakrio.com ?

This email is subjected to the email Terms of Use and Disclaimer:
http://www.sakrio.com/email-legal. Please read this first.
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130707/591ab091/attachment.html>

From sirinath at sakrio.com  Sun Jul  7 13:46:13 2013
From: sirinath at sakrio.com (Suminda Dharmasena)
Date: Sun, 7 Jul 2013 23:16:13 +0530
Subject: [IPython-dev] Python Spreadsheet & NoSQL & Simple DB Support
In-Reply-To: <CAAfFNYF2oYGiUNK59fr1H1OMTKUFwH=VF1dwFLaCNMwu=OUbvg@mail.gmail.com>
References: <CAAfFNYF2oYGiUNK59fr1H1OMTKUFwH=VF1dwFLaCNMwu=OUbvg@mail.gmail.com>
Message-ID: <CAAfFNYEW98gC4F0k6m-_tdZGu+-oPg2tUV-XQGsPSgCt_PmZvQ@mail.gmail.com>

Hi,

This is regarding IPython Notebooks.

Ability to self sustain the data, acquire new data, etc. from within the
note book itself.

Suminda
--
Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.) Lond.,
P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*: 6G ? 1st
Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*
: +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *Web
*: http://www.sakrio.com ?

This email is subjected to the email Terms of Use and Disclaimer:
http://www.sakrio.com/email-legal. Please read this first.
--


On 7 July 2013 23:13, Suminda Dharmasena <sirinath at sakrio.com> wrote:

> Hi,
>
> Is it possible to support embed data documents like the data in embed in a
> Spreadsheet or DB like manner.
>
> Suminda
> --
> Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.)
> Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*:
> 6G ? 1st Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*
> : +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *
> Web*: http://www.sakrio.com ?
>
> This email is subjected to the email Terms of Use and Disclaimer:
> http://www.sakrio.com/email-legal. Please read this first.
> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130707/d2e383cc/attachment.html>

From benjaminrk at gmail.com  Sun Jul  7 13:47:54 2013
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 7 Jul 2013 10:47:54 -0700
Subject: [IPython-dev] Python Spreadsheet & NoSQL & Simple DB Support
In-Reply-To: <CAAfFNYEW98gC4F0k6m-_tdZGu+-oPg2tUV-XQGsPSgCt_PmZvQ@mail.gmail.com>
References: <CAAfFNYF2oYGiUNK59fr1H1OMTKUFwH=VF1dwFLaCNMwu=OUbvg@mail.gmail.com>
	<CAAfFNYEW98gC4F0k6m-_tdZGu+-oPg2tUV-XQGsPSgCt_PmZvQ@mail.gmail.com>
Message-ID: <CAHNn8BWPY73Jrphs8aL3cEKr3fhv-vqNA28J=uhN9zECX=u7Rg@mail.gmail.com>

The model we generally have in mind is that the sharing unit is a git (or
other vcs) repo, which contains notebooks and data.  This means that data
files are not part of the notebook document itself.


On Sun, Jul 7, 2013 at 10:46 AM, Suminda Dharmasena <sirinath at sakrio.com>wrote:

> Hi,
>
> This is regarding IPython Notebooks.
>
> Ability to self sustain the data, acquire new data, etc. from within the
> note book itself.
>
> Suminda
> --
> Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.)
> Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*:
> 6G ? 1st Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*:
> +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *Web
> *: http://www.sakrio.com ?
>
> This email is subjected to the email Terms of Use and Disclaimer:
> http://www.sakrio.com/email-legal. Please read this first.
> --
>
>
> On 7 July 2013 23:13, Suminda Dharmasena <sirinath at sakrio.com> wrote:
>
>> Hi,
>>
>> Is it possible to support embed data documents like the data in embed in
>> a Spreadsheet or DB like manner.
>>
>> Suminda
>> --
>> Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.)
>> Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*:
>> 6G ? 1st Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*:
>> +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *
>> Web*: http://www.sakrio.com ?
>>
>> This email is subjected to the email Terms of Use and Disclaimer:
>> http://www.sakrio.com/email-legal. Please read this first.
>> --
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130707/2787d181/attachment.html>

From carl.input at gmail.com  Sun Jul  7 14:12:09 2013
From: carl.input at gmail.com (Carl Smith)
Date: Sun, 7 Jul 2013 19:12:09 +0100
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
	<4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
Message-ID: <CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>

The main question, I guess, is whether IPython should explicitly
support causes at all. Normally, an org will focus on it's own
problems, regardless of its members views on non-related issues. That
said, Mozilla and PHP have joined the IDL. Of course, Moz does have an
interest in this issue, but PHP doesn't, at least directly. I just
felt a little disappointed that it was PHP and not Python that I saw
on that list, and that left me confused; I've always thought it's
sensible to have different orgs for different problems, but
nonetheless felt the PSF should get involved.

This issue is not especially relevant to IPython in particular; I
approached this community as it's one that I feel close to, it's very
diverse and it has a long-standing tradition of contributing to free
software. I should have perhaps made it clearer that this isn't
IPython Dev stuff; it's just a topic this community should perhaps
discuss. There's no urgency, but, in the long run, IPython will
probably need to address the industry's political issues on some
level, even if the decision is to simply leave it to specialist
organisations.

Cheers

Carl


From asmeurer at gmail.com  Sun Jul  7 14:57:30 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 7 Jul 2013 13:57:30 -0500
Subject: [IPython-dev] Python Spreadsheet & NoSQL & Simple DB Support
In-Reply-To: <CAHNn8BWPY73Jrphs8aL3cEKr3fhv-vqNA28J=uhN9zECX=u7Rg@mail.gmail.com>
References: <CAAfFNYF2oYGiUNK59fr1H1OMTKUFwH=VF1dwFLaCNMwu=OUbvg@mail.gmail.com>
	<CAAfFNYEW98gC4F0k6m-_tdZGu+-oPg2tUV-XQGsPSgCt_PmZvQ@mail.gmail.com>
	<CAHNn8BWPY73Jrphs8aL3cEKr3fhv-vqNA28J=uhN9zECX=u7Rg@mail.gmail.com>
Message-ID: <CAKgW=6LrPcj7Ocn1SYe2Te=MCkW31atjLpv3Y3=ibqtJ_tQuxg@mail.gmail.com>

You might also look at Wakari bundles.

Aaron Meurer


On Sun, Jul 7, 2013 at 12:47 PM, MinRK <benjaminrk at gmail.com> wrote:

> The model we generally have in mind is that the sharing unit is a git (or
> other vcs) repo, which contains notebooks and data.  This means that data
> files are not part of the notebook document itself.
>
>
> On Sun, Jul 7, 2013 at 10:46 AM, Suminda Dharmasena <sirinath at sakrio.com>wrote:
>
>> Hi,
>>
>> This is regarding IPython Notebooks.
>>
>> Ability to self sustain the data, acquire new data, etc. from within the
>> note book itself.
>>
>> Suminda
>> --
>> Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.)
>> Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*:
>> 6G ? 1st Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*:
>> +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *
>> Web*: http://www.sakrio.com ?
>>
>> This email is subjected to the email Terms of Use and Disclaimer:
>> http://www.sakrio.com/email-legal. Please read this first.
>> --
>>
>>
>> On 7 July 2013 23:13, Suminda Dharmasena <sirinath at sakrio.com> wrote:
>>
>>> Hi,
>>>
>>> Is it possible to support embed data documents like the data in embed in
>>> a Spreadsheet or DB like manner.
>>>
>>> Suminda
>>> --
>>> Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.)
>>> Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*:
>>> 6G ? 1st Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*:
>>> +94-(0)711007945 ? *Tele*: +94-(0)11-5 864614 / 5 875614 / 2 825908 ? *
>>> Web*: http://www.sakrio.com ?
>>>
>>> This email is subjected to the email Terms of Use and Disclaimer:
>>> http://www.sakrio.com/email-legal. Please read this first.
>>> --
>>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130707/3940fada/attachment.html>

From jrjohansson at gmail.com  Mon Jul  8 00:08:37 2013
From: jrjohansson at gmail.com (jrjohansson at gmail.com)
Date: Mon, 8 Jul 2013 13:08:37 +0900
Subject: [IPython-dev] IPython magic that display version information for
	dependency modules
Message-ID: <CAA2hLfFt37JYf+FFmsackFAyJODQzb+W=SD=V5xmV4qFEzjm-A@mail.gmail.com>

Hi

To promote reproducibility of shared IPython notebooks I often find it
useful to include some version information for modules at the end of the
notebook (with lots of dependencies a lot of things can go wrong...).

I've create a simple IPython magic command extension that displays such
information in an HTML table (suitable for use in the notebook). See

http://nbviewer.ipython.org/urls/raw.github.com/jrjohansson/version_information/master/example.ipynb

Is there a built-in functionality of this kind in IPython that I'm not
aware of, or some other better way of doing this? If not I think it would
be a nice to have something like this included directly in IPython.

Best regards
Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130708/02fe82fb/attachment.html>

From bussonniermatthias at gmail.com  Mon Jul  8 02:44:57 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 8 Jul 2013 08:44:57 +0200
Subject: [IPython-dev] IPython magic that display version information
	for dependency modules
In-Reply-To: <CAA2hLfFt37JYf+FFmsackFAyJODQzb+W=SD=V5xmV4qFEzjm-A@mail.gmail.com>
References: <CAA2hLfFt37JYf+FFmsackFAyJODQzb+W=SD=V5xmV4qFEzjm-A@mail.gmail.com>
Message-ID: <E2A45536-8DD0-4CFC-BC25-BF0307554400@gmail.com>

Hi Rob, 

Le 8 juil. 2013 ? 06:08, jrjohansson at gmail.com a ?crit :

> Hi
> 
> To promote reproducibility of shared IPython notebooks I often find it useful to include some version information for modules at the end of the notebook (with lots of dependencies a lot of things can go wrong...).
> 
> I've create a simple IPython magic command extension that displays such information in an HTML table (suitable for use in the notebook). See
> 
> http://nbviewer.ipython.org/urls/raw.github.com/jrjohansson/version_information/master/example.ipynb
> 
> Is there a built-in functionality of this kind in IPython that I'm not aware of, or some other better way of doing this? If not I think it would be a nice to have something like this included directly in IPython.

No there is no tool like that in IPython, we believe that this in particular is not our job to do, but that we could provide structure
to do so. We  let the community free to decide and polish the idea first.

Your example look nice, but I will suggest some things. That are, by the way, some 
general remarks that are useful for custom objects/extension of anyone.


Don't return HTML, except in a _repr_html_.

This make the extension useless in terminal, not sure it it's works in qtconsole. 
It's a pain to parse programatically. It won't be converted to TeX and other format
by nbconvert....etc

I would propose to define many _repr_xxx_ by starting by the json one which is the easiest
to be machine-consumable and from this you can define _repr_html_ _repr_pretty_ , maybe a_repr_latex_
... By doing this, your extension will become compatible with all clients, and the notebook-document 
**will store all the repr** meaning that they will be available to nbconvert, or any script that consume a notebook.

Otherwise look nice, please add to the wiki on the list of extensions. 
-- 
Matthias



From jrjohansson at gmail.com  Mon Jul  8 05:13:12 2013
From: jrjohansson at gmail.com (jrjohansson at gmail.com)
Date: Mon, 8 Jul 2013 18:13:12 +0900
Subject: [IPython-dev] IPython magic that display version information
 for dependency modules
In-Reply-To: <E2A45536-8DD0-4CFC-BC25-BF0307554400@gmail.com>
References: <CAA2hLfFt37JYf+FFmsackFAyJODQzb+W=SD=V5xmV4qFEzjm-A@mail.gmail.com>
	<E2A45536-8DD0-4CFC-BC25-BF0307554400@gmail.com>
Message-ID: <CAA2hLfGUSkRPiXsuPEHh7VsU+HiLOR=pfNaPoHXVM1PCEksLoQ@mail.gmail.com>

Hi Matthias

I've updated the implementation according to your suggestions. Thanks for
the feedback.
Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130708/68a278fc/attachment.html>

From bussonniermatthias at gmail.com  Tue Jul  9 17:12:09 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 9 Jul 2013 23:12:09 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
Message-ID: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>

Hi list, 

As you all know, soon should be the release of IPython 1.0 which is theoretically due in one week.
To celebrate this, I suggest user and devs near Paris to gather for a drink this weeks, or next one.

For the dates, If people are available this week-end(13,14), monday(15), or thurstday(18)
please select below

http://www.doodle.com/8ii6v2nzacaxc5xg

Yes I know, I choose only date where I'm free, I can change if necessary
but I'm the one who proposed it would be a shame for me not to be available.

Depending on wether you prefer wine or beer, I propose the following places.

Beer,
L'academie de la bi?re  http://goo.gl/maps/NmMd4
La gueuse :  http://goo.gl/maps/a0KLL

Wine
If we are not too many, and depending on the date, I know a quiet wine bar, not too expensive also:
Les grains nobles: http://goo.gl/maps/K4aJb

I also hope this will inspire the same things around the world.

-- 
Matthias
P.S.: Unlike github's drinkups you have to pay your own drink.

From ellisonbg at gmail.com  Tue Jul  9 19:59:43 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 16:59:43 -0700
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
	<4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
	<CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>
Message-ID: <CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>

I personally like the goals of the IDL, but I think it is important
for projects like IPython to remain focused.  Because of this my vote
would be for this involvement to happen at the personal level.  I
would like IPython itself to remain a-political - with no opinions on
outside issues.  I feel this is especially important given the fact
that IPython is not a legal entity - it is only an informal group of
people who share collective copyright.  It is very difficult to
"speak" on behalf of our entire community because of this - both on
the practical and conceptual levels.  Groups like Mozilla and PHP or
even the PSF are legal entities so there is a clear path for
determining what exactly this type of support means (a vote of the
board).

Cheers,

Brian

On Sun, Jul 7, 2013 at 11:12 AM, Carl Smith <carl.input at gmail.com> wrote:
> The main question, I guess, is whether IPython should explicitly
> support causes at all. Normally, an org will focus on it's own
> problems, regardless of its members views on non-related issues. That
> said, Mozilla and PHP have joined the IDL. Of course, Moz does have an
> interest in this issue, but PHP doesn't, at least directly. I just
> felt a little disappointed that it was PHP and not Python that I saw
> on that list, and that left me confused; I've always thought it's
> sensible to have different orgs for different problems, but
> nonetheless felt the PSF should get involved.
>
> This issue is not especially relevant to IPython in particular; I
> approached this community as it's one that I feel close to, it's very
> diverse and it has a long-standing tradition of contributing to free
> software. I should have perhaps made it clearer that this isn't
> IPython Dev stuff; it's just a topic this community should perhaps
> discuss. There's no urgency, but, in the long run, IPython will
> probably need to address the industry's political issues on some
> level, even if the decision is to simply leave it to specialist
> organisations.
>
> Cheers
>
> Carl
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Jul  9 20:01:17 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 17:01:17 -0700
Subject: [IPython-dev] JavaScript docs...
In-Reply-To: <504E5503-4577-4A96-9720-38FFDA3CB223@gmail.com>
References: <CAH4pYpQ-LfwZ0rp7fWp9AQHWJmSZgTS1_qPt2kPsXa+Yryw32A@mail.gmail.com>
	<CAHAreOozifGQk78dm+Lyf=2S_tft-Ycbs_sV+5Q8Jp=5D9rVGg@mail.gmail.com>
	<504E5503-4577-4A96-9720-38FFDA3CB223@gmail.com>
Message-ID: <CAH4pYpTfvyV6AgYJ+hUX8h=ywZq_8BNW_gipsizgaEMovLYSrg@mail.gmail.com>

I have added a note to the dev meeting wiki page for us to discuss this.

On Sun, Jul 7, 2013 at 1:28 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 7 juil. 2013 ? 08:38, Fernando Perez a ?crit :
>
>> Just a couple of quick questions:
>>
>> - is it possible to avoid all the * in the comments for the doc
>> generation machinery? Technically JS only needs the matching /* */
>> pair, so unless the doc tools need all the extra * on every line,
>> those could be skipped.
>
> Apparently Yes it can be skipped but it still need the double asterisk on the first line
> and only spaces on subsequent line would be accepted, not stuff like
>
> /**
> //
> //
> //
> //*/
>
>>
>> - it it OK to put the 'docstring' after the function declarations,
>> like in python?
>
> It works, but the doc will link the declaration of the function on the line following the docstring.
>
>> That seems quite a bit more python-like and readable to me...
>
> If you decide to go this way, so be it, but JS is not Python, and I don't think
> we should try to match Python on this one.
>
> It will still be frustrating as our editors will still try to fill the comment with leading * when writing.
> we will not get use to docs of other js project, and we will drive js-contributor crazy in not following
> language convention.
>
> --
> Matthias
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Jul  9 20:07:47 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 17:07:47 -0700
Subject: [IPython-dev] Wiki chaos...
Message-ID: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>

Hi,

A while back I spend a couple of days organizing the IPython GitHub
wiki.  I know this will probably seem anal, but I think it is
important that we use a logical structure for each of our pages and
the wiki as a whole.  To induce this structure on the flat GitHub wiki
namespace, we are naming our wiki pages with a prefix such as "Dev :"
or "Cookbook: " or "Install: ".  This structure is also present in our
sub-indices - we have a separate index page for each of these
categories.  When someone created the wiki page for the lab meetings,
it was created at the top level, rather than under the "Dev: "
category and put in the top level index rather than the "Dev: " index.
 I could fix this myself in a few minutes, but I don't want to become
the "police" for our wiki organization.  Instead, I would like
everyone to help keep the wiki organized.  Can someone clean this up?
If we don't want to use this structure, but instead keep things
organized another way, I am completely open to that.  I just want the
wiki organization to decay into chaos like it did before...

Cheers,

Brian

--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From scopatz at gmail.com  Tue Jul  9 20:19:52 2013
From: scopatz at gmail.com (Anthony Scopatz)
Date: Tue, 9 Jul 2013 19:19:52 -0500
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
	<4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
	<CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>
	<CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>
Message-ID: <CAPk-6T61UBya=HScHFYXGvETsAnOk0QB5nNfDX+YyzMpTU6ZqA@mail.gmail.com>

On Tue, Jul 9, 2013 at 6:59 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> I personally like the goals of the IDL, but I think it is important
> for projects like IPython to remain focused.  Because of this my vote
> would be for this involvement to happen at the personal level.  I
> would like IPython itself to remain a-political - with no opinions on
> outside issues.  I feel this is especially important given the fact
> that IPython is not a legal entity - it is only an informal group of
> people who share collective copyright.  It is very difficult to
> "speak" on behalf of our entire community because of this - both on
> the practical and conceptual levels.  Groups like Mozilla and PHP or
> even the PSF are legal entities so there is a clear path for
> determining what exactly this type of support means (a vote of the
> board).
>

To hopefully not derail this thread too much and to speak to Brian's point
about being a legal entity, this is more the sort of thing that would be
appropriate for NumFOCUS to consider.  I suggest you join the NumFOCUS
mailing list and bring it up there <numfocus at googlegroups.com>.

Be Well
Anthony

1. http://numfocus.org/


>
> Cheers,
>
> Brian
>
> On Sun, Jul 7, 2013 at 11:12 AM, Carl Smith <carl.input at gmail.com> wrote:
> > The main question, I guess, is whether IPython should explicitly
> > support causes at all. Normally, an org will focus on it's own
> > problems, regardless of its members views on non-related issues. That
> > said, Mozilla and PHP have joined the IDL. Of course, Moz does have an
> > interest in this issue, but PHP doesn't, at least directly. I just
> > felt a little disappointed that it was PHP and not Python that I saw
> > on that list, and that left me confused; I've always thought it's
> > sensible to have different orgs for different problems, but
> > nonetheless felt the PSF should get involved.
> >
> > This issue is not especially relevant to IPython in particular; I
> > approached this community as it's one that I feel close to, it's very
> > diverse and it has a long-standing tradition of contributing to free
> > software. I should have perhaps made it clearer that this isn't
> > IPython Dev stuff; it's just a topic this community should perhaps
> > discuss. There's no urgency, but, in the long run, IPython will
> > probably need to address the industry's political issues on some
> > level, even if the decision is to simply leave it to specialist
> > organisations.
> >
> > Cheers
> >
> > Carl
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130709/bdb3b4bb/attachment.html>

From ellisonbg at gmail.com  Tue Jul  9 20:57:33 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 17:57:33 -0700
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <CAPk-6T61UBya=HScHFYXGvETsAnOk0QB5nNfDX+YyzMpTU6ZqA@mail.gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
	<4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
	<CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>
	<CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>
	<CAPk-6T61UBya=HScHFYXGvETsAnOk0QB5nNfDX+YyzMpTU6ZqA@mail.gmail.com>
Message-ID: <CAH4pYpTr8VdjC4GwDHcgjgsGe39nTWFMnqvfy09Xxa5+LKR=OA@mail.gmail.com>

Anthony,

This is not a derail at all.  NumFocus would be the perfect level to
handle this type of support at - NumFocus is a legal entity, it has a
board and even "member projects".  A simple vote of the board and
member projects would be easy to do for this type of thing.  I would
be very much in favor of seeing NumFocus support IDL.

Cheers,

Brian

On Tue, Jul 9, 2013 at 5:19 PM, Anthony Scopatz <scopatz at gmail.com> wrote:
>
> On Tue, Jul 9, 2013 at 6:59 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> I personally like the goals of the IDL, but I think it is important
>> for projects like IPython to remain focused.  Because of this my vote
>> would be for this involvement to happen at the personal level.  I
>> would like IPython itself to remain a-political - with no opinions on
>> outside issues.  I feel this is especially important given the fact
>> that IPython is not a legal entity - it is only an informal group of
>> people who share collective copyright.  It is very difficult to
>> "speak" on behalf of our entire community because of this - both on
>> the practical and conceptual levels.  Groups like Mozilla and PHP or
>> even the PSF are legal entities so there is a clear path for
>> determining what exactly this type of support means (a vote of the
>> board).
>
>
> To hopefully not derail this thread too much and to speak to Brian's point
> about being a legal entity, this is more the sort of thing that would be
> appropriate for NumFOCUS to consider.  I suggest you join the NumFOCUS
> mailing list and bring it up there <numfocus at googlegroups.com>.
>
> Be Well
> Anthony
>
> 1. http://numfocus.org/
>
>>
>>
>> Cheers,
>>
>> Brian
>>
>> On Sun, Jul 7, 2013 at 11:12 AM, Carl Smith <carl.input at gmail.com> wrote:
>> > The main question, I guess, is whether IPython should explicitly
>> > support causes at all. Normally, an org will focus on it's own
>> > problems, regardless of its members views on non-related issues. That
>> > said, Mozilla and PHP have joined the IDL. Of course, Moz does have an
>> > interest in this issue, but PHP doesn't, at least directly. I just
>> > felt a little disappointed that it was PHP and not Python that I saw
>> > on that list, and that left me confused; I've always thought it's
>> > sensible to have different orgs for different problems, but
>> > nonetheless felt the PSF should get involved.
>> >
>> > This issue is not especially relevant to IPython in particular; I
>> > approached this community as it's one that I feel close to, it's very
>> > diverse and it has a long-standing tradition of contributing to free
>> > software. I should have perhaps made it clearer that this isn't
>> > IPython Dev stuff; it's just a topic this community should perhaps
>> > discuss. There's no urgency, but, in the long run, IPython will
>> > probably need to address the industry's political issues on some
>> > level, even if the decision is to simply leave it to specialist
>> > organisations.
>> >
>> > Cheers
>> >
>> > Carl
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Tue Jul  9 21:09:13 2013
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 10 Jul 2013 02:09:13 +0100
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <CAPk-6T61UBya=HScHFYXGvETsAnOk0QB5nNfDX+YyzMpTU6ZqA@mail.gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
	<4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
	<CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>
	<CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>
	<CAPk-6T61UBya=HScHFYXGvETsAnOk0QB5nNfDX+YyzMpTU6ZqA@mail.gmail.com>
Message-ID: <CAP-uhDcYkN90pvBvAMa9wRm3cOxu111eLqQLniqpxzLb7uErpQ@mail.gmail.com>

Thanks Brain and Anthony for your responses. Brain's point makes lot
of sense. I hadn't really considered the huge difference between a
community and a legal body. You may be right about NumFOCUS Anthony,
but I'll not bring it up on their list personally, just because I've
never posted there before, and there's people on this list with
influence there already.

Thanks again to both of you for being so considerate.

All the best

Carl

On 10 July 2013 01:19, Anthony Scopatz <scopatz at gmail.com> wrote:
>
> On Tue, Jul 9, 2013 at 6:59 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> I personally like the goals of the IDL, but I think it is important
>> for projects like IPython to remain focused.  Because of this my vote
>> would be for this involvement to happen at the personal level.  I
>> would like IPython itself to remain a-political - with no opinions on
>> outside issues.  I feel this is especially important given the fact
>> that IPython is not a legal entity - it is only an informal group of
>> people who share collective copyright.  It is very difficult to
>> "speak" on behalf of our entire community because of this - both on
>> the practical and conceptual levels.  Groups like Mozilla and PHP or
>> even the PSF are legal entities so there is a clear path for
>> determining what exactly this type of support means (a vote of the
>> board).
>
>
> To hopefully not derail this thread too much and to speak to Brian's point
> about being a legal entity, this is more the sort of thing that would be
> appropriate for NumFOCUS to consider.  I suggest you join the NumFOCUS
> mailing list and bring it up there <numfocus at googlegroups.com>.
>
> Be Well
> Anthony
>
> 1. http://numfocus.org/
>
>>
>>
>> Cheers,
>>
>> Brian
>>
>> On Sun, Jul 7, 2013 at 11:12 AM, Carl Smith <carl.input at gmail.com> wrote:
>> > The main question, I guess, is whether IPython should explicitly
>> > support causes at all. Normally, an org will focus on it's own
>> > problems, regardless of its members views on non-related issues. That
>> > said, Mozilla and PHP have joined the IDL. Of course, Moz does have an
>> > interest in this issue, but PHP doesn't, at least directly. I just
>> > felt a little disappointed that it was PHP and not Python that I saw
>> > on that list, and that left me confused; I've always thought it's
>> > sensible to have different orgs for different problems, but
>> > nonetheless felt the PSF should get involved.
>> >
>> > This issue is not especially relevant to IPython in particular; I
>> > approached this community as it's one that I feel close to, it's very
>> > diverse and it has a long-standing tradition of contributing to free
>> > software. I should have perhaps made it clearer that this isn't
>> > IPython Dev stuff; it's just a topic this community should perhaps
>> > discuss. There's no urgency, but, in the long run, IPython will
>> > probably need to address the industry's political issues on some
>> > level, even if the decision is to simply leave it to specialist
>> > organisations.
>> >
>> > Cheers
>> >
>> > Carl
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg at gmail.com  Tue Jul  9 21:46:02 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 18:46:02 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <51D54574.5070901@creativetrax.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<1372844910.16126.107.camel@peregrin>
	<495C148C-AC63-48CD-9D2C-5784C35B8922@gmail.com>
	<51D54574.5070901@creativetrax.com>
Message-ID: <CAH4pYpTxcSihqvY_mOb1ePn3pm9Q5D+-5hGAKtV31ym-FQHFfg@mail.gmail.com>

On Thu, Jul 4, 2013 at 2:50 AM, Jason Grout <jason-sage at creativetrax.com> wrote:
> On 7/4/13 2:13 AM, Matthias BUSSONNIER wrote:
>>
>> This is one place where I sometime disagree with Brian, where I think
>> input widget for codemirror would be great. To compare with Gabriel
>> 'interactive'
>> code cell, I would be more inclined to provide the ability to bind with
>> get to Codemirror
>> like in http://livecoding.io/3419309
>> through reg-exp it bind to any variable in the codecell and pop a widget
>> to change the value.
>> you don't have to explicitly state which variable should be bound.
>> Implementation detail CM provide a method to get token at cursor, which
>> helps a lot?
>> It has also the advantage of working without changing the cell type.
>
> I think input widgets like the livecoding.io examples are a powerful way
> to input/change code.  In the end, you are just having the computer help
> you write code, instead of hand-typing each letter.  Rather than looking
> up a separate website that lets me pick an html color and gives me a hex
> value, then pasting that hex value into the IPython cell, the pop-up
> widget in the cell pastes or changes the hex value right there.
>
> For another take on input widgets, see my proof-of-concept example here:
> http://bl.ocks.org/jasongrout/5378313  It would be easy to imagine a
> slider widget replacing a number, like:
>
> n = <slider>
>
> and a quick key to toggle between the slider and the number inside the
> codemirror widget.
>
> FYI, we've had a short discussion on widgets already [1]

Yes, and this variation of widgets will definitely come up in our
development sprint in two weeks.

Cheers,

Brian

> Thanks,
>
> Jason
>
> [1]
> https://github.com/ipython/ipython/issues/2802#issuecomment-16711626,
> and ensuing comments
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Jul  9 22:32:47 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 19:32:47 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
Message-ID: <CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>

Gabriel,

> Thank you for taking the time to watch my video and think about the ideas
> I'm presenting. It is appreciated.

Cool, I enjoyed it.  Fantastic discussion!

> I am cognizant of the added complexity I am talking about. I disagree a bit
> with the size/damage being attributed to it, but the main place we seem to
> be on different pages is regarding what it buys us.

I 1/2 agree with you.  Obviously, every feature we add might add
complexity to the code base.  At the same time we are always striving
to try and reduce complexity with better design as we add new
features.  Here is one of the ways we think about new features:

Q: does the new feature violate important abstractions we have in place.

If the answer is no, then we do our normal job of considering the
costs of adding the feature versus the benefits.

If the answer is yes, then we *stop*.  The cost of changing one of our
core abstractions happens at an entirely different level.  It is the
type of thing we would think about for a very long time and have lots
of conversations with lots of different people.  We would possibly
even seek out major (>$100k) funding for the effort.  For these
discussions, we think about the bigger picture outside of the context
of that one feature.  If we decide to make a major change to one of
our core abstractions, we would probably plan at least 3 months - 2
years in advance.  Here is the roadmap for IPython for this type of
work through the end of 2014:

https://github.com/ipython/ipython/wiki/Roadmap:-IPython

Thinking about your proposed feature from this perspective: both the
task cells and alt cells introduce hierarchy and nesting into the
notebook.  This breaks our core abstraction that cells are not nested.
 In Jan-Feb our core development team had a discussion about this
abstraction exactly.  We decided that we definitely don't want to move
in the direction of allowing nesting in the notebook.  Because of this
we are in the process of removing the 1 level of nesting our notebook
format currently has, namely worksheets.  So for us, it is not just
about complexity - it is about breaking the abstractions.

The reason that these abstractions are so important is that they
provide powerful foundations for us to build on.  One place the
"notebook as a linear sequence of cell" abstraction comes into play is
in our work on nbconvert that will appear in 1.0 in the next few
weeks.  This allows to to convert notebooks very easily to a number of
different formats.  The other place this abstraction comes into play
is in our keyboard shortcuts.  We are striving for the notebook to be
usable for people who dont' touch the mouse (your traditional vi/emacs
users).  Nesting makes that very difficult.

Before you get too discouraged, please read on :-)

> The alt cells I'm talking about are not a cool rendering/interaction trick
> for linear notebooks. If they were I would absolutely agree that the nesting
> is overkill and not worth its cost.
>
> Linear/sequental notebooks and other dynamic document systems reproduce
> computational results. The documents my advisors (Duncan Temple Lang and
> Deborah Nolan) and I are working towards aim to describe and reproduce the
> research itself.

I appreciate your sharing more of the context.

> I once asked a room full of quants to raise their hands if their standard
> operating procedure was to read in the data, maybe clean it a bit, fit a
> single model, write up the results and be done. Not only did no one raise
> their hand, but the mere suggestion that that could be how it worked got a
> substantial laugh from the audience. Even though this is not how the work is
> done, however, it is the narrative encoded into a linear notebook.

Here is my experience of this.  I start out working in a very
non-linear manner.  As I work I discover things and change my code.
As I approach the point where I want to share my work, I start to
linearize it, otherwise it is very difficult for someone to take in.
In this context branching can be done by it has to be explicit.  In my
experience this is good.  If I want to run the analysis using 3
different algorithms, I run them in sequence and then show the results
of all three in the same place and draw conclusions.  All of this is
done - at the end of the day - in a linear notebook.

BUT, I completely agree that the notebook does not handle certain
types of branching very well.  Where the notebook starts to really
suck is for longer analyses that you want to repeat for differing
parameters or algorithms.  You talk more about this usage case below
and we have started to think about how we would handle this.  Here are
our current thoughts:

It would be nice to write a long notebook and then add metadata to the
notebook that indicates that some variables are to be treated as
"templated" variables.  Then we would create tools that would enable a
user to run a notebook over a range of templates:

for x in xvars:
  for y in yvars:
    for algo in myalgos
    run_notebook('MyCoolCode', x, y, algo)

The result would be **something** that allows the user to explore the
parameter space represented.  A single notebook would be used as the
"source" for this analysis and the result would be the set of all
paths through the notebook.  We have even thought about using our
soon-to-be-designed interactive widget architecture to enable the
results to be explored using different UI controls (sliders, etc) for
the xvar, yvar, algos.  This way you could somehow "load" the
resulting analysis into another notebook and explore things
interactively - with all of the computations already done.

We have other people interested in this type of workflow and it can
all be done within the context of our existing linear notebook model.
It is just assembling the existing abstractions in different ways.

> Scientists and data analysts are already doing branching in their work, but
> they don't have any good tools to record or describe it that way. So they
> comment out large blocks of code in linear scripts, or they save old
> attempts and alternate approaches in separate files, or they (hopefully not)
> simply delete or overwrite old parameter configurations with new ones when
> they decide the old one wasn't right. Not because they think these are good
> ideas, but because its all they have available and they are
> scientists/analysts.

I completely agree with this.

> Their job is to analyze data, extract insight and share their work in a
> useful manner; our job is to conceive and implement the tools they need to
> do that. They are good at their job, but we are struggling with ours.
>
> In my opinion, the single most important feature on display in my video is
> not that the alternative cells are rendered side-by-side, or that they can
> be executed and the software knows what that means in terms of executing
> their content; it is that the IPython notebook has become an authoring tool
> which analysts can use to easily create documents which describe what they
> actually did in a way that is reproducible, distributable, and most
> importantly, useful to the analyst.

I completely agree with this - I just think it can be done within our
existing abstractions - or at least I want to see us fail before
breaking the abstractions.

> Suddenly they don't need to comment out or overwrite code when they take a
> different approach. Suddenly they can distribute a document that actually
> describes what they did, instead of just what they found. Suddenly referees
> don't need to wonder or ask about whether an alternative analysis method was
> investigated. Suddenly professors can show statistics students what
> statisticians actually do, instead of how final results are generated.

Amen!

> You may read all this and think "that is great but way out of scope for what
> we want to achieve with IPython notebook". That is your right. And if the
> IPython team feels that way there is, of course, nothing I can do other than
> what I have done: explore these ideas on my own. I just want to be sure
> we're all talking about the same thing before that decision gets made.

No I don't think the usage case is out of scope.  I hope I have
convinced you that we are already thinking along these lines.  The
reason that IPython exists is because we are all working scientists
who have used crappy tools for years.  I wrote my entire Ph.D. thesis
and postdoc codes using a rats nest of C++, perl, bash and Makefiles.
It was even *advanced* for the day - I used version control and had
some tests.  But if I had to reproduce this research today, I would
start from scratch.  This wasn't my fault per se - it was the fact
that my tools didn't accurately express the reality of the
abstractions in my work flow.  But you know all of this...

Here is a question: are you and your advisors interested in meeting
with us at Berkeley and talking more about these things - with the
understanding that we are very much interested in the usage case you
describe - but probably not the nested implementation at this point.
Fernando is at Berkeley with much of the IPython dev team and I am 3
hours south at Cal Poly with another part of the team.  I don't think
it will be difficult to write prototypes of these capabilities using
the linear notebook, we just have to think about what the user
experience would look like.  We could probably do this after the
summer madness ends (probably Sept, Oct).

Even if you are not interested in talking with us in person, I hope
you are still willing to continue the discussion. The branching usage
case is very important to us and will be a part of our future work.
It really helps to think more about the design and user experience
questions.

Cheers,

Brian



> Detailed responses below.
>
>
> On Wed, Jul 3, 2013 at 6:59 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> Gabriel,
>>
>> I watched your video and there are some nice ideas here.  We are not
>> headed in this direction in terms of *implementation* but I think you
>> will find that similar *capabilities* will show up in the notebook
>> over time.  A few comments about the implementation aspects:
>>
>> First, the benefits of having a notebook be a linear sequence of cells
>> are massive:
>>
>> * Simple, simple simple - this makes it very easy to reason about
>> notebooks in code.  Nesting leads to complexity that is not worth the
>> cost.
>
>
> It does lead to complexity. Whether it is worth the cost depends on what we
> gain. In my opinion I think we gain a massive amount, as I tried to describe
> above.
>
>>
>> * You can get most of the benefits of nesting without the complexity.
>> As Min mentioned, there is an implied hirrarchy in the heading cells.
>> We plan on using that to allow group level actions - show/hide, run
>> group, move, cut/copy/paste, etc.
>
>
> With respect, I don't know that you will get the actual benefits I am aiming
> for this way. Branches (and thus the code contained in them) in this context
> are mutually exclusive.  A document which has branching that leads back into
> a single computation, such as the example in my video/screenshots, no longer
> makes sense to think about or execute as a sequential set of code blocks.
>
> It is not that we can view it as a branching structure and only execute one
> branch at a time, but that we must. Again, people do this already via
> deletion or commenting because all the code for all the branches cannot live
> in the same block of code. I seek to give them a tool to do it that is not
> damaging to the record or reproducibilty of what they did.
>
> If you are familiar with R code and look carefully at the code for the data
> cleaning branches in my example, you will see that if we put all of that
> code into a sequential notebook, with heading cells to differentiate the
> branches, and ran the code start to finish it would generate a plot not
> created by any of the 18 paths through the structured document. In other
> words, it generates the wrong plot regardless of what the "right" set of
> choices at the branching points is in that execution context.
>
>>
>> * It is not difficult to think about building a proper diff tool for
>> notebooks.  With nested cells this becomes horrific.
>
>
> This does become much more difficult, but there are still things which can
> be done. Archambault and others have done work on differencing graphs under
> the assumption that one of them modified to create the other ( see here and
> here for quickly retrieved examples)
>
> Thus we can combine graph differencing with differencing of the actual code,
> e.g. a combination of difference maps  and normal code diffing.
>
> This would require unique identifiers for all notebook cells, but from
> something said previously in this thread I gather that is likely to happen
> anyway.
>
>>
>> * Hierarchy puts a significant cognitive load on users
>
>
> Maybe. Again, this is something people are already implicitly doing. It is
> possible that users would have trouble thinking about what they are doing
> more explicitly in these terms, but its also entirely possible that this
> will actually simplify their thinking process.
>
> I admittedly think about these types of documents a lot, but I was
> pleasantly surprised at how easy it was to construct and interact with the
> example I made for the video once the UI hooks were in place. Constructing
> such documents by hand, which I have done a bit, IS quite unpleasant, but
> that is one of the issues I seek to address.
>
> Nested branching and other more complex structures are more difficult to
> think about, but users would only create those if they had a really good
> reason to do so (ie if they were already thinking in those terms) and just a
> couple levels of branching buy us an enormous amount of fidelity when
> describing data analyses.
>
>
>>
>>
>> Because of these things we don't have any plans on changing the
>> notebook document format or notebook UI to allow nested cells.
>>
>
> As I said above, it is absolutely the right of the IPython core team to make
> that call. I just want to make sure we are all talking about the same costs
> and the same gains when the cost-benefit analysis to make that decision is
> performed.
>
> I intend to continue pursuing these concepts. I would be thrilled to
> collaborate with the IPython team if you guys decide it is something you are
> interested in, but I will understand and carry on myself if you do not.
>
> Thanks for reading and for the discussion. Its great to talk to people with
> differing opinions about this instead of sitting alone in my thesis-filled
> bubble stewing :).
>
> ~G
>
>
>> Second, while it is tempting to generalize the notion of input to
>> include widgety things, it is more appropriate to put these things in
>> the output:
>>
>> * Our output architecture has the notion of multiple representations.
>> This allows us to build rich widget as you have done, but to still
>> provide static representations (png, jpg, latex).
>> * Having the multiple representations of output allows us to build the
>> rich widgets, but maintain a clear path for converting notebooks to
>> static formats (pdf, html, word, powerpoint).
>> * Insisting that input cells are pure code allows you to reason in a
>> clear manner about how a notebook works = code runs and leads to
>> output.  That reasoning can be applied in an automate manner by
>> running notebooks in batch mode, or building a test system based on
>> them.
>> * Putting widgets in the input area forces you to do regular
>> expression matching to replace those variables in the code.  This
>> limits you to an extremely simple event model where the only possible
>> event you can know about is substitute the regular expression and run
>> all the code.  What if you want different UI controls in the browser
>> to trigger different bits of code in the kernels when different fine
>> grained events happen.  Making the UI controls live on the Python and
>> JS side allows us to build this in a natural way.
>>
>> Th alt-cells you show bring up the issue of providence.  We have some
>> very initial thoughts about that, but it is way out of scope for the
>> project right now - we have a plates 10x overfull already.  We will
>> get there though eventually.
>>
>> Thanks for sharing your ideas.
>>
>> PS - for a bit more background about the context of our saying "no" to
>> this feature request, see this blog post:
>>
>> http://brianegranger.com/?p=249
>>
>> I also gave a talk about this at SciPy and will be posting my slides soon.
>>
>> Cheers,
>>
>> Brian
>>
>> On Wed, Jul 3, 2013 at 6:04 PM, Gabriel Becker <gmbecker at ucdavis.edu>
>> wrote:
>> > Matthias,
>> >
>> > Thanks for your detailed response.
>> >
>> >
>> > On Wed, Jul 3, 2013 at 1:25 AM, Matthias BUSSONNIER
>> > <bussonniermatthias at gmail.com> wrote:
>> >>
>> >> Gabriel,
>> >>
>> >> You screen shot are interesting,
>> >> At some point I played with gridster[1]
>> >>
>> >> and was more or less able to get cell to rearranges, but didn't keep
>> >> the
>> >> code.
>> >> You might be interested.
>> >>
>> >> Keep in mind that the notebook browser-interface we ship is only one
>> >> possible
>> >> frontend that can interpret ipynb files, nothing prevent you to write a
>> >> different frontend that display the notebook in a different format.
>> >>
>> >> This added to the fact that each cell can support arbitrary metadata,
>> >> you
>> >> should be able to arrange preexisting in structure that work together.
>> >> It
>> >> might
>> >> be a little difficult to do it right now as our javascript is not yet
>> >> modular
>> >> enough to be easily reused, but we are moving toward it.
>> >
>> >
>> > Respectfully, rolling my own frontend for ipynb files given all the work
>> > the
>> > IPython team has done on the excellent notebook browser interface would
>> > be
>> > an enormous and extremely wasteful duplication of effort. I don't think
>> > its
>> > the right way to pursue these features.
>> >
>> > Furthermore, if I were going to write an application offering the types
>> > of
>> > features I am talking about from scratch, there wouldn't be any good
>> > reason
>> > to base it on the unaltered ipynb format, as they don't easily support
>> > the
>> > structure required by those features without the additional cell types I
>> > implemented in my forked version.
>> >
>> >>
>> >> Right now I thing storing the notebook as a directed graph is
>> >> problematic
>> >> in a
>> >> few way,
>> >
>> >
>> > I'm not talking about storing the notebook as an actual directed graph
>> > data
>> > structure. There would be benefits to that but its not necessary and it
>> > isn't want I did in my forked version.
>> >
>> > The ability to have nested cells (cells which contain other cells) gets
>> > us
>> > everything we need structure wise, and is the basis of everything seen
>> > in
>> > both the video (other than interactive code cell stuff) and screenshots
>> > I
>> > posted. The ipynb file for the notebook pictured in the screenshot looks
>> > exactly like a normal ipynb file except that in the json there are cell
>> > declarations which have a cells field which contains the json
>> > descriptions
>> > of the cells contained in that cell.
>> >
>> >
>> >>
>> >> the first being that it is incompatible with the fact that people want
>> >> to be able to run notebook in a headless manner, which if you add
>> >> explicit
>> >> choice is not possible.
>> >
>> >
>> > This isn't the case. The json saved versions of notebooks with branching
>> > remember which version was most recently run. When an altset cell is
>> > executed, it runs only the most recently run (or currently "selected",
>> > though that means something else internally) branch. Thus by doing the
>> > naive
>> > thing and looping through all top level cells and executing them, the
>> > currently chosen path through the notebook can easily be run in a
>> > headless
>> > environment and give the correct results.
>> >
>> >>
>> >> This also contradict the fact that the notebook capture
>> >> both the input and the output of the computation.
>> >
>> >
>> > I don't really understand what you mean by this. In the JSON
>> > representation
>> > of an executed code cell, the input field contains the code, but not any
>> > values of variables used by the code, nor any indication of code which
>> > was
>> > run before executing the code cell.
>> >
>> > Changing and rerunning an earlier code cell without re-executing the
>> > cell in
>> > question can easily invalidate the output stored in the JSON, even
>> > without
>> > the concept of branching or choice.
>> >
>> >
>> >>
>> >> As you showed there is
>> >> actually 18 different combinations of data analysis, and they are not
>> >> all
>> >> stored in the notebook.
>> >
>> >
>> > The notebook knows and records which choices were made. There are 18
>> > different combinations of data analysis but only one was chosen by
>> > analyst
>> > as generating the final/most recent result.
>> >
>> > In the case of "publishing" about an analysis the notebook stores the
>> > path
>> > most chosen by the analyst, while retaining information about what else
>> > he
>> > or she did during the decision process.
>> >
>> > In the case of instruction, imagine how much easier it would be to teach
>> > data analysis if the students could actually see what data analysts do,
>> > instead of simply the final method they choose in a particular analysis.
>> >
>> >
>> >>
>> >>
>> >> I really thing this is an interesting project, and reusing only our
>> >> metadata in
>> >> the notebook, you should be able to  simulate it (store the dag in
>> >> notebook
>> >> level metadata, and cell id in cell metadata) then reconstruct the
>> >> graph
>> >> when
>> >> needed. Keep in mind that at some point we might/will add cell id to
>> >> the
>> >> notebook.
>> >>
>> >> To sum up, I don't think the current JS client is in it's current state
>> >> the
>> >> place to implement such an idea. The Dag for cell order might be an
>> >> idea
>> >> for
>> >> future notebook format but need to be well thought, and wait for cell
>> >> IDs.
>> >
>> >
>> > I apologize for not being clear. As I said in a response above, the
>> > directed
>> > graph idea was intended to be conceptual for thinking about the
>> > documents,
>> > not structural for actually storing them.
>> >
>> > What I actually did was simply allow cell nesting and change indexing so
>> > that it is with respect to the parent/container (cell or notebook)
>> > instead
>> > of always with respect to the notebook. This required some machinery
>> > changes
>> > but not too many and it is an extension in the mathematical sense in
>> > that
>> > indexing will behave identically to the old system for notebooks without
>> > any
>> > nesting while now meaningfully functioning for notebooks with nesting.
>> >
>> > ~G
>> >>
>> >>
>> >> --
>> >> Matthias
>> >>
>> >>
>> >>
>> >> [1] http://gridster.net/
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >
>> >
>> >
>> > --
>> > Gabriel Becker
>> > Graduate Student
>> > Statistics Department
>> > University of California, Davis
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Wed Jul 10 00:02:36 2013
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 10 Jul 2013 05:02:36 +0100
Subject: [IPython-dev] Notebook: File Browser
Message-ID: <CAP-uhDfyAAy7MNA8uiGXwb30NFsDN-ysTF4B5TyxPXxi6t=BiQ@mail.gmail.com>

I noticed that there's still plans to add a GUI file browser to the
dashboard. Can I just ask why we can't have a magic to open a
notebook? The Notebook already handles every file operation, except
opening notebooks, so what does adding a GUI file browser buy anyone?
especially if there's some kind of shell or single cell interface in
the dashboard.

It'd be nice if I could call `ls` in the Notebook and have the output
clickable, so each file would be a link. Clicking a directory would cd
to it, clicking a notebook would open it, clicking an image would
render it... something like that. That way user could still click
their way around the file system, just hitting links instead of icons,
and adding icons is easy in a browser, but probably going a bit too
far.


From ellisonbg at gmail.com  Wed Jul 10 00:08:18 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 9 Jul 2013 21:08:18 -0700
Subject: [IPython-dev] Notebook: File Browser
In-Reply-To: <CAP-uhDfyAAy7MNA8uiGXwb30NFsDN-ysTF4B5TyxPXxi6t=BiQ@mail.gmail.com>
References: <CAP-uhDfyAAy7MNA8uiGXwb30NFsDN-ysTF4B5TyxPXxi6t=BiQ@mail.gmail.com>
Message-ID: <CAH4pYpQh1T5LvN4KM+OKbpZ_=C4bWD9k56WcDpYGLvDo5_FTRw@mail.gmail.com>

The biggest change we have in the works is that the URLs for notebooks
will start to use paths and notebook names like this:

/notebook/path/to/my/notebook.ipynb

This will make it trivial to write the type of code you are talking
about.  But I think we will probably still have an actual GUI in the
dashboard for navigation.  We will be talking about this stuff at the
dev meeting in two weeks.

Cheers,

Brian

On Tue, Jul 9, 2013 at 9:02 PM, Carl Smith <carl.input at gmail.com> wrote:
> I noticed that there's still plans to add a GUI file browser to the
> dashboard. Can I just ask why we can't have a magic to open a
> notebook? The Notebook already handles every file operation, except
> opening notebooks, so what does adding a GUI file browser buy anyone?
> especially if there's some kind of shell or single cell interface in
> the dashboard.
>
> It'd be nice if I could call `ls` in the Notebook and have the output
> clickable, so each file would be a link. Clicking a directory would cd
> to it, clicking a notebook would open it, clicking an image would
> render it... something like that. That way user could still click
> their way around the file system, just hitting links instead of icons,
> and adding icons is easy in a browser, but probably going a bit too
> far.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Wed Jul 10 01:46:08 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 9 Jul 2013 22:46:08 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
Message-ID: <CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>

One problem with prefixing all pages is that those prefixes, if I
understand how GH wikis work correctly, have to be part of the visible
page title.  Which for some pages makes for pretty horrible names.
For example the gallery title is perfectly fine as

A gallery of interesting IPython Notebooks

but it would look pretty ugly if called

Cookbook: A gallery of interesting IPython Notebooks

or similar, IMO.

Since there's no actual auto-indexing machinery or other organizing
mechanism that these pseudo-namespace prefixes provide, I'm also not
convinced that it makes sense to mandate their use rigidly everywhere.
 For certain categories I think they do make sense, like say cookbook
recipes or sprints lists. But I think we're inevitably bound to have
'miscellaneous' pages that don't neatly fit in any single label.  And
creating a  zillion prefixes just so everything can have its own
prefix doesn't seem to be such a good idea either.

In summary, I think the problem is that wikis are inherently prone to
some of this slightly chaotic growth, which is why I've never liked
them too much.  I think they fill a useful role when used in a pretty
*limited* fashion, partly so that they can be manually tended to.  But
for larger-scale efforts, without a massive metadata machinery and an
army of people like what Wikipedia uses, it's kind of hopeless. For
wikipedia, pages have very rich categorical/tagging tools that the
entire wiki system understands, mines, and uses to offer management
and navigation.  We don't have that at GH, just a plain wiki in its
barest form.

Sorry to be a bit of a downer, but I'm just not convinced that more
prefixing is really going to really solve anything beyond a thin
cosmetic veneer issue.

Cheers,

f

ps - I'm pretty sure, BTW, that I'm the guilty party in having created
the lab meetings page without a prefix, sorry :)  Part of it is
probably that I find it so ugly to read those titles that I
unconsciously just ignored the idea... Feel free to change it if you
really think it will help.

On Tue, Jul 9, 2013 at 5:07 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Hi,
>
> A while back I spend a couple of days organizing the IPython GitHub
> wiki.  I know this will probably seem anal, but I think it is
> important that we use a logical structure for each of our pages and
> the wiki as a whole.  To induce this structure on the flat GitHub wiki
> namespace, we are naming our wiki pages with a prefix such as "Dev :"
> or "Cookbook: " or "Install: ".  This structure is also present in our
> sub-indices - we have a separate index page for each of these
> categories.  When someone created the wiki page for the lab meetings,
> it was created at the top level, rather than under the "Dev: "
> category and put in the top level index rather than the "Dev: " index.
>  I could fix this myself in a few minutes, but I don't want to become
> the "police" for our wiki organization.  Instead, I would like
> everyone to help keep the wiki organized.  Can someone clean this up?
> If we don't want to use this structure, but instead keep things
> organized another way, I am completely open to that.  I just want the
> wiki organization to decay into chaos like it did before...
>
> Cheers,
>
> Brian
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From fperez.net at gmail.com  Wed Jul 10 02:02:36 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 9 Jul 2013 23:02:36 -0700
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
Message-ID: <CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>

Fantastic if you guys have a drinkup, but you shouldn't tie it to 1.0
if you plan on going drinking in a week :)  It's pretty clear that our
schedule is going to slip a little bit: realistically, we'll put out
an alpha or beta before our dev meeting, let it simmer during that
week for testing/feedback, and return to polish up the release after
the dev meeting.

So by all means go and drink next week, but the real 1.0 drinkup(s)
will have to wait just a little longer :)

Cheers,

f

On Tue, Jul 9, 2013 at 2:12 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
> Hi list,
>
> As you all know, soon should be the release of IPython 1.0 which is theoretically due in one week.
> To celebrate this, I suggest user and devs near Paris to gather for a drink this weeks, or next one.
>
> For the dates, If people are available this week-end(13,14), monday(15), or thurstday(18)
> please select below
>
> http://www.doodle.com/8ii6v2nzacaxc5xg
>
> Yes I know, I choose only date where I'm free, I can change if necessary
> but I'm the one who proposed it would be a shame for me not to be available.
>
> Depending on wether you prefer wine or beer, I propose the following places.
>
> Beer,
> L'academie de la bi?re  http://goo.gl/maps/NmMd4
> La gueuse :  http://goo.gl/maps/a0KLL
>
> Wine
> If we are not too many, and depending on the date, I know a quiet wine bar, not too expensive also:
> Les grains nobles: http://goo.gl/maps/K4aJb
>
> I also hope this will inspire the same things around the world.
>
> --
> Matthias
> P.S.: Unlike github's drinkups you have to pay your own drink.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From fperez.net at gmail.com  Wed Jul 10 02:06:09 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 9 Jul 2013 23:06:09 -0700
Subject: [IPython-dev] Internet Freedom
In-Reply-To: <CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>
References: <CAP-uhDcKarsm7L6dFViaiSuOzTfZLX8AwAdog4vvXPjXwQunbQ@mail.gmail.com>
	<4316BB2F-3F8A-4F4A-8A8F-9E409A1A8C37@gmail.com>
	<CAP-uhDf39yjdtBS28jg+SPAFrvOmAdbQFZivVfbEBJkWKZYyWw@mail.gmail.com>
	<CAH4pYpSdw-px9b_WtnpVCeTN3QT6wjqFjo469ZecR3XmS=Gcrw@mail.gmail.com>
Message-ID: <CAHAreOpKfRVYdZEPaSY0S8YjHt9QO9UmUsaee4X0yTDkGkM8Ew@mail.gmail.com>

Catching up a bit on email...

I pretty much agree completely with what Brian and Anthony said, I
just wanted to record my voice on it.  As strongly as many of us may
feel about many of these issues, it is indeed somewhat orthogonal to
what brings us to IPython as a group of interested users and
developers, and in that sense Numfocus is a much better venue.

Nothing wrong with mentioning something like this here, Carl, but I
think that any concrete discussion and action is fully within
Numfocus' scope and mission, so that's the right place to continue...

Cheers,

f

On Tue, Jul 9, 2013 at 4:59 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> I personally like the goals of the IDL, but I think it is important
> for projects like IPython to remain focused.  Because of this my vote
> would be for this involvement to happen at the personal level.  I
> would like IPython itself to remain a-political - with no opinions on
> outside issues.  I feel this is especially important given the fact
> that IPython is not a legal entity - it is only an informal group of
> people who share collective copyright.  It is very difficult to
> "speak" on behalf of our entire community because of this - both on
> the practical and conceptual levels.  Groups like Mozilla and PHP or
> even the PSF are legal entities so there is a clear path for
> determining what exactly this type of support means (a vote of the
> board).
>
> Cheers,
>
> Brian
>
> On Sun, Jul 7, 2013 at 11:12 AM, Carl Smith <carl.input at gmail.com> wrote:
>> The main question, I guess, is whether IPython should explicitly
>> support causes at all. Normally, an org will focus on it's own
>> problems, regardless of its members views on non-related issues. That
>> said, Mozilla and PHP have joined the IDL. Of course, Moz does have an
>> interest in this issue, but PHP doesn't, at least directly. I just
>> felt a little disappointed that it was PHP and not Python that I saw
>> on that list, and that left me confused; I've always thought it's
>> sensible to have different orgs for different problems, but
>> nonetheless felt the PSF should get involved.
>>
>> This issue is not especially relevant to IPython in particular; I
>> approached this community as it's one that I feel close to, it's very
>> diverse and it has a long-standing tradition of contributing to free
>> software. I should have perhaps made it clearer that this isn't
>> IPython Dev stuff; it's just a topic this community should perhaps
>> discuss. There's no urgency, but, in the long run, IPython will
>> probably need to address the industry's political issues on some
>> level, even if the decision is to simply leave it to specialist
>> organisations.
>>
>> Cheers
>>
>> Carl
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From bussonniermatthias at gmail.com  Wed Jul 10 02:48:17 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 10 Jul 2013 08:48:17 +0200
Subject: [IPython-dev] Notebook: File Browser
In-Reply-To: <CAH4pYpQh1T5LvN4KM+OKbpZ_=C4bWD9k56WcDpYGLvDo5_FTRw@mail.gmail.com>
References: <CAP-uhDfyAAy7MNA8uiGXwb30NFsDN-ysTF4B5TyxPXxi6t=BiQ@mail.gmail.com>
	<CAH4pYpQh1T5LvN4KM+OKbpZ_=C4bWD9k56WcDpYGLvDo5_FTRw@mail.gmail.com>
Message-ID: <26F0167B-59E4-4F2C-8C2A-5D9B4AA4E4AF@gmail.com>





Le 10 juil. 2013 ? 06:08, Brian Granger a ?crit :

> The biggest change we have in the works is that the URLs for notebooks
> will start to use paths and notebook names like this:
> 
> /notebook/path/to/my/notebook.ipynb
> 
> This will make it trivial to write the type of code you are talking
> about.  But I think we will probably still have an actual GUI in the
> dashboard for navigation.  We will be talking about this stuff at the
> dev meeting in two weeks.

I disagree

If I'm on my laptop, 
the server on rackspaces 
and the kernels on amazon. 
The magic don't make sense.

(same if the 3 are on the same machine, but it's more difficult to see if you don't know the internal)

-- 
Matt



> 
> Cheers,
> 
> Brian
> 
> On Tue, Jul 9, 2013 at 9:02 PM, Carl Smith <carl.input at gmail.com> wrote:
>> I noticed that there's still plans to add a GUI file browser to the
>> dashboard. Can I just ask why we can't have a magic to open a
>> notebook? The Notebook already handles every file operation, except
>> opening notebooks, so what does adding a GUI file browser buy anyone?
>> especially if there's some kind of shell or single cell interface in
>> the dashboard.
>> 
>> It'd be nice if I could call `ls` in the Notebook and have the output
>> clickable, so each file would be a link. Clicking a directory would cd
>> to it, clicking a notebook would open it, clicking an image would
>> render it... something like that. That way user could still click
>> their way around the file system, just hitting links instead of icons,
>> and adding icons is easy in a browser, but probably going a bit too
>> far.
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From stefan at sun.ac.za  Wed Jul 10 04:40:20 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 10 Jul 2013 10:40:20 +0200
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
Message-ID: <CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>

On 10 Jul 2013 04:33, "Brian Granger" <ellisonbg at gmail.com> wrote:
> It would be nice to write a long notebook and then add metadata to the
> notebook that indicates that some variables are to be treated as
> "templated" variables.  Then we would create tools that would enable a
> user to run a notebook over a range of templates:
>
> for x in xvars:
>   for y in yvars:
>     for algo in myalgos
>     run_notebook('MyCoolCode', x, y, algo)

I think it would be feasible to build git-like cell revision tracking on
top of the linear notebook, especially once widgets are in place. Imagine a
per-cell and per-notebook check-in that allows you to swap out different
versions of a cell, or different notebook configurations. This can be done
with a very lightweight abstraction that does not interfere with the
underlying notebook structure at all (in fact, I'd just use git as the
actual backend, and then have a layer on top that knows how to pick out
tagged cells from a nb).

St?fan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/e965cc7c/attachment.html>

From carl.input at gmail.com  Wed Jul 10 07:07:57 2013
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 10 Jul 2013 12:07:57 +0100
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
Message-ID: <CAP-uhDeZ5Ndri3LAiRE_-6CJftcHX-KNicrRDj30rStFgYAuxQ@mail.gmail.com>

I saw a talk once that made the case that you'll always need a
Miscellaneous category for this kind of thing. Personally, I quite like the
prefixes.

Fernando, thanks for your reply on the IDL thread.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/54bcf0d4/attachment.html>

From asmeurer at gmail.com  Wed Jul 10 10:41:32 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Wed, 10 Jul 2013 09:41:32 -0500
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
Message-ID: <-1670235088980259958@unknownmsgid>

Embrace the chaos.

You don't use naming conventions for your branches in git. That's because
branches are cheap and throwaway, and it would be a waste of time to
organize them. Wikis are the same way.

Take a look at the SymPy wiki pages
https://github.com/sympy/sympy/wiki/_pages. The only official naming
convention we have is for GSoC pages. We use the wiki for throwaway tests,
developer guides, user written pages about uses of SymPy, little bits of
"documentation" that don't belong in our docs like comparisons to other
computer algebra systems, and prototypes and write ups for new features.

Our organization comes from a listing of the important pages on the main
page. That only accounts for about half the wiki pages. For the rest, there
is the list of all pages, which is what I usually use. If you want a
search, clone the git and use git grep.

You probably won't embrace the chaos, so my recommendation is to create an
index page (or just use the main page) and organize the pages by there by a
list of links under headers. That's much less painful than page naming
conventions.

Aaron Meurer

Sent from my iPad

On Jul 10, 2013, at 12:47 AM, Fernando Perez <fperez.net at gmail.com> wrote:

One problem with prefixing all pages is that those prefixes, if I
understand how GH wikis work correctly, have to be part of the visible
page title.  Which for some pages makes for pretty horrible names.
For example the gallery title is perfectly fine as

A gallery of interesting IPython Notebooks

but it would look pretty ugly if called

Cookbook: A gallery of interesting IPython Notebooks

or similar, IMO.

Since there's no actual auto-indexing machinery or other organizing
mechanism that these pseudo-namespace prefixes provide, I'm also not
convinced that it makes sense to mandate their use rigidly everywhere.
For certain categories I think they do make sense, like say cookbook
recipes or sprints lists. But I think we're inevitably bound to have
'miscellaneous' pages that don't neatly fit in any single label.  And
creating a  zillion prefixes just so everything can have its own
prefix doesn't seem to be such a good idea either.

In summary, I think the problem is that wikis are inherently prone to
some of this slightly chaotic growth, which is why I've never liked
them too much.  I think they fill a useful role when used in a pretty
*limited* fashion, partly so that they can be manually tended to.  But
for larger-scale efforts, without a massive metadata machinery and an
army of people like what Wikipedia uses, it's kind of hopeless. For
wikipedia, pages have very rich categorical/tagging tools that the
entire wiki system understands, mines, and uses to offer management
and navigation.  We don't have that at GH, just a plain wiki in its
barest form.

Sorry to be a bit of a downer, but I'm just not convinced that more
prefixing is really going to really solve anything beyond a thin
cosmetic veneer issue.

Cheers,

f

ps - I'm pretty sure, BTW, that I'm the guilty party in having created
the lab meetings page without a prefix, sorry :)  Part of it is
probably that I find it so ugly to read those titles that I
unconsciously just ignored the idea... Feel free to change it if you
really think it will help.

On Tue, Jul 9, 2013 at 5:07 PM, Brian Granger <ellisonbg at gmail.com> wrote:

Hi,


A while back I spend a couple of days organizing the IPython GitHub

wiki.  I know this will probably seem anal, but I think it is

important that we use a logical structure for each of our pages and

the wiki as a whole.  To induce this structure on the flat GitHub wiki

namespace, we are naming our wiki pages with a prefix such as "Dev :"

or "Cookbook: " or "Install: ".  This structure is also present in our

sub-indices - we have a separate index page for each of these

categories.  When someone created the wiki page for the lab meetings,

it was created at the top level, rather than under the "Dev: "

category and put in the top level index rather than the "Dev: " index.

I could fix this myself in a few minutes, but I don't want to become

the "police" for our wiki organization.  Instead, I would like

everyone to help keep the wiki organized.  Can someone clean this up?

If we don't want to use this structure, but instead keep things

organized another way, I am completely open to that.  I just want the

wiki organization to decay into chaos like it did before...


Cheers,


Brian


--

Brian E. Granger

Cal Poly State University, San Luis Obispo

bgranger at calpoly.edu and ellisonbg at gmail.com

_______________________________________________

IPython-dev mailing list

IPython-dev at scipy.org

http://mail.scipy.org/mailman/listinfo/ipython-dev




-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/345a8bf5/attachment.html>

From nelle.varoquaux at gmail.com  Wed Jul 10 11:43:25 2013
From: nelle.varoquaux at gmail.com (Nelle Varoquaux)
Date: Wed, 10 Jul 2013 17:43:25 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
Message-ID: <CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>

If you want more people to join, you can send a mail to the
scikit-learn mailing list. Many developpers there use IPython for
their work, and many of them also happen to work near Paris.

We're also having an international sprint the next week (from the 22nd
to the 28th), and IPython developpers are more than welcome to come
and join us for drinks.
I'm not yet sure where and when we will go for drinks, but I'll post
the news on the http://afpy.ro website.

On 10 July 2013 08:02, Fernando Perez <fperez.net at gmail.com> wrote:
> Fantastic if you guys have a drinkup, but you shouldn't tie it to 1.0
> if you plan on going drinking in a week :)  It's pretty clear that our
> schedule is going to slip a little bit: realistically, we'll put out
> an alpha or beta before our dev meeting, let it simmer during that
> week for testing/feedback, and return to polish up the release after
> the dev meeting.
>
> So by all means go and drink next week, but the real 1.0 drinkup(s)
> will have to wait just a little longer :)
>
> Cheers,
>
> f
>
> On Tue, Jul 9, 2013 at 2:12 PM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> Hi list,
>>
>> As you all know, soon should be the release of IPython 1.0 which is theoretically due in one week.
>> To celebrate this, I suggest user and devs near Paris to gather for a drink this weeks, or next one.
>>
>> For the dates, If people are available this week-end(13,14), monday(15), or thurstday(18)
>> please select below
>>
>> http://www.doodle.com/8ii6v2nzacaxc5xg
>>
>> Yes I know, I choose only date where I'm free, I can change if necessary
>> but I'm the one who proposed it would be a shame for me not to be available.
>>
>> Depending on wether you prefer wine or beer, I propose the following places.
>>
>> Beer,
>> L'academie de la bi?re  http://goo.gl/maps/NmMd4
>> La gueuse :  http://goo.gl/maps/a0KLL
>>
>> Wine
>> If we are not too many, and depending on the date, I know a quiet wine bar, not too expensive also:
>> Les grains nobles: http://goo.gl/maps/K4aJb
>>
>> I also hope this will inspire the same things around the world.
>>
>> --
>> Matthias
>> P.S.: Unlike github's drinkups you have to pay your own drink.
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Wed Jul 10 11:53:23 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 08:53:23 -0700
Subject: [IPython-dev] Notebook: File Browser
In-Reply-To: <26F0167B-59E4-4F2C-8C2A-5D9B4AA4E4AF@gmail.com>
References: <CAP-uhDfyAAy7MNA8uiGXwb30NFsDN-ysTF4B5TyxPXxi6t=BiQ@mail.gmail.com>
	<CAH4pYpQh1T5LvN4KM+OKbpZ_=C4bWD9k56WcDpYGLvDo5_FTRw@mail.gmail.com>
	<26F0167B-59E4-4F2C-8C2A-5D9B4AA4E4AF@gmail.com>
Message-ID: <E7E9D49D-C33C-4610-8FA5-C789F4D7C214@gmail.com>

Very good point.

Sent from my iPhone

On Jul 9, 2013, at 11:48 PM, Matthias BUSSONNIER <bussonniermatthias at gmail.com> wrote:

> 
> 
> 
> 
> Le 10 juil. 2013 ? 06:08, Brian Granger a ?crit :
> 
>> The biggest change we have in the works is that the URLs for notebooks
>> will start to use paths and notebook names like this:
>> 
>> /notebook/path/to/my/notebook.ipynb
>> 
>> This will make it trivial to write the type of code you are talking
>> about.  But I think we will probably still have an actual GUI in the
>> dashboard for navigation.  We will be talking about this stuff at the
>> dev meeting in two weeks.
> 
> I disagree
> 
> If I'm on my laptop, 
> the server on rackspaces 
> and the kernels on amazon. 
> The magic don't make sense.
> 
> (same if the 3 are on the same machine, but it's more difficult to see if you don't know the internal)
> 
> -- 
> Matt
> 
> 
> 
>> 
>> Cheers,
>> 
>> Brian
>> 
>> On Tue, Jul 9, 2013 at 9:02 PM, Carl Smith <carl.input at gmail.com> wrote:
>>> I noticed that there's still plans to add a GUI file browser to the
>>> dashboard. Can I just ask why we can't have a magic to open a
>>> notebook? The Notebook already handles every file operation, except
>>> opening notebooks, so what does adding a GUI file browser buy anyone?
>>> especially if there's some kind of shell or single cell interface in
>>> the dashboard.
>>> 
>>> It'd be nice if I could call `ls` in the Notebook and have the output
>>> clickable, so each file would be a link. Clicking a directory would cd
>>> to it, clicking a notebook would open it, clicking an image would
>>> render it... something like that. That way user could still click
>>> their way around the file system, just hitting links instead of icons,
>>> and adding icons is easy in a browser, but probably going a bit too
>>> far.
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> 
>> 
>> -- 
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Wed Jul 10 11:54:24 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 08:54:24 -0700
Subject: [IPython-dev] Fwd: [bower] Make it possible to install a particular
	non-semver git repo tag (#433)
References: <bower/bower/issues/433/20725173@github.com>
Message-ID: <94996D21-77A6-4EB2-B621-31DDF497E025@gmail.com>

Good news on the bower front.

Sent from my iPhone

Begin forwarded message:

> From: Andr? Cruz <notifications at github.com>
> Date: July 10, 2013, 12:12:54 AM PDT
> To: bower/bower <bower at noreply.github.com>
> Cc: "Brian E. Granger" <ellisonbg at gmail.com>
> Subject: Re: [bower] Make it possible to install a particular non-semver git repo tag (#433)
> Reply-To: bower/bower <reply+i-13727286-6d72b0e143bddf5926cca3bcd41711900c008694-27600 at reply.github.com>
> 
> In the rewrite, you can now install semvers, tags and commit-hashes.
> Give it a try npm install -g bower-canary. You might need to uninstall previous bower.
> 
> ?
> Reply to this email directly or view it on GitHub.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/666715bf/attachment.html>

From bussonniermatthias at gmail.com  Wed Jul 10 12:00:15 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 10 Jul 2013 18:00:15 +0200
Subject: [IPython-dev] Notebook: File Browser
In-Reply-To: <E7E9D49D-C33C-4610-8FA5-C789F4D7C214@gmail.com>
References: <CAP-uhDfyAAy7MNA8uiGXwb30NFsDN-ysTF4B5TyxPXxi6t=BiQ@mail.gmail.com>
	<CAH4pYpQh1T5LvN4KM+OKbpZ_=C4bWD9k56WcDpYGLvDo5_FTRw@mail.gmail.com>
	<26F0167B-59E4-4F2C-8C2A-5D9B4AA4E4AF@gmail.com>
	<E7E9D49D-C33C-4610-8FA5-C789F4D7C214@gmail.com>
Message-ID: <2BE89D71-9135-4DC3-A16F-F9E70C581F5E@gmail.com>


Le 10 juil. 2013 ? 17:53, Brian Granger a ?crit :

> Very good point.

As always :-)

That being said, I wish someone will write a cl-tool capable of dialoging with notebook, so you can actually
upload/start kernels from the shell, eventually opening the correct url in your browser? kind of urwid dashboard.

So you could actually do what the OP asked, but the kernel need to know of server URL and credential.

-- 
M


> 
> Sent from my iPhone
> 
> On Jul 9, 2013, at 11:48 PM, Matthias BUSSONNIER <bussonniermatthias at gmail.com> wrote:
> 
>> 
>> 
>> 
>> 
>> Le 10 juil. 2013 ? 06:08, Brian Granger a ?crit :
>> 
>>> The biggest change we have in the works is that the URLs for notebooks
>>> will start to use paths and notebook names like this:
>>> 
>>> /notebook/path/to/my/notebook.ipynb
>>> 
>>> This will make it trivial to write the type of code you are talking
>>> about.  But I think we will probably still have an actual GUI in the
>>> dashboard for navigation.  We will be talking about this stuff at the
>>> dev meeting in two weeks.
>> 
>> I disagree
>> 
>> If I'm on my laptop, 
>> the server on rackspaces 
>> and the kernels on amazon. 
>> The magic don't make sense.
>> 
>> (same if the 3 are on the same machine, but it's more difficult to see if you don't know the internal)
>> 
>> -- 
>> Matt
>> 
>> 
>> 
>>> 
>>> Cheers,
>>> 
>>> Brian
>>> 
>>> On Tue, Jul 9, 2013 at 9:02 PM, Carl Smith <carl.input at gmail.com> wrote:
>>>> I noticed that there's still plans to add a GUI file browser to the
>>>> dashboard. Can I just ask why we can't have a magic to open a
>>>> notebook? The Notebook already handles every file operation, except
>>>> opening notebooks, so what does adding a GUI file browser buy anyone?
>>>> especially if there's some kind of shell or single cell interface in
>>>> the dashboard.
>>>> 
>>>> It'd be nice if I could call `ls` in the Notebook and have the output
>>>> clickable, so each file would be a link. Clicking a directory would cd
>>>> to it, clicking a notebook would open it, clicking an image would
>>>> render it... something like that. That way user could still click
>>>> their way around the file system, just hitting links instead of icons,
>>>> and adding icons is easy in a browser, but probably going a bit too
>>>> far.
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> 
>>> 
>>> -- 
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From bussonniermatthias at gmail.com  Wed Jul 10 12:08:56 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 10 Jul 2013 18:08:56 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
Message-ID: <463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>


Le 10 juil. 2013 ? 17:43, Nelle Varoquaux a ?crit :

> If you want more people to join, you can send a mail to the
> scikit-learn mailing list. Many developpers there use IPython for
> their work, and many of them also happen to work near Paris.

Please forward the mail to scikit-learn if you think it's a good idea. 
I think it's better if it comes from a regular user which is already on the list.

> We're also having an international sprint the next week (from the 22nd
> to the 28th), and IPython developpers are more than welcome to come
> and join us for drinks.
> I'm not yet sure where and when we will go for drinks, but I'll post
> the news on the http://afpy.ro website.


Great. Added to my rss feed.

> Le 10 juil. 2013 ? 08:02, Fernando Perez a ?crit :


> Fantastic if you guys have a drinkup, but you shouldn't tie it to 1.0
> if you plan on going drinking in a week :)  It's pretty clear that our
> schedule is going to slip a little bit: realistically, we'll put out
> an alpha or beta before our dev meeting, let it simmer during that
> week for testing/feedback, and return to polish up the release after
> the dev meeting.
> 
> So by all means go and drink next week, but the real 1.0 drinkup(s)
> will have to wait just a little longer :)
> 

Will you include Drinkup in your next grant ?

-- 
Matthias

From ellisonbg at gmail.com  Wed Jul 10 12:12:35 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 09:12:35 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
Message-ID: <CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>

On Tue, Jul 9, 2013 at 10:46 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> One problem with prefixing all pages is that those prefixes, if I
> understand how GH wikis work correctly, have to be part of the visible
> page title.  Which for some pages makes for pretty horrible names.
> For example the gallery title is perfectly fine as
>
> A gallery of interesting IPython Notebooks
>
> but it would look pretty ugly if called
>
> Cookbook: A gallery of interesting IPython Notebook

The gallery is one of the few things that is important enough to be
top-level in my opinion.  If the Gallery grows beyond a single page, I
would then create a "Gallery: " namespace.

> or similar, IMO.
>
> Since there's no actual auto-indexing machinery or other organizing
> mechanism that these pseudo-namespace prefixes provide, I'm also not
> convinced that it makes sense to mandate their use rigidly everywhere.
>  For certain categories I think they do make sense, like say cookbook
> recipes or sprints lists. But I think we're inevitably bound to have
> 'miscellaneous' pages that don't neatly fit in any single label.  And
> creating a  zillion prefixes just so everything can have its own
> prefix doesn't seem to be such a good idea either.

I agree we will absolutely have misc pages that don't fit well in a
prefix.  But the lab meetings page is not one of those (it clearly
fits into the Dev: category).

There are a couple of reasons I created the prefixes:

1) In our previous wiki, there was little to no organization.  Because
of this, people who wanted to add new content couldn't figure see
which pages might already have content of that type and they would
always add new pages.  This meant that we had overlapping information
on multiple pages - and all of it grew stale.  It made the wiki
entirely useless.
2) When you view a list of all our wiki pages:

https://github.com/ipython/ipython/wiki/_pages

the prefixes keep pages grouped together.  We do this type of named
based grouping all the time in our local file systems (to
organize/sort a sequence of notebooks for a tutorial for example).  I
don't see why wiki pages would be any different.  Is it the way I
choose to do the prefixes? ("Foo: ")
2) Because there is no better way to group wiki pages together on GitHub :(

> In summary, I think the problem is that wikis are inherently prone to
> some of this slightly chaotic growth, which is why I've never liked
> them too much.  I think they fill a useful role when used in a pretty
> *limited* fashion, partly so that they can be manually tended to.  But
> for larger-scale efforts, without a massive metadata machinery and an
> army of people like what Wikipedia uses, it's kind of hopeless. For
> wikipedia, pages have very rich categorical/tagging tools that the
> entire wiki system understands, mines, and uses to offer management
> and navigation.  We don't have that at GH, just a plain wiki in its
> barest form.
>
> Sorry to be a bit of a downer, but I'm just not convinced that more
> prefixing is really going to really solve anything beyond a thin
> cosmetic veneer issue.

But prefixing is only 1/2 the point.  The other half is the prefix
specific index pages:

https://github.com/ipython/ipython/wiki/Dev:-Index
https://github.com/ipython/ipython/wiki/IPEPs:-IPython-Enhancement-Proposals
https://github.com/ipython/ipython/wiki/Cookbook%3A-Index
etc.

Are you proposing that we abandon this organization and instead put
every wiki page on the home page as you have done with the lab
meetings page?

Cheers,

Brian


> Cheers,
>
> f
>
> ps - I'm pretty sure, BTW, that I'm the guilty party in having created
> the lab meetings page without a prefix, sorry :)  Part of it is
> probably that I find it so ugly to read those titles that I
> unconsciously just ignored the idea... Feel free to change it if you
> really think it will help.
>
> On Tue, Jul 9, 2013 at 5:07 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> Hi,
>>
>> A while back I spend a couple of days organizing the IPython GitHub
>> wiki.  I know this will probably seem anal, but I think it is
>> important that we use a logical structure for each of our pages and
>> the wiki as a whole.  To induce this structure on the flat GitHub wiki
>> namespace, we are naming our wiki pages with a prefix such as "Dev :"
>> or "Cookbook: " or "Install: ".  This structure is also present in our
>> sub-indices - we have a separate index page for each of these
>> categories.  When someone created the wiki page for the lab meetings,
>> it was created at the top level, rather than under the "Dev: "
>> category and put in the top level index rather than the "Dev: " index.
>>  I could fix this myself in a few minutes, but I don't want to become
>> the "police" for our wiki organization.  Instead, I would like
>> everyone to help keep the wiki organized.  Can someone clean this up?
>> If we don't want to use this structure, but instead keep things
>> organized another way, I am completely open to that.  I just want the
>> wiki organization to decay into chaos like it did before...
>>
>> Cheers,
>>
>> Brian
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Jul 10 12:16:43 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 09:16:43 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <-1670235088980259958@unknownmsgid>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<-1670235088980259958@unknownmsgid>
Message-ID: <CAH4pYpTRGx6RxtOAu7t13NO7VY-K6R7yYR_XKO9KO3WXR0pN8g@mail.gmail.com>

Aaron,

On Wed, Jul 10, 2013 at 7:41 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
> Embrace the chaos.
>
> You don't use naming conventions for your branches in git. That's because
> branches are cheap and throwaway, and it would be a waste of time to
> organize them. Wikis are the same way.

Our wiki pages are not throwaway.  Our *official* development
resources are there:

* IPEPs
* Our official github workflow, description of how we use issues,
coding style guide, etc.
* Our roadmap

All of this is highly curates and written by/maintained core
developers.  Some of the content is less curated (our Cookbook).

> Take a look at the SymPy wiki pages
> https://github.com/sympy/sympy/wiki/_pages. The only official naming
> convention we have is for GSoC pages. We use the wiki for throwaway tests,
> developer guides, user written pages about uses of SymPy, little bits of
> "documentation" that don't belong in our docs like comparisons to other
> computer algebra systems, and prototypes and write ups for new features.

Most of the active parts of our wiki are not user written.  They are
written by core devs.  Our choice was to put all of our development
information on GitHub rather than Sphinx - to keep it all in the same
place.

> Our organization comes from a listing of the important pages on the main
> page. That only accounts for about half the wiki pages. For the rest, there
> is the list of all pages, which is what I usually use. If you want a search,
> clone the git and use git grep.

That is exactly (in part) what I have done - we have separate index
pages for different categories.

> You probably won't embrace the chaos, so my recommendation is to create an
> index page (or just use the main page) and organize the pages by there by a
> list of links under headers. That's much less painful than page naming
> conventions.

We have already done that....:)

Cheers,

Brian

> Aaron Meurer
>
> Sent from my iPad
>
> On Jul 10, 2013, at 12:47 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> One problem with prefixing all pages is that those prefixes, if I
> understand how GH wikis work correctly, have to be part of the visible
> page title.  Which for some pages makes for pretty horrible names.
> For example the gallery title is perfectly fine as
>
> A gallery of interesting IPython Notebooks
>
> but it would look pretty ugly if called
>
> Cookbook: A gallery of interesting IPython Notebooks
>
> or similar, IMO.
>
> Since there's no actual auto-indexing machinery or other organizing
> mechanism that these pseudo-namespace prefixes provide, I'm also not
> convinced that it makes sense to mandate their use rigidly everywhere.
> For certain categories I think they do make sense, like say cookbook
> recipes or sprints lists. But I think we're inevitably bound to have
> 'miscellaneous' pages that don't neatly fit in any single label.  And
> creating a  zillion prefixes just so everything can have its own
> prefix doesn't seem to be such a good idea either.
>
> In summary, I think the problem is that wikis are inherently prone to
> some of this slightly chaotic growth, which is why I've never liked
> them too much.  I think they fill a useful role when used in a pretty
> *limited* fashion, partly so that they can be manually tended to.  But
> for larger-scale efforts, without a massive metadata machinery and an
> army of people like what Wikipedia uses, it's kind of hopeless. For
> wikipedia, pages have very rich categorical/tagging tools that the
> entire wiki system understands, mines, and uses to offer management
> and navigation.  We don't have that at GH, just a plain wiki in its
> barest form.
>
> Sorry to be a bit of a downer, but I'm just not convinced that more
> prefixing is really going to really solve anything beyond a thin
> cosmetic veneer issue.
>
> Cheers,
>
> f
>
> ps - I'm pretty sure, BTW, that I'm the guilty party in having created
> the lab meetings page without a prefix, sorry :)  Part of it is
> probably that I find it so ugly to read those titles that I
> unconsciously just ignored the idea... Feel free to change it if you
> really think it will help.
>
> On Tue, Jul 9, 2013 at 5:07 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> Hi,
>
>
> A while back I spend a couple of days organizing the IPython GitHub
>
> wiki.  I know this will probably seem anal, but I think it is
>
> important that we use a logical structure for each of our pages and
>
> the wiki as a whole.  To induce this structure on the flat GitHub wiki
>
> namespace, we are naming our wiki pages with a prefix such as "Dev :"
>
> or "Cookbook: " or "Install: ".  This structure is also present in our
>
> sub-indices - we have a separate index page for each of these
>
> categories.  When someone created the wiki page for the lab meetings,
>
> it was created at the top level, rather than under the "Dev: "
>
> category and put in the top level index rather than the "Dev: " index.
>
> I could fix this myself in a few minutes, but I don't want to become
>
> the "police" for our wiki organization.  Instead, I would like
>
> everyone to help keep the wiki organized.  Can someone clean this up?
>
> If we don't want to use this structure, but instead keep things
>
> organized another way, I am completely open to that.  I just want the
>
> wiki organization to decay into chaos like it did before...
>
>
> Cheers,
>
>
> Brian
>
>
> --
>
> Brian E. Granger
>
> Cal Poly State University, San Luis Obispo
>
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
>
> IPython-dev mailing list
>
> IPython-dev at scipy.org
>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Jul 10 12:23:11 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 09:23:11 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
Message-ID: <CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>

Nice idea - some of the branching could be built using git...

On Wed, Jul 10, 2013 at 1:40 AM, St?fan van der Walt <stefan at sun.ac.za> wrote:
>
> On 10 Jul 2013 04:33, "Brian Granger" <ellisonbg at gmail.com> wrote:
>> It would be nice to write a long notebook and then add metadata to the
>> notebook that indicates that some variables are to be treated as
>> "templated" variables.  Then we would create tools that would enable a
>> user to run a notebook over a range of templates:
>>
>> for x in xvars:
>>   for y in yvars:
>>     for algo in myalgos
>>     run_notebook('MyCoolCode', x, y, algo)
>
> I think it would be feasible to build git-like cell revision tracking on top
> of the linear notebook, especially once widgets are in place. Imagine a
> per-cell and per-notebook check-in that allows you to swap out different
> versions of a cell, or different notebook configurations. This can be done
> with a very lightweight abstraction that does not interfere with the
> underlying notebook structure at all (in fact, I'd just use git as the
> actual backend, and then have a layer on top that knows how to pick out
> tagged cells from a nb).
>
> St?fan
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fereyes at lbl.gov  Wed Jul 10 12:24:35 2013
From: fereyes at lbl.gov (Francis Reyes)
Date: Wed, 10 Jul 2013 09:24:35 -0700
Subject: [IPython-dev] =?windows-1252?q?Storing_data_=85?=
Message-ID: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>

Hi

I'd like to associate data generated within the notebook with the actual notebook for that session.  

Right now I'm writing files to the file system, but that places the additional burden of having the manage the data storage. 

[1] How are people dealing with the task of storing data (which may be specific to that ipython notebook, or even better, specific to the notebook's session). 

[2] Is there something (existing or planned) in the API that allows me to store session level data in the ipynb? 

Thanks,

F

Francis Reyes
Advanced Light Source
1 Cyclotron Road
Berkeley, CA 94720





From ellisonbg at gmail.com  Wed Jul 10 12:45:00 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 09:45:00 -0700
Subject: [IPython-dev] =?windows-1252?q?Storing_data_=85?=
In-Reply-To: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>
References: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>
Message-ID: <CAH4pYpR1Dd+WMCRPU8c1Z_eCwFArrHuPSXgW5t+tEiEGrCUijQ@mail.gmail.com>

On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:
> Hi
>
> I'd like to associate data generated within the notebook with the actual notebook for that session.
>
> Right now I'm writing files to the file system, but that places the additional burden of having the manage the data storage.
>
> [1] How are people dealing with the task of storing data (which may be specific to that ipython notebook, or even better, specific to the notebook's session).

For the most part we are encouraging users to store notebook+data in
git repos.  Trying to bundle data inside a notebook would pull the
notebook format in a direction we don't want to go - plus git is
perfect for this as long as the data is not too big (if it is big see
git-annex).  With that said, MIn had a nice hack that stored data in
the notebook document.  Min, do you remember where this is?

But part of our philosophy is that pretty much no code, data or
notebooks should ever live outside a version controlled directory.  We
are pretty fanatical about this.  When we think about the notebook
promoting reproducible and sharable science, version control is
critically important.  Once you accept this perspective, it becomes
unthinkable for a notebook to ever be found outside its repo where its
data lives.  Part of the reason we are fanatical about this is that
the second you start to imagine other ways of bundling other assets
with notebooks, you are heading down a path of 1) reinventing version
control systemsor 2) you are doing silly things like zipping up all of
the files into a impenetrable binary blob.

> [2] Is there something (existing or planned) in the API that allows me to store session level data in the ipynb?

I don't think we will go in this direction.

Cheers,

Brian

> Thanks,
>
> F
>
> Francis Reyes
> Advanced Light Source
> 1 Cyclotron Road
> Berkeley, CA 94720
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From asmeurer at gmail.com  Wed Jul 10 13:03:39 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Wed, 10 Jul 2013 12:03:39 -0500
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAH4pYpTRGx6RxtOAu7t13NO7VY-K6R7yYR_XKO9KO3WXR0pN8g@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<-1670235088980259958@unknownmsgid>
	<CAH4pYpTRGx6RxtOAu7t13NO7VY-K6R7yYR_XKO9KO3WXR0pN8g@mail.gmail.com>
Message-ID: <CAKgW=6LDexe6anVXTh4n3eYan7nytZVz0EqLzBQO6dzGX=nGjQ@mail.gmail.com>

On Wed, Jul 10, 2013 at 11:16 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> Aaron,
>
> On Wed, Jul 10, 2013 at 7:41 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
> > Embrace the chaos.
> >
> > You don't use naming conventions for your branches in git. That's because
> > branches are cheap and throwaway, and it would be a waste of time to
> > organize them. Wikis are the same way.
>
> Our wiki pages are not throwaway.  Our *official* development
> resources are there:
>

Do you allow throwaway wiki pages? For SymPy, we encourage people to dump
interesting things on the wiki.

There are also official things there too, like development guidelines and
gsoc reports. But I guess our workflow is not as officialized as yours (we
don't have peps or anything).


>
> * IPEPs
> * Our official github workflow, description of how we use issues,
> coding style guide, etc.
> * Our roadmap
>
> All of this is highly curates and written by/maintained core
> developers.  Some of the content is less curated (our Cookbook).
>
> > Take a look at the SymPy wiki pages
> > https://github.com/sympy/sympy/wiki/_pages. The only official naming
> > convention we have is for GSoC pages. We use the wiki for throwaway
> tests,
> > developer guides, user written pages about uses of SymPy, little bits of
> > "documentation" that don't belong in our docs like comparisons to other
> > computer algebra systems, and prototypes and write ups for new features.
>
> Most of the active parts of our wiki are not user written.  They are
> written by core devs.  Our choice was to put all of our development
> information on GitHub rather than Sphinx - to keep it all in the same
> place.
>
> > Our organization comes from a listing of the important pages on the main
> > page. That only accounts for about half the wiki pages. For the rest,
> there
> > is the list of all pages, which is what I usually use. If you want a
> search,
> > clone the git and use git grep.
>
> That is exactly (in part) what I have done - we have separate index
> pages for different categories.
>
> > You probably won't embrace the chaos, so my recommendation is to create
> an
> > index page (or just use the main page) and organize the pages by there
> by a
> > list of links under headers. That's much less painful than page naming
> > conventions.
>
> We have already done that....:)
>

So why do you need prefixes?

Aaron Meurer


>
> Cheers,
>
> Brian
>
> > Aaron Meurer
> >
> > Sent from my iPad
> >
> > On Jul 10, 2013, at 12:47 AM, Fernando Perez <fperez.net at gmail.com>
> wrote:
> >
> > One problem with prefixing all pages is that those prefixes, if I
> > understand how GH wikis work correctly, have to be part of the visible
> > page title.  Which for some pages makes for pretty horrible names.
> > For example the gallery title is perfectly fine as
> >
> > A gallery of interesting IPython Notebooks
> >
> > but it would look pretty ugly if called
> >
> > Cookbook: A gallery of interesting IPython Notebooks
> >
> > or similar, IMO.
> >
> > Since there's no actual auto-indexing machinery or other organizing
> > mechanism that these pseudo-namespace prefixes provide, I'm also not
> > convinced that it makes sense to mandate their use rigidly everywhere.
> > For certain categories I think they do make sense, like say cookbook
> > recipes or sprints lists. But I think we're inevitably bound to have
> > 'miscellaneous' pages that don't neatly fit in any single label.  And
> > creating a  zillion prefixes just so everything can have its own
> > prefix doesn't seem to be such a good idea either.
> >
> > In summary, I think the problem is that wikis are inherently prone to
> > some of this slightly chaotic growth, which is why I've never liked
> > them too much.  I think they fill a useful role when used in a pretty
> > *limited* fashion, partly so that they can be manually tended to.  But
> > for larger-scale efforts, without a massive metadata machinery and an
> > army of people like what Wikipedia uses, it's kind of hopeless. For
> > wikipedia, pages have very rich categorical/tagging tools that the
> > entire wiki system understands, mines, and uses to offer management
> > and navigation.  We don't have that at GH, just a plain wiki in its
> > barest form.
> >
> > Sorry to be a bit of a downer, but I'm just not convinced that more
> > prefixing is really going to really solve anything beyond a thin
> > cosmetic veneer issue.
> >
> > Cheers,
> >
> > f
> >
> > ps - I'm pretty sure, BTW, that I'm the guilty party in having created
> > the lab meetings page without a prefix, sorry :)  Part of it is
> > probably that I find it so ugly to read those titles that I
> > unconsciously just ignored the idea... Feel free to change it if you
> > really think it will help.
> >
> > On Tue, Jul 9, 2013 at 5:07 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
> >
> > Hi,
> >
> >
> > A while back I spend a couple of days organizing the IPython GitHub
> >
> > wiki.  I know this will probably seem anal, but I think it is
> >
> > important that we use a logical structure for each of our pages and
> >
> > the wiki as a whole.  To induce this structure on the flat GitHub wiki
> >
> > namespace, we are naming our wiki pages with a prefix such as "Dev :"
> >
> > or "Cookbook: " or "Install: ".  This structure is also present in our
> >
> > sub-indices - we have a separate index page for each of these
> >
> > categories.  When someone created the wiki page for the lab meetings,
> >
> > it was created at the top level, rather than under the "Dev: "
> >
> > category and put in the top level index rather than the "Dev: " index.
> >
> > I could fix this myself in a few minutes, but I don't want to become
> >
> > the "police" for our wiki organization.  Instead, I would like
> >
> > everyone to help keep the wiki organized.  Can someone clean this up?
> >
> > If we don't want to use this structure, but instead keep things
> >
> > organized another way, I am completely open to that.  I just want the
> >
> > wiki organization to decay into chaos like it did before...
> >
> >
> > Cheers,
> >
> >
> > Brian
> >
> >
> > --
> >
> > Brian E. Granger
> >
> > Cal Poly State University, San Luis Obispo
> >
> > bgranger at calpoly.edu and ellisonbg at gmail.com
> >
> > _______________________________________________
> >
> > IPython-dev mailing list
> >
> > IPython-dev at scipy.org
> >
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> >
> > --
> > Fernando Perez (@fperez_org; http://fperez.org)
> > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> > fernando.perez-at-berkeley: contact me here for any direct mail
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/7a68df5f/attachment.html>

From ellisonbg at gmail.com  Wed Jul 10 13:43:41 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 10:43:41 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAKgW=6LDexe6anVXTh4n3eYan7nytZVz0EqLzBQO6dzGX=nGjQ@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<-1670235088980259958@unknownmsgid>
	<CAH4pYpTRGx6RxtOAu7t13NO7VY-K6R7yYR_XKO9KO3WXR0pN8g@mail.gmail.com>
	<CAKgW=6LDexe6anVXTh4n3eYan7nytZVz0EqLzBQO6dzGX=nGjQ@mail.gmail.com>
Message-ID: <CAH4pYpTY8Sw9JjDkLiaAQ5O8rPQmDNwapMTY_afGYkqmCX-Asw@mail.gmail.com>

The prefixes group the pages together when people view the "Pages"
tab.  This helps them to see what content already exists in each
category and where they should add their new content.

On Wed, Jul 10, 2013 at 10:03 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
>
>
>
> On Wed, Jul 10, 2013 at 11:16 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> Aaron,
>>
>> On Wed, Jul 10, 2013 at 7:41 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
>> > Embrace the chaos.
>> >
>> > You don't use naming conventions for your branches in git. That's
>> > because
>> > branches are cheap and throwaway, and it would be a waste of time to
>> > organize them. Wikis are the same way.
>>
>> Our wiki pages are not throwaway.  Our *official* development
>> resources are there:
>
>
> Do you allow throwaway wiki pages? For SymPy, we encourage people to dump
> interesting things on the wiki.
>
> There are also official things there too, like development guidelines and
> gsoc reports. But I guess our workflow is not as officialized as yours (we
> don't have peps or anything).
>
>>
>>
>> * IPEPs
>> * Our official github workflow, description of how we use issues,
>> coding style guide, etc.
>> * Our roadmap
>>
>> All of this is highly curates and written by/maintained core
>> developers.  Some of the content is less curated (our Cookbook).
>>
>> > Take a look at the SymPy wiki pages
>> > https://github.com/sympy/sympy/wiki/_pages. The only official naming
>> > convention we have is for GSoC pages. We use the wiki for throwaway
>> > tests,
>> > developer guides, user written pages about uses of SymPy, little bits of
>> > "documentation" that don't belong in our docs like comparisons to other
>> > computer algebra systems, and prototypes and write ups for new features.
>>
>> Most of the active parts of our wiki are not user written.  They are
>> written by core devs.  Our choice was to put all of our development
>> information on GitHub rather than Sphinx - to keep it all in the same
>> place.
>>
>> > Our organization comes from a listing of the important pages on the main
>> > page. That only accounts for about half the wiki pages. For the rest,
>> > there
>> > is the list of all pages, which is what I usually use. If you want a
>> > search,
>> > clone the git and use git grep.
>>
>> That is exactly (in part) what I have done - we have separate index
>> pages for different categories.
>>
>> > You probably won't embrace the chaos, so my recommendation is to create
>> > an
>> > index page (or just use the main page) and organize the pages by there
>> > by a
>> > list of links under headers. That's much less painful than page naming
>> > conventions.
>>
>> We have already done that....:)
>
>
> So why do you need prefixes?
>
> Aaron Meurer
>
>>
>>
>> Cheers,
>>
>> Brian
>>
>> > Aaron Meurer
>> >
>> > Sent from my iPad
>> >
>> > On Jul 10, 2013, at 12:47 AM, Fernando Perez <fperez.net at gmail.com>
>> > wrote:
>> >
>> > One problem with prefixing all pages is that those prefixes, if I
>> > understand how GH wikis work correctly, have to be part of the visible
>> > page title.  Which for some pages makes for pretty horrible names.
>> > For example the gallery title is perfectly fine as
>> >
>> > A gallery of interesting IPython Notebooks
>> >
>> > but it would look pretty ugly if called
>> >
>> > Cookbook: A gallery of interesting IPython Notebooks
>> >
>> > or similar, IMO.
>> >
>> > Since there's no actual auto-indexing machinery or other organizing
>> > mechanism that these pseudo-namespace prefixes provide, I'm also not
>> > convinced that it makes sense to mandate their use rigidly everywhere.
>> > For certain categories I think they do make sense, like say cookbook
>> > recipes or sprints lists. But I think we're inevitably bound to have
>> > 'miscellaneous' pages that don't neatly fit in any single label.  And
>> > creating a  zillion prefixes just so everything can have its own
>> > prefix doesn't seem to be such a good idea either.
>> >
>> > In summary, I think the problem is that wikis are inherently prone to
>> > some of this slightly chaotic growth, which is why I've never liked
>> > them too much.  I think they fill a useful role when used in a pretty
>> > *limited* fashion, partly so that they can be manually tended to.  But
>> > for larger-scale efforts, without a massive metadata machinery and an
>> > army of people like what Wikipedia uses, it's kind of hopeless. For
>> > wikipedia, pages have very rich categorical/tagging tools that the
>> > entire wiki system understands, mines, and uses to offer management
>> > and navigation.  We don't have that at GH, just a plain wiki in its
>> > barest form.
>> >
>> > Sorry to be a bit of a downer, but I'm just not convinced that more
>> > prefixing is really going to really solve anything beyond a thin
>> > cosmetic veneer issue.
>> >
>> > Cheers,
>> >
>> > f
>> >
>> > ps - I'm pretty sure, BTW, that I'm the guilty party in having created
>> > the lab meetings page without a prefix, sorry :)  Part of it is
>> > probably that I find it so ugly to read those titles that I
>> > unconsciously just ignored the idea... Feel free to change it if you
>> > really think it will help.
>> >
>> > On Tue, Jul 9, 2013 at 5:07 PM, Brian Granger <ellisonbg at gmail.com>
>> > wrote:
>> >
>> > Hi,
>> >
>> >
>> > A while back I spend a couple of days organizing the IPython GitHub
>> >
>> > wiki.  I know this will probably seem anal, but I think it is
>> >
>> > important that we use a logical structure for each of our pages and
>> >
>> > the wiki as a whole.  To induce this structure on the flat GitHub wiki
>> >
>> > namespace, we are naming our wiki pages with a prefix such as "Dev :"
>> >
>> > or "Cookbook: " or "Install: ".  This structure is also present in our
>> >
>> > sub-indices - we have a separate index page for each of these
>> >
>> > categories.  When someone created the wiki page for the lab meetings,
>> >
>> > it was created at the top level, rather than under the "Dev: "
>> >
>> > category and put in the top level index rather than the "Dev: " index.
>> >
>> > I could fix this myself in a few minutes, but I don't want to become
>> >
>> > the "police" for our wiki organization.  Instead, I would like
>> >
>> > everyone to help keep the wiki organized.  Can someone clean this up?
>> >
>> > If we don't want to use this structure, but instead keep things
>> >
>> > organized another way, I am completely open to that.  I just want the
>> >
>> > wiki organization to decay into chaos like it did before...
>> >
>> >
>> > Cheers,
>> >
>> >
>> > Brian
>> >
>> >
>> > --
>> >
>> > Brian E. Granger
>> >
>> > Cal Poly State University, San Luis Obispo
>> >
>> > bgranger at calpoly.edu and ellisonbg at gmail.com
>> >
>> > _______________________________________________
>> >
>> > IPython-dev mailing list
>> >
>> > IPython-dev at scipy.org
>> >
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> >
>> > --
>> > Fernando Perez (@fperez_org; http://fperez.org)
>> > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>> > fernando.perez-at-berkeley: contact me here for any direct mail
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From benjaminrk at gmail.com  Wed Jul 10 14:16:09 2013
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 10 Jul 2013 11:16:09 -0700
Subject: [IPython-dev] =?utf-8?q?Storing_data_=E2=80=A6?=
In-Reply-To: <CAH4pYpR1Dd+WMCRPU8c1Z_eCwFArrHuPSXgW5t+tEiEGrCUijQ@mail.gmail.com>
References: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>
	<CAH4pYpR1Dd+WMCRPU8c1Z_eCwFArrHuPSXgW5t+tEiEGrCUijQ@mail.gmail.com>
Message-ID: <CAHNn8BXH9BDeyKVefOd=Cwuz_RkTtpXspfjT=w5W28OA5eB_vw@mail.gmail.com>

On Wed, Jul 10, 2013 at 9:45 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:
> > Hi
> >
> > I'd like to associate data generated within the notebook with the actual
> notebook for that session.
> >
> > Right now I'm writing files to the file system, but that places the
> additional burden of having the manage the data storage.
> >
> > [1] How are people dealing with the task of storing data (which may be
> specific to that ipython notebook, or even better, specific to the
> notebook's session).
>
> For the most part we are encouraging users to store notebook+data in
> git repos.  Trying to bundle data inside a notebook would pull the
> notebook format in a direction we don't want to go - plus git is
> perfect for this as long as the data is not too big (if it is big see
> git-annex).  With that said, MIn had a nice hack that stored data in
> the notebook document.  Min, do you remember where this is?
>

My guess is you are referring to this one<http://nbviewer.ipython.org/5303304>,
where I made download links to in-memory data (which technically does embed
the data into the notebook document).



>
> But part of our philosophy is that pretty much no code, data or
> notebooks should ever live outside a version controlled directory.  We
> are pretty fanatical about this.  When we think about the notebook
> promoting reproducible and sharable science, version control is
> critically important.  Once you accept this perspective, it becomes
> unthinkable for a notebook to ever be found outside its repo where its
> data lives.  Part of the reason we are fanatical about this is that
> the second you start to imagine other ways of bundling other assets
> with notebooks, you are heading down a path of 1) reinventing version
> control systemsor 2) you are doing silly things like zipping up all of
> the files into a impenetrable binary blob.
>
> > [2] Is there something (existing or planned) in the API that allows me
> to store session level data in the ipynb?
>
> I don't think we will go in this direction.
>
> Cheers,
>
> Brian
>
> > Thanks,
> >
> > F
> >
> > Francis Reyes
> > Advanced Light Source
> > 1 Cyclotron Road
> > Berkeley, CA 94720
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/0fa7907b/attachment.html>

From gmbecker at ucdavis.edu  Wed Jul 10 14:25:54 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Wed, 10 Jul 2013 11:25:54 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
	<CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>
Message-ID: <CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>

Stefan,

First off I love that other people are joining the conversion. The more
ideas are thrown around the better we all will be!

Using git (or version control in general) for branching is an interesting
idea. I had/have/am considering it but for historical comparisons, which I
view as somewhat conceptually different than alternatives based branching.
I.e. I have reimplemented the *same* code, lets run both versions and
inspect the differences (or hopefully lack thereof!). This is partially
branching, but also partially QA.

git is a Good Thing, and seems to be considered the Right Way (tm) of using
IPython notebook according to the devs. I don't disagree with this; code
belongs in version control and notebooks are code with extra bits attached.

That having been said, there is a far cry between git being the place that
the devs think notebooks *should* live and not having notebooks in git
being *unsupported*. I would argue building actual application features on
git brings us pretty firmly into the latter category.

Without wanting to put words in his mouth, Min seems to agree. According to
my reading of the autosave IPEP he balked at basing the autosave feature
explicitly on git based similar concerns. I feel he made the right choice,
but the same reasoning applied in this case would lead to the same result I
think (not basing features explicitly on git).

He is of course perfectly capable of talking for himself however, so I'll
stop (sorry Min!) and simply speak for myself. I think it would be a
mistake to move from strongly encouraging users to use git or other version
control with their notebooks to actually basing major features* of the
application directly on the assumption that the notebook lives in git with
a specific set of tags/branches/whatever that IPython is going to
programmatically interpret in a certain way.

*Note this is different than things like a "check in my changes" button if
a notebook can tell it is version controlled somehow. At least in my
opinion.

~G



On Wed, Jul 10, 2013 at 9:23 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> Nice idea - some of the branching could be built using git...
>
> On Wed, Jul 10, 2013 at 1:40 AM, St?fan van der Walt <stefan at sun.ac.za>
> wrote:
> >
> > On 10 Jul 2013 04:33, "Brian Granger" <ellisonbg at gmail.com> wrote:
> >> It would be nice to write a long notebook and then add metadata to the
> >> notebook that indicates that some variables are to be treated as
> >> "templated" variables.  Then we would create tools that would enable a
> >> user to run a notebook over a range of templates:
> >>
> >> for x in xvars:
> >>   for y in yvars:
> >>     for algo in myalgos
> >>     run_notebook('MyCoolCode', x, y, algo)
> >
> > I think it would be feasible to build git-like cell revision tracking on
> top
> > of the linear notebook, especially once widgets are in place. Imagine a
> > per-cell and per-notebook check-in that allows you to swap out different
> > versions of a cell, or different notebook configurations. This can be
> done
> > with a very lightweight abstraction that does not interfere with the
> > underlying notebook structure at all (in fact, I'd just use git as the
> > actual backend, and then have a layer on top that knows how to pick out
> > tagged cells from a nb).
> >
> > St?fan
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/767a0929/attachment.html>

From stefan at sun.ac.za  Wed Jul 10 14:40:57 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 10 Jul 2013 20:40:57 +0200
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
	<CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>
	<CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>
Message-ID: <CABDkGQ=F1W25zOMRta=_9KaHxnyWha03VfG1FhvfaL3WJspR3g@mail.gmail.com>

On Wed, Jul 10, 2013 at 8:25 PM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
>
> That having been said, there is a far cry between git being the place that the devs think notebooks *should* live and not having notebooks in git being unsupported. I would argue building actual application features on git brings us pretty firmly into the latter category.

The approach I suggested depends only on one IPython feature: per-cell
meta-data.  I don't think that will go away any time soon.  The
solution is bolted on top, so it does not rely on any support inside
of IPython.  Also, you can then choose which configurations to run and
generate your output.  In the end, you'll still have to compare the
output somewhat manually (even though we can potentially track the
outputs via version control too, making things easier).

St?fan


From ellisonbg at gmail.com  Wed Jul 10 14:48:09 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 11:48:09 -0700
Subject: [IPython-dev] =?windows-1252?q?Storing_data_=85?=
In-Reply-To: <CAHNn8BXH9BDeyKVefOd=Cwuz_RkTtpXspfjT=w5W28OA5eB_vw@mail.gmail.com>
References: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>
	<CAH4pYpR1Dd+WMCRPU8c1Z_eCwFArrHuPSXgW5t+tEiEGrCUijQ@mail.gmail.com>
	<CAHNn8BXH9BDeyKVefOd=Cwuz_RkTtpXspfjT=w5W28OA5eB_vw@mail.gmail.com>
Message-ID: <CAH4pYpQmUfAtz3K2nB8sr0xx_4X__jok0r2=WeD1=XcO7bG16A@mail.gmail.com>

Ahh, yes thanks Min!

On Wed, Jul 10, 2013 at 11:16 AM, MinRK <benjaminrk at gmail.com> wrote:
>
>
>
> On Wed, Jul 10, 2013 at 9:45 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:
>> > Hi
>> >
>> > I'd like to associate data generated within the notebook with the actual
>> > notebook for that session.
>> >
>> > Right now I'm writing files to the file system, but that places the
>> > additional burden of having the manage the data storage.
>> >
>> > [1] How are people dealing with the task of storing data (which may be
>> > specific to that ipython notebook, or even better, specific to the
>> > notebook's session).
>>
>> For the most part we are encouraging users to store notebook+data in
>> git repos.  Trying to bundle data inside a notebook would pull the
>> notebook format in a direction we don't want to go - plus git is
>> perfect for this as long as the data is not too big (if it is big see
>> git-annex).  With that said, MIn had a nice hack that stored data in
>> the notebook document.  Min, do you remember where this is?
>
>
> My guess is you are referring to this one, where I made download links to
> in-memory data (which technically does embed the data into the notebook
> document).
>
>
>>
>>
>> But part of our philosophy is that pretty much no code, data or
>> notebooks should ever live outside a version controlled directory.  We
>> are pretty fanatical about this.  When we think about the notebook
>> promoting reproducible and sharable science, version control is
>> critically important.  Once you accept this perspective, it becomes
>> unthinkable for a notebook to ever be found outside its repo where its
>> data lives.  Part of the reason we are fanatical about this is that
>> the second you start to imagine other ways of bundling other assets
>> with notebooks, you are heading down a path of 1) reinventing version
>> control systemsor 2) you are doing silly things like zipping up all of
>> the files into a impenetrable binary blob.
>>
>> > [2] Is there something (existing or planned) in the API that allows me
>> > to store session level data in the ipynb?
>>
>> I don't think we will go in this direction.
>>
>> Cheers,
>>
>> Brian
>>
>> > Thanks,
>> >
>> > F
>> >
>> > Francis Reyes
>> > Advanced Light Source
>> > 1 Cyclotron Road
>> > Berkeley, CA 94720
>> >
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Jul 10 14:55:17 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 11:55:17 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
	<CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>
	<CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>
Message-ID: <CAH4pYpQiXuzMNH9LTkRzEKjXka=5Rd7iCXdaqa0HwCVm-RucOw@mail.gmail.com>

Gabriel,

On Wed, Jul 10, 2013 at 11:25 AM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
> Stefan,
>
> First off I love that other people are joining the conversion. The more
> ideas are thrown around the better we all will be!
>
> Using git (or version control in general) for branching is an interesting
> idea. I had/have/am considering it but for historical comparisons, which I
> view as somewhat conceptually different than alternatives based branching.
> I.e. I have reimplemented the *same* code, lets run both versions and
> inspect the differences (or hopefully lack thereof!). This is partially
> branching, but also partially QA.
>
> git is a Good Thing, and seems to be considered the Right Way (tm) of using
> IPython notebook according to the devs. I don't disagree with this; code
> belongs in version control and notebooks are code with extra bits attached.
>
> That having been said, there is a far cry between git being the place that
> the devs think notebooks *should* live and not having notebooks in git being
> unsupported. I would argue building actual application features on git
> brings us pretty firmly into the latter category.

I don't think the notebook will ever require git, but we have plans on
adding in depth integration with git.

> Without wanting to put words in his mouth, Min seems to agree. According to
> my reading of the autosave IPEP he balked at basing the autosave feature
> explicitly on git based similar concerns. I feel he made the right choice,
> but the same reasoning applied in this case would lead to the same result I
> think (not basing features explicitly on git).

While git support was not in the initial autosave work, it was
designed with git in mind.  The idea is that git could be plugged in
to track the checkpoints rather than our .ipynb_checkpoints directory.
 We haven't figured out how checkpoints would exactly be mapped onto
git concepts (is a checkpoint a commit?), but this is really important
to us and will be happening.  But there will come a day when clicking
"Save and checkpoint" in the notebook will invoke git if the notebook
is in a git repo...

Cheers,

Brian

> He is of course perfectly capable of talking for himself however, so I'll
> stop (sorry Min!) and simply speak for myself. I think it would be a mistake
> to move from strongly encouraging users to use git or other version control
> with their notebooks to actually basing major features* of the application
> directly on the assumption that the notebook lives in git with a specific
> set of tags/branches/whatever that IPython is going to programmatically
> interpret in a certain way.
>
> *Note this is different than things like a "check in my changes" button if a
> notebook can tell it is version controlled somehow. At least in my opinion.
>
> ~G
>
>
>
> On Wed, Jul 10, 2013 at 9:23 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> Nice idea - some of the branching could be built using git...
>>
>> On Wed, Jul 10, 2013 at 1:40 AM, St?fan van der Walt <stefan at sun.ac.za>
>> wrote:
>> >
>> > On 10 Jul 2013 04:33, "Brian Granger" <ellisonbg at gmail.com> wrote:
>> >> It would be nice to write a long notebook and then add metadata to the
>> >> notebook that indicates that some variables are to be treated as
>> >> "templated" variables.  Then we would create tools that would enable a
>> >> user to run a notebook over a range of templates:
>> >>
>> >> for x in xvars:
>> >>   for y in yvars:
>> >>     for algo in myalgos
>> >>     run_notebook('MyCoolCode', x, y, algo)
>> >
>> > I think it would be feasible to build git-like cell revision tracking on
>> > top
>> > of the linear notebook, especially once widgets are in place. Imagine a
>> > per-cell and per-notebook check-in that allows you to swap out different
>> > versions of a cell, or different notebook configurations. This can be
>> > done
>> > with a very lightweight abstraction that does not interfere with the
>> > underlying notebook structure at all (in fact, I'd just use git as the
>> > actual backend, and then have a layer on top that knows how to pick out
>> > tagged cells from a nb).
>> >
>> > St?fan
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From sychan at lbl.gov  Wed Jul 10 15:37:56 2013
From: sychan at lbl.gov (Stephen Chan)
Date: Wed, 10 Jul 2013 12:37:56 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
Message-ID: <CA+n9Yfqi0DB+462MiVpJKSOZ=bYjVw3XBLtwu842yqdOU18wkA@mail.gmail.com>

On Tue, Jul 9, 2013 at 7:32 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> It would be nice to write a long notebook and then add metadata to the
> notebook that indicates that some variables are to be treated as
> "templated" variables.  Then we would create tools that would enable a
> user to run a notebook over a range of templates:
>
> for x in xvars:
>   for y in yvars:
>     for algo in myalgos
>     run_notebook('MyCoolCode', x, y, algo)
>
> The result would be **something** that allows the user to explore the
> parameter space represented.  A single notebook would be used as the
> "source" for this analysis and the result would be the set of all

    I think this would be really useful, and it would really promote
the use of notebooks as re-usable "recipes" to provide usable code for
non-code savvy users. Its not hard to imagine a tool that examines the
metadata of a notebook and brings up a form (in whatever the current
presentation environment may be) for a non-coder to re-use the code
developed by someone else.
    You might also want to be able to grab the results of the final
output cell and make that available as input to other notebooks.

    Steve


From damianavila at gmail.com  Wed Jul 10 15:47:39 2013
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Wed, 10 Jul 2013 16:47:39 -0300
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CABDkGQ=F1W25zOMRta=_9KaHxnyWha03VfG1FhvfaL3WJspR3g@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
	<CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>
	<CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>
	<CABDkGQ=F1W25zOMRta=_9KaHxnyWha03VfG1FhvfaL3WJspR3g@mail.gmail.com>
Message-ID: <51DDBA5B.6040306@gmail.com>

El 10/07/13 15:40, St?fan van der Walt escribi?:
> The approach I suggested depends only on one IPython feature: per-cell
> meta-data.  I don't think that will go away any time soon.

Just a note... per-cell metadata is available in the current master of 
IPython.
You can access to the per-cell metadata through the new CellToolBar menu.
For example, I use the metadata in each cell to mark them as different 
kind of elements to be rendered (later) by the reveal slideshow exporter.

Dami?n.





From fperez.net at gmail.com  Wed Jul 10 18:15:05 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 10 Jul 2013 15:15:05 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
Message-ID: <CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>

On Wed, Jul 10, 2013 at 9:12 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> I agree we will absolutely have misc pages that don't fit well in a
> prefix.  But the lab meetings page is not one of those (it clearly
> fits into the Dev: category).

Yup, sorry about that one, as I said, I'm pretty sure I did that :)

> There are a couple of reasons I created the prefixes:
>
> 1) In our previous wiki, there was little to no organization.  Because
> of this, people who wanted to add new content couldn't figure see
> which pages might already have content of that type and they would
> always add new pages.  This meant that we had overlapping information
> on multiple pages - and all of it grew stale.  It made the wiki
> entirely useless.
> 2) When you view a list of all our wiki pages:
>
> https://github.com/ipython/ipython/wiki/_pages
>
> the prefixes keep pages grouped together.  We do this type of named
> based grouping all the time in our local file systems (to
> organize/sort a sequence of notebooks for a tutorial for example).  I
> don't see why wiki pages would be any different.  Is it the way I
> choose to do the prefixes? ("Foo: ")
> 2) Because there is no better way to group wiki pages together on GitHub :(

This #2 is really the main issue IMO.  I personally would much prefer
not having this kind of prefixing in names and having categorical
metadata (labels) along with a smarter wiki that would auto-create
indices.  The problem is that now, any time we make/rename/delete
pages, we also have to manually remember to go to the -Index pages and
update things.  I hate that.

But having said that, I'm pretty sure that I'm less bothered by having
prefixes than you are with not having them, therefore I'm all for a
workflow that makes you happy :)  As you mentioned in the JS doc
discussion, we really need to find practices that make our everyday
activities as smooth and pleasant as possible, given the constraints
available (in this case, the rather primitive GH wiki engine).

So I'm OK with having prefixes, and will be more careful in the future
as I make pages, I promise :)  I do worry that the index pages, being
manually maintained, will be perennially incorrect and outdated, but
I'm not sure what we can do about that.

As a show of good faith, I've just renamed the lab meeting page :)

https://github.com/ipython/ipython/wiki/Dev:-Lab-meetings-on-Air

Cheers,

f


From fperez.net at gmail.com  Wed Jul 10 18:16:19 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 10 Jul 2013 15:16:19 -0700
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
	<463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
Message-ID: <CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>

On Wed, Jul 10, 2013 at 9:08 AM, Matthias BUSSONNIER

> Will you include Drinkup in your next grant ?

I'm not sure that can be a line-item in a grant application, I'm afraid ;)


From carl.input at gmail.com  Wed Jul 10 18:29:56 2013
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 10 Jul 2013 23:29:56 +0100
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
	<463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
	<CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>
Message-ID: <CAP-uhDfbu-7vz9UjCqdm2C6fAhjRHW1R4WxfAzTeSJg1jupU2Q@mail.gmail.com>

Community building??

On 10 July 2013 23:16, Fernando Perez <fperez.net at gmail.com> wrote:
> On Wed, Jul 10, 2013 at 9:08 AM, Matthias BUSSONNIER
>
>> Will you include Drinkup in your next grant ?
>
> I'm not sure that can be a line-item in a grant application, I'm afraid ;)
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From asmeurer at gmail.com  Wed Jul 10 18:34:17 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Wed, 10 Jul 2013 17:34:17 -0500
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
Message-ID: <CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>

On Wed, Jul 10, 2013 at 5:15 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Wed, Jul 10, 2013 at 9:12 AM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>
> > I agree we will absolutely have misc pages that don't fit well in a
> > prefix.  But the lab meetings page is not one of those (it clearly
> > fits into the Dev: category).
>
> Yup, sorry about that one, as I said, I'm pretty sure I did that :)
>
> > There are a couple of reasons I created the prefixes:
> >
> > 1) In our previous wiki, there was little to no organization.  Because
> > of this, people who wanted to add new content couldn't figure see
> > which pages might already have content of that type and they would
> > always add new pages.  This meant that we had overlapping information
> > on multiple pages - and all of it grew stale.  It made the wiki
> > entirely useless.
> > 2) When you view a list of all our wiki pages:
> >
> > https://github.com/ipython/ipython/wiki/_pages
> >
> > the prefixes keep pages grouped together.  We do this type of named
> > based grouping all the time in our local file systems (to
> > organize/sort a sequence of notebooks for a tutorial for example).  I
> > don't see why wiki pages would be any different.  Is it the way I
> > choose to do the prefixes? ("Foo: ")
> > 2) Because there is no better way to group wiki pages together on GitHub
> :(
>
> This #2 is really the main issue IMO.  I personally would much prefer
> not having this kind of prefixing in names and having categorical
> metadata (labels) along with a smarter wiki that would auto-create
> indices.  The problem is that now, any time we make/rename/delete
> pages, we also have to manually remember to go to the -Index pages and
> update things.  I hate that.
>
> But having said that, I'm pretty sure that I'm less bothered by having
> prefixes than you are with not having them, therefore I'm all for a
> workflow that makes you happy :)  As you mentioned in the JS doc
> discussion, we really need to find practices that make our everyday
> activities as smooth and pleasant as possible, given the constraints
> available (in this case, the rather primitive GH wiki engine).
>
> So I'm OK with having prefixes, and will be more careful in the future
> as I make pages, I promise :)  I do worry that the index pages, being
> manually maintained, will be perennially incorrect and outdated, but
> I'm not sure what we can do about that.
>

Write a little script that makes it easier to check and update these things
(remember that the wiki is available in git).  I would find this very
useful for the SymPy wiki as well.

Aaron Meurer


> As a show of good faith, I've just renamed the lab meeting page :)
>
> https://github.com/ipython/ipython/wiki/Dev:-Lab-meetings-on-Air
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/8d18dc65/attachment.html>

From fperez.net at gmail.com  Wed Jul 10 19:36:13 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 10 Jul 2013 16:36:13 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
	<CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
Message-ID: <CAHAreOrUweVPMnheiGOPDv_KYPnXzAYJgEqa1kPr5QXhjOjghA@mail.gmail.com>

On Wed, Jul 10, 2013 at 3:34 PM, Aaron Meurer <asmeurer at gmail.com> wrote:

> Write a little script that makes it easier to check and update these things
> (remember that the wiki is available in git).  I would find this very useful
> for the SymPy wiki as well.

I don't have the time for this right now, but that's a good point I
keep forgetting: with the wiki being available as a git repo, it's not
so hard to build additional local support tools for it.

Cheers,

f


From damianavila at gmail.com  Wed Jul 10 22:10:13 2013
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Wed, 10 Jul 2013 23:10:13 -0300
Subject: [IPython-dev] IPython Notebook based journal?
In-Reply-To: <51D434EC.3060502@hilboll.de>
References: <51D434EC.3060502@hilboll.de>
Message-ID: <51DE1405.7070005@gmail.com>

El 03/07/13 11:27, Andreas Hilboll escribi?:
> Hi,
>
> I'm very fond of the IPython Notebook and would like to use it as a
> general-purpose lab book. I'm also fan of software like rednotebook_,
> which is basically a diary with the possiblity to tag and categorize
> entries.
>
> Do you see any way of integrating the IPython Notebook in a GUI similar
> to rednotebook? I.e. have a calendar, have the possibility to create one
> or more IPython Notebooks per day, sort them into categories and/or tag
> them, and have the IPython notebook integrated in the main GUI?
>
> Thanks for your thoughts,
> Andreas.
>
> _rednotebook:http://rednotebook.sourceforge.net/
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

It would be an interesting project making a lab notebook based in 
IPython... I will do some test and let you know if I arrive to something 
enough consistent.

BTW, in the mean time, you can use a static blog engine as Nikola, which 
supports making post with the IPython notebook, as a kind of lab 
notebook (my blog as example: http://www.damian.oquanta.info 
<http://www.damian.oquanta.info/>).

You have date/time entries, tags, search, html-powered and easily 
publishable... you don't have a GUI but you have enough infrastructure 
to make an ordered lab notebook.

Cheers.

Dami?n.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/4e099d72/attachment.html>

From ellisonbg at gmail.com  Thu Jul 11 00:04:05 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 21:04:05 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
	<CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
Message-ID: <CAH4pYpS2-qzheBw7FU7pv0sQjf6V9xknxdV2PihrHRroi3B=vA@mail.gmail.com>

Aaron,

Great idea, it would be quite easy to autogenerate sub index pages
based on the prefixes.  I will try to get around to that.

Cheers,

Brian

On Wed, Jul 10, 2013 at 3:34 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>
>
>
> On Wed, Jul 10, 2013 at 5:15 PM, Fernando Perez <fperez.net at gmail.com>
> wrote:
>>
>> On Wed, Jul 10, 2013 at 9:12 AM, Brian Granger <ellisonbg at gmail.com>
>> wrote:
>>
>> > I agree we will absolutely have misc pages that don't fit well in a
>> > prefix.  But the lab meetings page is not one of those (it clearly
>> > fits into the Dev: category).
>>
>> Yup, sorry about that one, as I said, I'm pretty sure I did that :)
>>
>> > There are a couple of reasons I created the prefixes:
>> >
>> > 1) In our previous wiki, there was little to no organization.  Because
>> > of this, people who wanted to add new content couldn't figure see
>> > which pages might already have content of that type and they would
>> > always add new pages.  This meant that we had overlapping information
>> > on multiple pages - and all of it grew stale.  It made the wiki
>> > entirely useless.
>> > 2) When you view a list of all our wiki pages:
>> >
>> > https://github.com/ipython/ipython/wiki/_pages
>> >
>> > the prefixes keep pages grouped together.  We do this type of named
>> > based grouping all the time in our local file systems (to
>> > organize/sort a sequence of notebooks for a tutorial for example).  I
>> > don't see why wiki pages would be any different.  Is it the way I
>> > choose to do the prefixes? ("Foo: ")
>> > 2) Because there is no better way to group wiki pages together on GitHub
>> > :(
>>
>> This #2 is really the main issue IMO.  I personally would much prefer
>> not having this kind of prefixing in names and having categorical
>> metadata (labels) along with a smarter wiki that would auto-create
>> indices.  The problem is that now, any time we make/rename/delete
>> pages, we also have to manually remember to go to the -Index pages and
>> update things.  I hate that.
>>
>> But having said that, I'm pretty sure that I'm less bothered by having
>> prefixes than you are with not having them, therefore I'm all for a
>> workflow that makes you happy :)  As you mentioned in the JS doc
>> discussion, we really need to find practices that make our everyday
>> activities as smooth and pleasant as possible, given the constraints
>> available (in this case, the rather primitive GH wiki engine).
>>
>> So I'm OK with having prefixes, and will be more careful in the future
>> as I make pages, I promise :)  I do worry that the index pages, being
>> manually maintained, will be perennially incorrect and outdated, but
>> I'm not sure what we can do about that.
>
>
> Write a little script that makes it easier to check and update these things
> (remember that the wiki is available in git).  I would find this very useful
> for the SymPy wiki as well.
>
> Aaron Meurer
>
>>
>> As a show of good faith, I've just renamed the lab meeting page :)
>>
>> https://github.com/ipython/ipython/wiki/Dev:-Lab-meetings-on-Air
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Thu Jul 11 00:06:48 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 10 Jul 2013 21:06:48 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
Message-ID: <CAH4pYpShj8EpUL-mc-CAJn030EPVtPH6t=ZOhBYJixm+71qrKQ@mail.gmail.com>

On Wed, Jul 10, 2013 at 3:15 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Wed, Jul 10, 2013 at 9:12 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> I agree we will absolutely have misc pages that don't fit well in a
>> prefix.  But the lab meetings page is not one of those (it clearly
>> fits into the Dev: category).
>
> Yup, sorry about that one, as I said, I'm pretty sure I did that :)

No problem, when I am in a hurry I find the prefixes annoying as well...

>> There are a couple of reasons I created the prefixes:
>>
>> 1) In our previous wiki, there was little to no organization.  Because
>> of this, people who wanted to add new content couldn't figure see
>> which pages might already have content of that type and they would
>> always add new pages.  This meant that we had overlapping information
>> on multiple pages - and all of it grew stale.  It made the wiki
>> entirely useless.
>> 2) When you view a list of all our wiki pages:
>>
>> https://github.com/ipython/ipython/wiki/_pages
>>
>> the prefixes keep pages grouped together.  We do this type of named
>> based grouping all the time in our local file systems (to
>> organize/sort a sequence of notebooks for a tutorial for example).  I
>> don't see why wiki pages would be any different.  Is it the way I
>> choose to do the prefixes? ("Foo: ")
>> 2) Because there is no better way to group wiki pages together on GitHub :(
>
> This #2 is really the main issue IMO.  I personally would much prefer
> not having this kind of prefixing in names and having categorical
> metadata (labels) along with a smarter wiki that would auto-create
> indices.  The problem is that now, any time we make/rename/delete
> pages, we also have to manually remember to go to the -Index pages and
> update things.  I hate that.

Yep me too.  Even simple folders would be a huge upgrade for this wiki.

> But having said that, I'm pretty sure that I'm less bothered by having
> prefixes than you are with not having them, therefore I'm all for a
> workflow that makes you happy :)  As you mentioned in the JS doc
> discussion, we really need to find practices that make our everyday
> activities as smooth and pleasant as possible, given the constraints
> available (in this case, the rather primitive GH wiki engine).
>
> So I'm OK with having prefixes, and will be more careful in the future
> as I make pages, I promise :)  I do worry that the index pages, being
> manually maintained, will be perennially incorrect and outdated, but
> I'm not sure what we can do about that.
>
> As a show of good faith, I've just renamed the lab meeting page :)
>
> https://github.com/ipython/ipython/wiki/Dev:-Lab-meetings-on-Air

Great, thanks. I am going to ad it to the Dev: Index page right now as well.

Cheers,

Brian

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From gmbecker at ucdavis.edu  Thu Jul 11 13:13:35 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Thu, 11 Jul 2013 10:13:35 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CABDkGQ=F1W25zOMRta=_9KaHxnyWha03VfG1FhvfaL3WJspR3g@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CABDkGQnBAGzajzMp=-czN0SDqLOdo4D0h09214C5+HFu8O_qGA@mail.gmail.com>
	<CAH4pYpT1asJS3xc3cnYCR37du-Dx33JWqkofG+MoeeqteVgw3Q@mail.gmail.com>
	<CADwqtCO-LAZOjC583Yprb31HkaGveAti5F5bdwojh+p_2vTTSA@mail.gmail.com>
	<CABDkGQ=F1W25zOMRta=_9KaHxnyWha03VfG1FhvfaL3WJspR3g@mail.gmail.com>
Message-ID: <CADwqtCN5yP+ndfu4ZTV6-kD6-8HTtsOCVKy7V9X0oG4PpP-Smw@mail.gmail.com>

Stefan,

My biggest issue with this is that branching is not a cell-level phenomenon.

The example in my video/screenshots have branches that each only contain
one cell, but that was largely for simplicity and the ability to fit it all
on screen for illustration purposes.

In most cases a branch in the sense that I am envisioning them will
actually be a sequence of cells which together make up a particular
alternative (strategy|algorithm|implementation|*). They need to be tracked
as a group and operated on as a group. It is explicitly invalid (though not
prevented currently in my PoC) when given 2 branches with 3 cells each, to
run the first cell in branch one, then the second cell from branch 2, then
the third from branch one again. That doesn't constitute a valid path
through the document.

Having the branches stored externally (which is what putting them in git
amounts to) also seems like it would gut our ability to easily reason about
the structure of and navigate through the document. Under this model
navigating between paths through the analysis actually changes content of
the notebook itself. Also given four distinct sets of branches, how does
that translate into the actual document structure? Even attempting to do so
requires scraping data from the git repo, and its not clear how
straightforward things would be even in that case.

Nesting branches (alternative analysis strategies, one branch of which
contains alternative choices for model specification, for example) seem
tough to do as well in a way that we can easily extract and reason about
when we want to render or navigate the notebook.

~G



On Wed, Jul 10, 2013 at 11:40 AM, St?fan van der Walt <stefan at sun.ac.za>wrote:

> On Wed, Jul 10, 2013 at 8:25 PM, Gabriel Becker <gmbecker at ucdavis.edu>
> wrote:
> >
> > That having been said, there is a far cry between git being the place
> that the devs think notebooks *should* live and not having notebooks in git
> being unsupported. I would argue building actual application features on
> git brings us pretty firmly into the latter category.
>
> The approach I suggested depends only on one IPython feature: per-cell
> meta-data.  I don't think that will go away any time soon.  The
> solution is bolted on top, so it does not rely on any support inside
> of IPython.  Also, you can then choose which configurations to run and
> generate your output.  In the end, you'll still have to compare the
> output somewhat manually (even though we can potentially track the
> outputs via version control too, making things easier).
>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130711/1475e249/attachment.html>

From gmbecker at ucdavis.edu  Thu Jul 11 18:04:24 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Thu, 11 Jul 2013 15:04:24 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
Message-ID: <CADwqtCP-cJxFqde_9YwsvzdOSRfx_tkrVcTxaE69VxU+Ni43Rw@mail.gmail.com>

Brian,

(Specific responses out of order for clarity and idea/narrative flow)

Last things first: I cannot speak for my advisors of course, but I would
love to come down to Berkeley and chat with you guys about this (and other)
stuff!

Now on to the incredibly long (sorry, but there was a lot of ground to
cover! :( ) response to the rest of your mail...

On Tue, Jul 9, 2013 at 7:32 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Gabriel,
>
> > Thank you for taking the time to watch my video and think about the ideas
> > I'm presenting. It is appreciated.
>
> Cool, I enjoyed it.  Fantastic discussion!
>

I agree. I'm finding this extremely valuable. Being forced to articulate my
ideas/understanding of the situation is extremely useful for me personally.
(I'll probably be cutting and pasting large chunks of what I find myself
writing here when I go to write this stuff up).


>
> Before you get too discouraged, please read on :-)
>
> <snip>
>
> > I once asked a room full of quants to raise their hands if their standard
> > operating procedure was to read in the data, maybe clean it a bit, fit a
> > single model, write up the results and be done. Not only did no one raise
> > their hand, but the mere suggestion that that could be how it worked got
> a
> > substantial laugh from the audience. Even though this is not how the
> work is
> > done, however, it is the narrative encoded into a linear notebook.
>
> Here is my experience of this.  I start out working in a very
> non-linear manner.  As I work I discover things and change my code.
> As I approach the point where I want to share my work, I start to
> linearize it, otherwise it is very difficult for someone to take in.
> In this context branching can be done by it has to be explicit.  In my
> experience this is good.  If I want to run the analysis using 3
> different algorithms, I run them in sequence and then show the results
> of all three in the same place and draw conclusions.  All of this is
> done - at the end of the day - in a linear notebook.
>

That isn't quite the use-case I see branching/alternate supporting
notebooks targeting directly. It is an important one, and one that
branching can be leveraged for, but as you point out, it can be achieved
via careful "linearization" of the analysis.

A main difference between this and what I have in mind is that all three
"branches" are things that need to be run and are intended to be compared
as a necessary component of the final results. In this sense they aren't
actually alternatives in the sense I tend to mean when using that word.

I am  targeting situations where the not-chosen-as-final branches
*don't*contribute directly to the final results. Things that give you
insight into
the data which inform your decision on a final model/algorithm/etc but
aren't actually used to generate the results.

One simple example of this is statistical models which you fit because they
appear appropriate *a priori* based on the type of data and question you
have, but that fail after-the-fact diagnostics. The fact that you fit that
model, the results it gave you, the fact that it failed, and the insight
the particular way it failed gave you into your data are crucial pieces of
your *research*, but reproducing them is not necessary to reproduce your *
results*.

People viewing/running your notebook don't need to look at or execute the
cells where you fit that model ... unless they do.

Branching/DAG notebooks allow a single document to encompass the
*research*you did, while providing easy access to various views
corresponding to the
generation of intermediate, alternative, and final *results*.

These more complex notebooks allow the viewer to ask and answer important
questions such as "What else did (s)he try here?" and potentially even "Why
did (s)he choose this particular analysis strategy?". These questions can
be answered in the text or external supplementary materials in a linear
notebook, but this is a significant barrier to reproducibility of the
research process (as opposed to the analysis results).

Note that sometimes the viewer is you coming back to an analysis after some
span of time so that the reasoning behind your decisions is no longer fresh.

As a practical/UI standpoint unselected branches can be hidden almost
entirely (in theory, not currently in my PoC :p), resulting in a view
equivalent to (any) the only view offered by a linear notebook. This means
that from a viewer (and author since a straight line IS a DAG and nesting
isn't forced) standpoint, what I'm describing is in essense a strict
extension of what the notebook does now, rather than a change.


> BUT, I completely agree that the notebook does not handle certain
> types of branching very well.  Where the notebook starts to really
> suck is for longer analyses that you want to repeat for differing
> parameters or algorithms.  You talk more about this usage case below
> and we have started to think about how we would handle this.  Here are
> our current thoughts:
>
> It would be nice to write a long notebook and then add metadata to the
> notebook that indicates that some variables are to be treated as
> "templated" variables.  Then we would create tools that would enable a
> user to run a notebook over a range of templates:
>
> for x in xvars:
>   for y in yvars:
>     for algo in myalgos
>     run_notebook('MyCoolCode', x, y, algo)
>
> The result would be **something** that allows the user to explore the
> parameter space represented.  A single notebook would be used as the
> "source" for this analysis and the result would be the set of all
> paths through the notebook.  We have even thought about using our
> soon-to-be-designed interactive widget architecture to enable the
> results to be explored using different UI controls (sliders, etc) for
> the xvar, yvar, algos.  This way you could somehow "load" the
> resulting analysis into another notebook and explore things
> interactively - with all of the computations already done.
>
>
This is a very powerful and exciting use-case. In fact it is one I am
investigating myself in the context of a different project unrelated to
IPython notebook. I call the set of results generated by such repeated runs
with different input sets (ie paths through the document) the "robustness
set" of the notebook with respect to the particular output variable being
investigated.

The key here is that the robustness we are talking about is not only with
respect to data/tuning parameters, but also with respect to *the
decisions/choices made during the analysis process* *itself*. These
decisions are often the difference between valid and invalid conclusions,
but are rarely talked about during discussions about reproducible
research/science AFAIK (I'd love to be wrong about that, even if it would
make me look silly/foolish here).

The DAG conceptual model buys us a lot here too though. Instead of having
to run the entire notebook, you can calculate all possible paths through
the DAG for any arbitrary (connected) starting and ending points. So we can
rerun only pieces of  large notebooks to investigate any variable/plot
regardless of whether it constitutes a final result of the
notebook/analsyis.



> We have other people interested in this type of workflow and it can
> all be done within the context of our existing linear notebook model.
> It is just assembling the existing abstractions in different ways.
>
>
That is a plus. There is what I consider to be a pretty major drawback to
this approach though.

It is easy to see how this would work in the case of variables representing
individual number/string/boolean valued parameters without much
perturbation of the code.

Trying to write an analysis script that can graciously handle substantially
dissimilar analysis methods, on the other hand, is more problematic. We can
do it, of course, but at that point we are moving much more into the realm
of a program rather than an analysis script.

Consider the example of classifying new data based on a training set via
KNN, SVM, and GLM approaches. These approaches all need different sets of
parameters, return different types of objects as the output of the fitting
function, may have subtley different behaviour when being used for
prediction, etc.

The abstractions necessary to deal with these differences are likely in my
opinion to be highly costly in terms of how easy it is for readers of the
notebook to follow and understand what the code is doing.

With actual branching, the code in each branch is exactly the same as if it
were in a normal linear notebook which implemented only that one branch,
making it much more likely to be straightforward and easy to read.

One of my targeted use-cases is publications which can more accurately
convey the research which was done while still able to offer the clarity of
focus of what we do now, so I think that is quite important. YMMV.

And now the sticking point.

>
> <snip>

Q: does the new feature violate important abstractions we have in place.
>
> If the answer is no, then we do our normal job of considering the
> costs of adding the feature versus the benefits.
>
> If the answer is yes, then we *stop*.
>

I really do appreciate the IPython team's position. I think there is some
relevant nuance involved in this particular case, however, which makes the
does it change? yes:no test overly coarse. I attempt to make my case for
this below.

I think the answer to the questions "does this new feature violate
important abstractions?" and "is it impossible/burdensomely difficult to
alter important existing abstractions in a way that supports this feature
without affecting the uses of the abstraction?" , may be different here,
despite being the same in the overwhelming majority of cases.  And I would
argue the second test offers identical protections as the first against the
various pitfalls of making major changes to large projects willie-nillie
(which i assure you I do understand are very real).

I'm not advocating a dramatic about-face on the issue complete with parade
and skywriting that "IPython is pursuing an exciting new thing starting
today!". I do, however,  think it is perhapsworth consideration at a
somewhat narrower and more immediate scale than it would be otherwise.


> <snip>
>
> Thinking about your proposed feature from this perspective: both the
> task cells and alt cells introduce hierarchy and nesting into the
> notebook.  This breaks our core abstraction that cells are not nested.
>  In Jan-Feb our core development team had a discussion about this
> abstraction exactly.  We decided that we definitely don't want to move
> in the direction of allowing nesting in the notebook.  Because of this
> we are in the process of removing the 1 level of nesting our notebook
> format currently has, namely worksheets.  So for us, it is not just
> about complexity - it is about breaking the abstractions.
>

I do understand this position. I'd like to think I am bringing up points
not raised during that meeting, but whether or not that is the case
abstractions ARE important.

I guess I am/was thinking about the abstraction in place in IPython
notebook a bit differently than you are describing it.

For the next few paragraphs: process == (render|transform|execute|*)

In my mind the abstraction/computational model is that a notebook is an
ordered set of cells and to process the notebook you simply go through the
cells in order and process them. What process means is dependent on the
type of cell, and there are various pieces of code in various places
(mostly the frontends and nbconvert AFAIK) that know how to handle each
cell type.

Under this formulation the change in abstraction is actually pretty small.
The only addition is the statement that code which processes cells is
responsible for initiating/handling the processing of any child cells those
cells contain. The easy easiest example of this is the execute method on my
task cells, which simply loops through each of its children and (if
applicable) calls their execute method.

With this change we still have a notebook defined as an ordered set of (top
level) cells, and we can still process a notebook by stepping through each
of them in order and processing that cell.

Some changes to the concept of next/previous cells and cell position (for
positional insertion, etc) were required and cells must be aware of their
direct parent (which will be either a cell or the notebook itself), but I
would argue these aren't actually important attributes of the abstraction
itself and the changes were actually fairly narrow and (AFAICS) pretty
painless and straightforward after some careful though/planning.



> The reason that these abstractions are so important is that they
> provide powerful foundations for us to build on.  One place the
> "notebook as a linear sequence of cell" abstraction comes into play is
> in our work on nbconvert that will appear in 1.0 in the next few
> weeks.  This allows to to convert notebooks very easily to a number of
> different formats.


I haven't tackled nbconvert yet on my experimental fork, but I fully intend
to as I agree entirely that the ability to generate things like linear pdfs
and other static views is utterly crucial. The fact that a notebook with
branches can generate a pdf that looks like it came from a linear notebook
(ie the "static article" view) is a *major* selling point/core feature of
what I'm trying to do with branching notebooks. It is key that people be
able to meet the needs they are meeting now; if we can't, meeting the more
nebulous needs they aren't isn't likely to save us (me) from irrelevance.

Under my alternate description of the computational model described above
nbconvert will behave pretty much as it always has: step through the
notebook and process the cells in order into whatever format you are
targeting. The one exception is the cells processing their children, but
the scale of this change is not particularly large for the specific types
of nesting I'm going for.

Tasks would likely simply render their children without making any mark
themselves in most cases, while altsets would do the same, but only for the
"active" branch. This involves a bit of looping and a bunch of calls to
existing code that knows how to transform the existing (core) cell types,
but really not much else.




>  The other place this abstraction comes into play
> is in our keyboard shortcuts.  We are striving for the notebook to be
> usable for people who dont' touch the mouse (your traditional vi/emacs
> users).  Nesting makes that very difficult.
>

I admit this one is tougher, though I've done some small amount thinking
about it (currently hitting "down" on a container cell enters it while
hitting down on the last cell in a container navigates to the cell "after"
the container in my PoC).

I think this is surmountable though, and worth the effort if it were the
only thing holding IPython notebook back from offering a
change/alternative/"fix" to how we talk about research and what we can do
with the documents we use to describe it.

Wow that was a lot of text. Thanks for making it all the way to the end!

~G


<snip>
>
>

-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130711/a7628a03/attachment.html>

From bussonniermatthias at gmail.com  Fri Jul 12 08:57:04 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 12 Jul 2013 14:57:04 +0200
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAH4pYpS2-qzheBw7FU7pv0sQjf6V9xknxdV2PihrHRroi3B=vA@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
	<CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
	<CAH4pYpS2-qzheBw7FU7pv0sQjf6V9xknxdV2PihrHRroi3B=vA@mail.gmail.com>
Message-ID: <D835420F-F0B1-4B16-BE82-15584CB7DFDC@gmail.com>


Le 11 juil. 2013 ? 06:04, Brian Granger a ?crit :

> Aaron,
> 
> Great idea, it would be quite easy to autogenerate sub index pages
> based on the prefixes.  I will try to get around to that.
> 
> Cheers,
> 
> Brian


In case of you guys not knowing, Gollum support sidebars, headers, footer, and TOCs
https://github.com/gollum/gollum/wiki#sidebar-files 

-- 
M



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130712/22b3abde/attachment.html>

From pi at berkeley.edu  Fri Jul 12 11:58:46 2013
From: pi at berkeley.edu (Paul Ivanov)
Date: Fri, 12 Jul 2013 08:58:46 -0700
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <D835420F-F0B1-4B16-BE82-15584CB7DFDC@gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
	<CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
	<CAH4pYpS2-qzheBw7FU7pv0sQjf6V9xknxdV2PihrHRroi3B=vA@mail.gmail.com>
	<D835420F-F0B1-4B16-BE82-15584CB7DFDC@gmail.com>
Message-ID: <20130712155846.GC18092@HbI-OTOH.berkeley.edu>

Matthias BUSSONNIER, on 2013-07-12 14:57,  wrote:
> In case of you guys not knowing, Gollum support sidebars, headers, footer, and TOCs
> https://github.com/gollum/gollum/wiki#sidebar-files 

Thanks, Matthias, and to save someone looking it up, I also
didn't know that "Gollum is a simple wiki system built on top of
Git that powers GitHub Wikis."

best,
-- 
                   _
                  / \
                A*   \^   -
             ,./   _.`\\ / \
            / ,--.S    \/   \
           /  `"~,_     \    \
     __o           ?
   _ \<,_         /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
http://pirsquared.org


From asmeurer at gmail.com  Fri Jul 12 12:05:36 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Fri, 12 Jul 2013 11:05:36 -0500
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <20130712155846.GC18092@HbI-OTOH.berkeley.edu>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
	<CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
	<CAH4pYpS2-qzheBw7FU7pv0sQjf6V9xknxdV2PihrHRroi3B=vA@mail.gmail.com>
	<D835420F-F0B1-4B16-BE82-15584CB7DFDC@gmail.com>
	<20130712155846.GC18092@HbI-OTOH.berkeley.edu>
Message-ID: <CAKgW=6KX+R57yORN8Ub4wMoSsxbF==Oh90NGi=FHFZrgf-g4PQ@mail.gmail.com>

Does someone have examples of this? Not all Gollum features work on GitHub,
so I'd like to see it before I believe it.

Aaron Meurer



On Fri, Jul 12, 2013 at 10:58 AM, Paul Ivanov <pi at berkeley.edu> wrote:

> Matthias BUSSONNIER, on 2013-07-12 14:57,  wrote:
> > In case of you guys not knowing, Gollum support sidebars, headers,
> footer, and TOCs
> > https://github.com/gollum/gollum/wiki#sidebar-files
>
> Thanks, Matthias, and to save someone looking it up, I also
> didn't know that "Gollum is a simple wiki system built on top of
> Git that powers GitHub Wikis."
>
> best,
> --
>                    _
>                   / \
>                 A*   \^   -
>              ,./   _.`\\ / \
>             / ,--.S    \/   \
>            /  `"~,_     \    \
>      __o           ?
>    _ \<,_         /:\
> --(_)/-(_)----.../ | \
> --------------.......J
> Paul Ivanov
> http://pirsquared.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130712/d28a738e/attachment.html>

From bussonniermatthias at gmail.com  Fri Jul 12 12:14:40 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 12 Jul 2013 18:14:40 +0200
Subject: [IPython-dev] Wiki chaos...
In-Reply-To: <CAKgW=6KX+R57yORN8Ub4wMoSsxbF==Oh90NGi=FHFZrgf-g4PQ@mail.gmail.com>
References: <CAH4pYpSUvgKZs9Z+Dee4LAdnjAeiKoNm4KiocGmWJxzk6t8Wtg@mail.gmail.com>
	<CAHAreOpeCRbAO7esHqvQUitsUPm8_RENXAdiVt=c4ryy_TBJ=A@mail.gmail.com>
	<CAH4pYpRZVJLTzHo5E_Er0jaQUC-3yG3LEXZtcFD+R-ni+ZKr3w@mail.gmail.com>
	<CAHAreOoUcDMu5dKsCxOSpQ4rw3vMTDPa3iyRbpdV1FwCc4j1rQ@mail.gmail.com>
	<CAKgW=6KfLCN=OnPCwhM-zJE5srzDJPke5MpoVRW6LaNxLMiSCg@mail.gmail.com>
	<CAH4pYpS2-qzheBw7FU7pv0sQjf6V9xknxdV2PihrHRroi3B=vA@mail.gmail.com>
	<D835420F-F0B1-4B16-BE82-15584CB7DFDC@gmail.com>
	<20130712155846.GC18092@HbI-OTOH.berkeley.edu>
	<CAKgW=6KX+R57yORN8Ub4wMoSsxbF==Oh90NGi=FHFZrgf-g4PQ@mail.gmail.com>
Message-ID: <42D3BE99-ACF7-4E1A-9B3A-7D8FDDC7D676@gmail.com>



Le 12 juil. 2013 ? 17:58, Paul Ivanov a ?crit :

> Thanks, Matthias, and to save someone looking it up, I also
> didn't know that "Gollum is a simple wiki system built on top of
> Git that powers GitHub Wikis."


Oups, sorry , in my mind Gollum == Github (My Precious) Wiki, and I was sure to have wrote the later.



Le 12 juil. 2013 ? 18:05, Aaron Meurer a ?crit :

> Does someone have examples of this? Not all Gollum features work on GitHub, so I'd like to see it before I believe it.

I did saw it on github once, that's actually how I discovered that. 

Where is another question.

-- 
M

From ellisonbg at gmail.com  Fri Jul 12 12:21:50 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 12 Jul 2013 09:21:50 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCP-cJxFqde_9YwsvzdOSRfx_tkrVcTxaE69VxU+Ni43Rw@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CADwqtCP-cJxFqde_9YwsvzdOSRfx_tkrVcTxaE69VxU+Ni43Rw@mail.gmail.com>
Message-ID: <CAH4pYpT4TBgHgqgZXLEN48+GWBk3Lk-_y+NY-QboFC+LH0t=rg@mail.gmail.com>

Gabriel,

Due to some travel for IPython and our family moving, I will probably
drop of the face of the planet in the next few days.  I will reemerge
in early-mid August if you want to continue this discussion at that
point.  Replies inline below...

> Last things first: I cannot speak for my advisors of course, but I would
> love to come down to Berkeley and chat with you guys about this (and other)
> stuff!

Great, let's talk in Sept. to figure out a time that would work.

>> Here is my experience of this.  I start out working in a very
>> non-linear manner.  As I work I discover things and change my code.
>> As I approach the point where I want to share my work, I start to
>> linearize it, otherwise it is very difficult for someone to take in.
>> In this context branching can be done by it has to be explicit.  In my
>> experience this is good.  If I want to run the analysis using 3
>> different algorithms, I run them in sequence and then show the results
>> of all three in the same place and draw conclusions.  All of this is
>> done - at the end of the day - in a linear notebook.
>
>
> That isn't quite the use-case I see branching/alternate supporting notebooks
> targeting directly. It is an important one, and one that branching can be
> leveraged for, but as you point out, it can be achieved via careful
> "linearization" of the analysis.
>
> A main difference between this and what I have in mind is that all three
> "branches" are things that need to be run and are intended to be compared as
> a necessary component of the final results. In this sense they aren't
> actually alternatives in the sense I tend to mean when using that word.
>
> I am  targeting situations where the not-chosen-as-final branches don't
> contribute directly to the final results. Things that give you insight into
> the data which inform your decision on a final model/algorithm/etc but
> aren't actually used to generate the results.

I am following this and this usage case makes sense - although I
haven't found it as often in my own work.

> One simple example of this is statistical models which you fit because they
> appear appropriate a priori based on the type of data and question you have,
> but that fail after-the-fact diagnostics. The fact that you fit that model,
> the results it gave you, the fact that it failed, and the insight the
> particular way it failed gave you into your data are crucial pieces of your
> research, but reproducing them is not necessary to reproduce your results.
>
> People viewing/running your notebook don't need to look at or execute the
> cells where you fit that model ... unless they do.

Yes, in statistical modeling this issue would definitely come up - it
is part of providing the full record of what was done - even the
negative results.

> Branching/DAG notebooks allow a single document to encompass the research
> you did, while providing easy access to various views corresponding to the
> generation of intermediate, alternative, and final results.
>
> These more complex notebooks allow the viewer to ask and answer important
> questions such as "What else did (s)he try here?" and potentially even "Why
> did (s)he choose this particular analysis strategy?". These questions can be
> answered in the text or external supplementary materials in a linear
> notebook, but this is a significant barrier to reproducibility of the
> research process (as opposed to the analysis results).

I can see that, however, I think the pure alt cells lack a critical
feature.  They treat all branches as being equally important.  In
reality, the branch that is chosen as the "best" one will likely
require further analysis and discussion that that other branches
don't.  Putting the different branches side by side makes it a little
like "choose your own adventure" - when in reality, the author of the
research want to steer the reader along a very particular path.  The
alternative paths maybe useful to have around, but they should be be
given equal weight as the "best" one.  But, maybe it is just
presentation and can be accounted for in descriptive text.

At the same time I can image situations where the author really does
want the different branches to receive equal weight as alternatives.

> Note that sometimes the viewer is you coming back to an analysis after some
> span of time so that the reasoning behind your decisions is no longer fresh.

Yes, this is an extremely common - if not the most common - usage case of all...

> As a practical/UI standpoint unselected branches can be hidden almost
> entirely (in theory, not currently in my PoC :p), resulting in a view
> equivalent to (any) the only view offered by a linear notebook. This means
> that from a viewer (and author since a straight line IS a DAG and nesting
> isn't forced) standpoint, what I'm describing is in essense a strict
> extension of what the notebook does now, rather than a change.

I would be *more* interested in alt-cell approaches that present the
notebook as a linear entity in all cases, but that has the alt-cell
logic underneath.  For example, what about the following:

* A user writes the different N alt cells in linear sequence
* The result is a purely linear notebook where one of the N cells should be run.
* We write a JavaScript plugin for the notebook that does a couple of things:

1. It provides a cell toolbar for marking those cells as members of an
alt-set.  This would simple modify the cell level metadata and allow
the author to provide titles of each alt-member.
2. It provides the logic for building a UI for viewing one of the
alt-set members at a time.  It could be as simple as injecting a drop
down menu that shows one and hides the rest.

* This plugin could simple walk the notebook cells and find all the
alt-cell sets and build this supplementary UI.
* This plugin could also have settings that allow the author to select
the "best" member of the alt-set.
* nbconvert Transformers could use the cell level metadata to export
the notebook in different formats.

As I write about this - I think this would be extremely nice, and it
would not be difficult to write at all.  Because of how our JavaScript
plugins work, it could be developed outside IPython initially.  The
question of inclusion in the official code base could be handled
later.  Honestly, this approach should be much easier than the work
you have already done.

Best of all the resulting notebooks would remain standard linear
notebooks that could be shared today on nbviewer, etc.  It would just
work.

Are you interested in taking a shot at this?  I think that would be awesome.

>>
>> BUT, I completely agree that the notebook does not handle certain
>> types of branching very well.  Where the notebook starts to really
>> suck is for longer analyses that you want to repeat for differing
>> parameters or algorithms.  You talk more about this usage case below
>> and we have started to think about how we would handle this.  Here are
>> our current thoughts:
>>
>> It would be nice to write a long notebook and then add metadata to the
>> notebook that indicates that some variables are to be treated as
>> "templated" variables.  Then we would create tools that would enable a
>> user to run a notebook over a range of templates:
>>
>> for x in xvars:
>>   for y in yvars:
>>     for algo in myalgos
>>     run_notebook('MyCoolCode', x, y, algo)
>>
>> The result would be **something** that allows the user to explore the
>> parameter space represented.  A single notebook would be used as the
>> "source" for this analysis and the result would be the set of all
>> paths through the notebook.  We have even thought about using our
>> soon-to-be-designed interactive widget architecture to enable the
>> results to be explored using different UI controls (sliders, etc) for
>> the xvar, yvar, algos.  This way you could somehow "load" the
>> resulting analysis into another notebook and explore things
>> interactively - with all of the computations already done.
>>
>
> This is a very powerful and exciting use-case. In fact it is one I am
> investigating myself in the context of a different project unrelated to
> IPython notebook. I call the set of results generated by such repeated runs
> with different input sets (ie paths through the document) the "robustness
> set" of the notebook with respect to the particular output variable being
> investigated.

Yes, this is a sort to batch mode for the notebook.

> The key here is that the robustness we are talking about is not only with
> respect to data/tuning parameters, but also with respect to the
> decisions/choices made during the analysis process itself. These decisions
> are often the difference between valid and invalid conclusions, but are
> rarely talked about during discussions about reproducible research/science
> AFAIK (I'd love to be wrong about that, even if it would make me look
> silly/foolish here).
>
> The DAG conceptual model buys us a lot here too though. Instead of having to
> run the entire notebook, you can calculate all possible paths through the
> DAG for any arbitrary (connected) starting and ending points. So we can
> rerun only pieces of  large notebooks to investigate any variable/plot
> regardless of whether it constitutes a final result of the
> notebook/analsyis.

Yes, this type of analysis could also be done by the JavaScript plugin
approach above.

>
>>
>> We have other people interested in this type of workflow and it can
>> all be done within the context of our existing linear notebook model.
>> It is just assembling the existing abstractions in different ways.
>>
>
> That is a plus. There is what I consider to be a pretty major drawback to
> this approach though.
>
> It is easy to see how this would work in the case of variables representing
> individual number/string/boolean valued parameters without much perturbation
> of the code.
>
> Trying to write an analysis script that can graciously handle substantially
> dissimilar analysis methods, on the other hand, is more problematic. We can
> do it, of course, but at that point we are moving much more into the realm
> of a program rather than an analysis script.

Yes, definitely.

> Consider the example of classifying new data based on a training set via
> KNN, SVM, and GLM approaches. These approaches all need different sets of
> parameters, return different types of objects as the output of the fitting
> function, may have subtley different behaviour when being used for
> prediction, etc.

Yep, that is the big challenge with the branching idea in general.  It
is not always true that the members of the alt sets can be swapped
out.

> The abstractions necessary to deal with these differences are likely in my
> opinion to be highly costly in terms of how easy it is for readers of the
> notebook to follow and understand what the code is doing.
>
> With actual branching, the code in each branch is exactly the same as if it
> were in a normal linear notebook which implemented only that one branch,
> making it much more likely to be straightforward and easy to read.

But I think the same issue exists with any approach to branching
right?  I am thinking the scripted notebook could have the same type
of API - the important point is that the templated variables, while
simple types, could trigger different code paths.

algo = 0 # a template variable

if algo == 0:
  # alt-cell #1
elif algo == 1:
  # alt-cell #3
...

This is not pretty but it would work...

> One of my targeted use-cases is publications which can more accurately
> convey the research which was done while still able to offer the clarity of
> focus of what we do now, so I think that is quite important. YMMV.
>
> And now the sticking point.
>>
>>
>> <snip>
>>
>> Q: does the new feature violate important abstractions we have in place.
>>
>> If the answer is no, then we do our normal job of considering the
>> costs of adding the feature versus the benefits.
>>
>> If the answer is yes, then we *stop*.
>
>
> I really do appreciate the IPython team's position. I think there is some
> relevant nuance involved in this particular case, however, which makes the
> does it change? yes:no test overly coarse. I attempt to make my case for
> this below.
>
> I think the answer to the questions "does this new feature violate important
> abstractions?" and "is it impossible/burdensomely difficult to alter
> important existing abstractions in a way that supports this feature without
> affecting the uses of the abstraction?" , may be different here, despite
> being the same in the overwhelming majority of cases.  And I would argue the
> second test offers identical protections as the first against the various
> pitfalls of making major changes to large projects willie-nillie (which i
> assure you I do understand are very real).
>
> I'm not advocating a dramatic about-face on the issue complete with parade
> and skywriting that "IPython is pursuing an exciting new thing starting
> today!". I do, however,  think it is perhapsworth consideration at a
> somewhat narrower and more immediate scale than it would be otherwise.

I hope you can see that I really like the general idea and think the
usage cases you are describing are really important.  I think I can
speak for the project in saying that we want the notebook to be useful
for things like this.  But I think our abstractions are important
enough that we make every attempt to see how we can do these while
leveraging our existing abstractions.  This is partially a question
about implementation, but also partly a question about how the new
features are thought about.  The reason we don't like to break
abstractions for new features is that we have found an interesting
relationship between abstraction breaking and new features.  We have
found that when a new feature/idea breaks a core abstraction that we
have thought about very carefully, it is usually because the feature
has not been fully understood.  Time and time again, we have found
that when we take the time to fully understand the feature, it usually
fits within our abstractions beautifully and is even much better that
we ever imagined it could be.

The plugin idea above is a perfect example of this.  By preserving the
abstractions the new feature itself a multiplication of even new
functionality:

* The resulting notebooks can still be version controlled.  This means
that the different alt-cell can be thrown into git and when we develop
a visual diff tool for notebooks, they will *just work*.
* The notebooks can immediately leverage the abstractions we have put
into place for converting notebooks to different formats.  You could
write custom transformers to present the notebook in a reveal.js
giving alt-cells special treatment.
* All of this can be done, and into the hands of user, without going
through those overly conservative IPython developers ;-)
* It will just work with nbviewer as well.
* It provides a cleanly abstracted foundation for other people to build upon

In summary, we are trying to build an architecture that allows a few
simple abstractions (we actually don't have that many!) to combine in
boundless ways to create features we never planned on, but that "just
work".

The upside of this, is that when we have encountered features that are
important to us that really do require us to break or re-vision core
abstractions - we gladly undertake this work.  Mainly because we feel
that the new abstractions will be even more powerful.

>>
>> <snip>
>>
>>
>> Thinking about your proposed feature from this perspective: both the
>> task cells and alt cells introduce hierarchy and nesting into the
>> notebook.  This breaks our core abstraction that cells are not nested.
>>  In Jan-Feb our core development team had a discussion about this
>> abstraction exactly.  We decided that we definitely don't want to move
>> in the direction of allowing nesting in the notebook.  Because of this
>> we are in the process of removing the 1 level of nesting our notebook
>> format currently has, namely worksheets.  So for us, it is not just
>> about complexity - it is about breaking the abstractions.
>
>
> I do understand this position. I'd like to think I am bringing up points not
> raised during that meeting, but whether or not that is the case abstractions
> ARE important.
>
> I guess I am/was thinking about the abstraction in place in IPython notebook
> a bit differently than you are describing it.
>
> For the next few paragraphs: process == (render|transform|execute|*)
>
> In my mind the abstraction/computational model is that a notebook is an
> ordered set of cells and to process the notebook you simply go through the
> cells in order and process them. What process means is dependent on the type
> of cell, and there are various pieces of code in various places (mostly the
> frontends and nbconvert AFAIK) that know how to handle each cell type.
>
> Under this formulation the change in abstraction is actually pretty small.
> The only addition is the statement that code which processes cells is
> responsible for initiating/handling the processing of any child cells those
> cells contain. The easy easiest example of this is the execute method on my
> task cells, which simply loops through each of its children and (if
> applicable) calls their execute method.
>
> With this change we still have a notebook defined as an ordered set of (top
> level) cells, and we can still process a notebook by stepping through each
> of them in order and processing that cell.
>
> Some changes to the concept of next/previous cells and cell position (for
> positional insertion, etc) were required and cells must be aware of their
> direct parent (which will be either a cell or the notebook itself), but I
> would argue these aren't actually important attributes of the abstraction
> itself and the changes were actually fairly narrow and (AFAICS) pretty
> painless and straightforward after some careful though/planning.

This is an interesting way of thinking about nesting that I had not
thought about.

>
>>
>> The reason that these abstractions are so important is that they
>> provide powerful foundations for us to build on.  One place the
>> "notebook as a linear sequence of cell" abstraction comes into play is
>> in our work on nbconvert that will appear in 1.0 in the next few
>> weeks.  This allows to to convert notebooks very easily to a number of
>> different formats.
>
>
> I haven't tackled nbconvert yet on my experimental fork, but I fully intend
> to as I agree entirely that the ability to generate things like linear pdfs
> and other static views is utterly crucial. The fact that a notebook with
> branches can generate a pdf that looks like it came from a linear notebook
> (ie the "static article" view) is a *major* selling point/core feature of
> what I'm trying to do with branching notebooks. It is key that people be
> able to meet the needs they are meeting now; if we can't, meeting the more
> nebulous needs they aren't isn't likely to save us (me) from irrelevance.
>
> Under my alternate description of the computational model described above
> nbconvert will behave pretty much as it always has: step through the
> notebook and process the cells in order into whatever format you are
> targeting. The one exception is the cells processing their children, but the
> scale of this change is not particularly large for the specific types of
> nesting I'm going for.
>
> Tasks would likely simply render their children without making any mark
> themselves in most cases, while altsets would do the same, but only for the
> "active" branch. This involves a bit of looping and a bunch of calls to
> existing code that knows how to transform the existing (core) cell types,
> but really not much else.
>
>
>
>>
>>  The other place this abstraction comes into play
>> is in our keyboard shortcuts.  We are striving for the notebook to be
>> usable for people who dont' touch the mouse (your traditional vi/emacs
>> users).  Nesting makes that very difficult.
>
>
> I admit this one is tougher, though I've done some small amount thinking
> about it (currently hitting "down" on a container cell enters it while
> hitting down on the last cell in a container navigates to the cell "after"
> the container in my PoC).
>
> I think this is surmountable though, and worth the effort if it were the
> only thing holding IPython notebook back from offering a
> change/alternative/"fix" to how we talk about research and what we can do
> with the documents we use to describe it.
>
> Wow that was a lot of text. Thanks for making it all the way to the end!

I made it!

Cheers,

Brian

> ~G
>
>
>> <snip>
>>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Fri Jul 12 18:26:42 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 13 Jul 2013 00:26:42 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <CAP-uhDfbu-7vz9UjCqdm2C6fAhjRHW1R4WxfAzTeSJg1jupU2Q@mail.gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
	<463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
	<CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>
	<CAP-uhDfbu-7vz9UjCqdm2C6fAhjRHW1R4WxfAzTeSJg1jupU2Q@mail.gmail.com>
Message-ID: <3452DDA7-3ADA-4D4B-94BB-03B00474E082@gmail.com>

Le 11 juil. 2013 ? 00:29, Carl Smith a ?crit :

> Community building??

We can try, 

With the few responses, I think Monday will be the best.
I Propose we meet directly at the bar.

Les grains nobles: http://goo.gl/maps/K4aJb

Feel free to come by even if you haven't answerd.
-- 
Matthias



From nelle.varoquaux at gmail.com  Fri Jul 12 18:41:05 2013
From: nelle.varoquaux at gmail.com (Nelle Varoquaux)
Date: Sat, 13 Jul 2013 00:41:05 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <3452DDA7-3ADA-4D4B-94BB-03B00474E082@gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
	<463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
	<CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>
	<CAP-uhDfbu-7vz9UjCqdm2C6fAhjRHW1R4WxfAzTeSJg1jupU2Q@mail.gmail.com>
	<3452DDA7-3ADA-4D4B-94BB-03B00474E082@gmail.com>
Message-ID: <CAE-UAvS-qVX+mYTrZzP4t6w8A9-d=MfVFU_LY+eUD7HTX3HXzw@mail.gmail.com>

On 13 July 2013 00:26, Matthias BUSSONNIER <bussonniermatthias at gmail.com> wrote:
> Le 11 juil. 2013 ? 00:29, Carl Smith a ?crit :
>
>> Community building??
>
> We can try,
>
> With the few responses, I think Monday will be the best.
> I Propose we meet directly at the bar.

Around what time ?

>
> Les grains nobles: http://goo.gl/maps/K4aJb
>
> Feel free to come by even if you haven't answerd.
> --
> Matthias
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From bussonniermatthias at gmail.com  Fri Jul 12 18:47:04 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 13 Jul 2013 00:47:04 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <CAE-UAvS-qVX+mYTrZzP4t6w8A9-d=MfVFU_LY+eUD7HTX3HXzw@mail.gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
	<463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
	<CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>
	<CAP-uhDfbu-7vz9UjCqdm2C6fAhjRHW1R4WxfAzTeSJg1jupU2Q@mail.gmail.com>
	<3452DDA7-3ADA-4D4B-94BB-03B00474E082@gmail.com>
	<CAE-UAvS-qVX+mYTrZzP4t6w8A9-d=MfVFU_LY+eUD7HTX3HXzw@mail.gmail.com>
Message-ID: <871B3EE7-E239-459A-A9FF-A44203B198D9@gmail.com>


Le 13 juil. 2013 ? 00:41, Nelle Varoquaux a ?crit :
> 
> Around what time ?

I'll be there around 19h I guess. 
That what's I wrote on doodle. 

Just in case, my phone number (in a non-machin readable form).
9661?70039 (with a leading 0, or +33)
-- 
Matthias

From gmbecker at ucdavis.edu  Sat Jul 13 19:29:33 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Sat, 13 Jul 2013 16:29:33 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpT4TBgHgqgZXLEN48+GWBk3Lk-_y+NY-QboFC+LH0t=rg@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CADwqtCP-cJxFqde_9YwsvzdOSRfx_tkrVcTxaE69VxU+Ni43Rw@mail.gmail.com>
	<CAH4pYpT4TBgHgqgZXLEN48+GWBk3Lk-_y+NY-QboFC+LH0t=rg@mail.gmail.com>
Message-ID: <CADwqtCOE2hoQ99Kdaym1c6w1nYd1T=7i8R+cPrgL=6JRx_OiPQ@mail.gmail.com>

On Fri, Jul 12, 2013 at 9:21 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> Gabriel,
>
> Due to some travel for IPython and our family moving, I will probably
> drop of the face of the planet in the next few days.  I will reemerge
> in early-mid August if you want to continue this discussion at that
> point.  Replies inline below...
>

I will look forward to that. My research plan calls for me to be pushing
hard on these concepts this summer, so there will likely be even more to
talk about then :)



>
> > Last things first: I cannot speak for my advisors of course, but I would
> > love to come down to Berkeley and chat with you guys about this (and
> other)
> > stuff!
>
> Great, let's talk in Sept. to figure out a time that would work.
>

Definitely.


>
> >> Here is my experience of this.  I start out working in a very
> >> non-linear manner.  As I work I discover things and change my code.
> >> As I approach the point where I want to share my work, I start to
> >> linearize it, otherwise it is very difficult for someone to take in.
> >> In this context branching can be done by it has to be explicit.  In my
> >> experience this is good.  If I want to run the analysis using 3
> >> different algorithms, I run them in sequence and then show the results
> >> of all three in the same place and draw conclusions.  All of this is
> >> done - at the end of the day - in a linear notebook.
> >
> >
> > That isn't quite the use-case I see branching/alternate supporting
> notebooks
> > targeting directly. It is an important one, and one that branching can be
> > leveraged for, but as you point out, it can be achieved via careful
> > "linearization" of the analysis.
> >
> > A main difference between this and what I have in mind is that all three
> > "branches" are things that need to be run and are intended to be
> compared as
> > a necessary component of the final results. In this sense they aren't
> > actually alternatives in the sense I tend to mean when using that word.
> >
> > I am  targeting situations where the not-chosen-as-final branches don't
> > contribute directly to the final results. Things that give you insight
> into
> > the data which inform your decision on a final model/algorithm/etc but
> > aren't actually used to generate the results.
>
> I am following this and this usage case makes sense - although I
> haven't found it as often in my own work.
>
> > One simple example of this is statistical models which you fit because
> they
> > appear appropriate a priori based on the type of data and question you
> have,
> > but that fail after-the-fact diagnostics. The fact that you fit that
> model,
> > the results it gave you, the fact that it failed, and the insight the
> > particular way it failed gave you into your data are crucial pieces of
> your
> > research, but reproducing them is not necessary to reproduce your
> results.
> >
> > People viewing/running your notebook don't need to look at or execute the
> > cells where you fit that model ... unless they do.
>
> Yes, in statistical modeling this issue would definitely come up - it
> is part of providing the full record of what was done - even the
> negative results.
>
> > Branching/DAG notebooks allow a single document to encompass the research
> > you did, while providing easy access to various views corresponding to
> the
> > generation of intermediate, alternative, and final results.
> >
> > These more complex notebooks allow the viewer to ask and answer important
> > questions such as "What else did (s)he try here?" and potentially even
> "Why
> > did (s)he choose this particular analysis strategy?". These questions
> can be
> > answered in the text or external supplementary materials in a linear
> > notebook, but this is a significant barrier to reproducibility of the
> > research process (as opposed to the analysis results).
>
> I can see that, however, I think the pure alt cells lack a critical
> feature.  They treat all branches as being equally important.  In
> reality, the branch that is chosen as the "best" one will likely
> require further analysis and discussion that that other branches
> don't.  Putting the different branches side by side makes it a little
> like "choose your own adventure" - when in reality, the author of the
> research want to steer the reader along a very particular path.  The
> alternative paths maybe useful to have around, but they should be be
> given equal weight as the "best" one.  But, maybe it is just
> presentation and can be accounted for in descriptive text.
>
> At the same time I can image situations where the author really does
> want the different branches to receive equal weight as alternatives.
>

I agree. The alt/altset cell mechanism (whatever form it takes in the
actual objects) is the scaffolding on which we will build what we actually
want. It definitely is important to have "primary" and "non-primary"
branches in many cases, and other types of differentiation which inform
behavior are also pretty important.

What exactly we want in terms of different types of branches, different
behaviors when authoring/processing/querying non-linear notebook
structures, etc is a major component of what I'm working on right now. So
more on this soon.

I will think about the rest of what you said and suggested and we can take
it back up when you get back.

Have a good trip/move

~G


>
> > Note that sometimes the viewer is you coming back to an analysis after
> some
> > span of time so that the reasoning behind your decisions is no longer
> fresh.
>
> Yes, this is an extremely common - if not the most common - usage case of
> all...
>
> > As a practical/UI standpoint unselected branches can be hidden almost
> > entirely (in theory, not currently in my PoC :p), resulting in a view
> > equivalent to (any) the only view offered by a linear notebook. This
> means
> > that from a viewer (and author since a straight line IS a DAG and nesting
> > isn't forced) standpoint, what I'm describing is in essense a strict
> > extension of what the notebook does now, rather than a change.
>
> I would be *more* interested in alt-cell approaches that present the
> notebook as a linear entity in all cases, but that has the alt-cell
> logic underneath.  For example, what about the following:
>
> * A user writes the different N alt cells in linear sequence
> * The result is a purely linear notebook where one of the N cells should
> be run.
> * We write a JavaScript plugin for the notebook that does a couple of
> things:
>
> 1. It provides a cell toolbar for marking those cells as members of an
> alt-set.  This would simple modify the cell level metadata and allow
> the author to provide titles of each alt-member.
> 2. It provides the logic for building a UI for viewing one of the
> alt-set members at a time.  It could be as simple as injecting a drop
> down menu that shows one and hides the rest.
>
> * This plugin could simple walk the notebook cells and find all the
> alt-cell sets and build this supplementary UI.
> * This plugin could also have settings that allow the author to select
> the "best" member of the alt-set.
> * nbconvert Transformers could use the cell level metadata to export
> the notebook in different formats.
>
> As I write about this - I think this would be extremely nice, and it
> would not be difficult to write at all.  Because of how our JavaScript
> plugins work, it could be developed outside IPython initially.  The
> question of inclusion in the official code base could be handled
> later.  Honestly, this approach should be much easier than the work
> you have already done.
>
> Best of all the resulting notebooks would remain standard linear
> notebooks that could be shared today on nbviewer, etc.  It would just
> work.
>
> Are you interested in taking a shot at this?  I think that would be
> awesome.
>
> >>
> >> BUT, I completely agree that the notebook does not handle certain
> >> types of branching very well.  Where the notebook starts to really
> >> suck is for longer analyses that you want to repeat for differing
> >> parameters or algorithms.  You talk more about this usage case below
> >> and we have started to think about how we would handle this.  Here are
> >> our current thoughts:
> >>
> >> It would be nice to write a long notebook and then add metadata to the
> >> notebook that indicates that some variables are to be treated as
> >> "templated" variables.  Then we would create tools that would enable a
> >> user to run a notebook over a range of templates:
> >>
> >> for x in xvars:
> >>   for y in yvars:
> >>     for algo in myalgos
> >>     run_notebook('MyCoolCode', x, y, algo)
> >>
> >> The result would be **something** that allows the user to explore the
> >> parameter space represented.  A single notebook would be used as the
> >> "source" for this analysis and the result would be the set of all
> >> paths through the notebook.  We have even thought about using our
> >> soon-to-be-designed interactive widget architecture to enable the
> >> results to be explored using different UI controls (sliders, etc) for
> >> the xvar, yvar, algos.  This way you could somehow "load" the
> >> resulting analysis into another notebook and explore things
> >> interactively - with all of the computations already done.
> >>
> >
> > This is a very powerful and exciting use-case. In fact it is one I am
> > investigating myself in the context of a different project unrelated to
> > IPython notebook. I call the set of results generated by such repeated
> runs
> > with different input sets (ie paths through the document) the "robustness
> > set" of the notebook with respect to the particular output variable being
> > investigated.
>
> Yes, this is a sort to batch mode for the notebook.
>
> > The key here is that the robustness we are talking about is not only with
> > respect to data/tuning parameters, but also with respect to the
> > decisions/choices made during the analysis process itself. These
> decisions
> > are often the difference between valid and invalid conclusions, but are
> > rarely talked about during discussions about reproducible
> research/science
> > AFAIK (I'd love to be wrong about that, even if it would make me look
> > silly/foolish here).
> >
> > The DAG conceptual model buys us a lot here too though. Instead of
> having to
> > run the entire notebook, you can calculate all possible paths through the
> > DAG for any arbitrary (connected) starting and ending points. So we can
> > rerun only pieces of  large notebooks to investigate any variable/plot
> > regardless of whether it constitutes a final result of the
> > notebook/analsyis.
>
> Yes, this type of analysis could also be done by the JavaScript plugin
> approach above.
>
> >
> >>
> >> We have other people interested in this type of workflow and it can
> >> all be done within the context of our existing linear notebook model.
> >> It is just assembling the existing abstractions in different ways.
> >>
> >
> > That is a plus. There is what I consider to be a pretty major drawback to
> > this approach though.
> >
> > It is easy to see how this would work in the case of variables
> representing
> > individual number/string/boolean valued parameters without much
> perturbation
> > of the code.
> >
> > Trying to write an analysis script that can graciously handle
> substantially
> > dissimilar analysis methods, on the other hand, is more problematic. We
> can
> > do it, of course, but at that point we are moving much more into the
> realm
> > of a program rather than an analysis script.
>
> Yes, definitely.
>
> > Consider the example of classifying new data based on a training set via
> > KNN, SVM, and GLM approaches. These approaches all need different sets of
> > parameters, return different types of objects as the output of the
> fitting
> > function, may have subtley different behaviour when being used for
> > prediction, etc.
>
> Yep, that is the big challenge with the branching idea in general.  It
> is not always true that the members of the alt sets can be swapped
> out.
>
> > The abstractions necessary to deal with these differences are likely in
> my
> > opinion to be highly costly in terms of how easy it is for readers of the
> > notebook to follow and understand what the code is doing.
> >
> > With actual branching, the code in each branch is exactly the same as if
> it
> > were in a normal linear notebook which implemented only that one branch,
> > making it much more likely to be straightforward and easy to read.
>
> But I think the same issue exists with any approach to branching
> right?  I am thinking the scripted notebook could have the same type
> of API - the important point is that the templated variables, while
> simple types, could trigger different code paths.
>
> algo = 0 # a template variable
>
> if algo == 0:
>   # alt-cell #1
> elif algo == 1:
>   # alt-cell #3
> ...
>
> This is not pretty but it would work...
>
> > One of my targeted use-cases is publications which can more accurately
> > convey the research which was done while still able to offer the clarity
> of
> > focus of what we do now, so I think that is quite important. YMMV.
> >
> > And now the sticking point.
> >>
> >>
> >> <snip>
> >>
> >> Q: does the new feature violate important abstractions we have in place.
> >>
> >> If the answer is no, then we do our normal job of considering the
> >> costs of adding the feature versus the benefits.
> >>
> >> If the answer is yes, then we *stop*.
> >
> >
> > I really do appreciate the IPython team's position. I think there is some
> > relevant nuance involved in this particular case, however, which makes
> the
> > does it change? yes:no test overly coarse. I attempt to make my case for
> > this below.
> >
> > I think the answer to the questions "does this new feature violate
> important
> > abstractions?" and "is it impossible/burdensomely difficult to alter
> > important existing abstractions in a way that supports this feature
> without
> > affecting the uses of the abstraction?" , may be different here, despite
> > being the same in the overwhelming majority of cases.  And I would argue
> the
> > second test offers identical protections as the first against the various
> > pitfalls of making major changes to large projects willie-nillie (which i
> > assure you I do understand are very real).
> >
> > I'm not advocating a dramatic about-face on the issue complete with
> parade
> > and skywriting that "IPython is pursuing an exciting new thing starting
> > today!". I do, however,  think it is perhapsworth consideration at a
> > somewhat narrower and more immediate scale than it would be otherwise.
>
> I hope you can see that I really like the general idea and think the
> usage cases you are describing are really important.  I think I can
> speak for the project in saying that we want the notebook to be useful
> for things like this.  But I think our abstractions are important
> enough that we make every attempt to see how we can do these while
> leveraging our existing abstractions.  This is partially a question
> about implementation, but also partly a question about how the new
> features are thought about.  The reason we don't like to break
> abstractions for new features is that we have found an interesting
> relationship between abstraction breaking and new features.  We have
> found that when a new feature/idea breaks a core abstraction that we
> have thought about very carefully, it is usually because the feature
> has not been fully understood.  Time and time again, we have found
> that when we take the time to fully understand the feature, it usually
> fits within our abstractions beautifully and is even much better that
> we ever imagined it could be.
>
> The plugin idea above is a perfect example of this.  By preserving the
> abstractions the new feature itself a multiplication of even new
> functionality:
>
> * The resulting notebooks can still be version controlled.  This means
> that the different alt-cell can be thrown into git and when we develop
> a visual diff tool for notebooks, they will *just work*.
> * The notebooks can immediately leverage the abstractions we have put
> into place for converting notebooks to different formats.  You could
> write custom transformers to present the notebook in a reveal.js
> giving alt-cells special treatment.
> * All of this can be done, and into the hands of user, without going
> through those overly conservative IPython developers ;-)
> * It will just work with nbviewer as well.
> * It provides a cleanly abstracted foundation for other people to build
> upon
>
> In summary, we are trying to build an architecture that allows a few
> simple abstractions (we actually don't have that many!) to combine in
> boundless ways to create features we never planned on, but that "just
> work".
>
> The upside of this, is that when we have encountered features that are
> important to us that really do require us to break or re-vision core
> abstractions - we gladly undertake this work.  Mainly because we feel
> that the new abstractions will be even more powerful.
>
> >>
> >> <snip>
> >>
> >>
> >> Thinking about your proposed feature from this perspective: both the
> >> task cells and alt cells introduce hierarchy and nesting into the
> >> notebook.  This breaks our core abstraction that cells are not nested.
> >>  In Jan-Feb our core development team had a discussion about this
> >> abstraction exactly.  We decided that we definitely don't want to move
> >> in the direction of allowing nesting in the notebook.  Because of this
> >> we are in the process of removing the 1 level of nesting our notebook
> >> format currently has, namely worksheets.  So for us, it is not just
> >> about complexity - it is about breaking the abstractions.
> >
> >
> > I do understand this position. I'd like to think I am bringing up points
> not
> > raised during that meeting, but whether or not that is the case
> abstractions
> > ARE important.
> >
> > I guess I am/was thinking about the abstraction in place in IPython
> notebook
> > a bit differently than you are describing it.
> >
> > For the next few paragraphs: process == (render|transform|execute|*)
> >
> > In my mind the abstraction/computational model is that a notebook is an
> > ordered set of cells and to process the notebook you simply go through
> the
> > cells in order and process them. What process means is dependent on the
> type
> > of cell, and there are various pieces of code in various places (mostly
> the
> > frontends and nbconvert AFAIK) that know how to handle each cell type.
> >
> > Under this formulation the change in abstraction is actually pretty
> small.
> > The only addition is the statement that code which processes cells is
> > responsible for initiating/handling the processing of any child cells
> those
> > cells contain. The easy easiest example of this is the execute method on
> my
> > task cells, which simply loops through each of its children and (if
> > applicable) calls their execute method.
> >
> > With this change we still have a notebook defined as an ordered set of
> (top
> > level) cells, and we can still process a notebook by stepping through
> each
> > of them in order and processing that cell.
> >
> > Some changes to the concept of next/previous cells and cell position (for
> > positional insertion, etc) were required and cells must be aware of their
> > direct parent (which will be either a cell or the notebook itself), but I
> > would argue these aren't actually important attributes of the abstraction
> > itself and the changes were actually fairly narrow and (AFAICS) pretty
> > painless and straightforward after some careful though/planning.
>
> This is an interesting way of thinking about nesting that I had not
> thought about.
>
> >
> >>
> >> The reason that these abstractions are so important is that they
> >> provide powerful foundations for us to build on.  One place the
> >> "notebook as a linear sequence of cell" abstraction comes into play is
> >> in our work on nbconvert that will appear in 1.0 in the next few
> >> weeks.  This allows to to convert notebooks very easily to a number of
> >> different formats.
> >
> >
> > I haven't tackled nbconvert yet on my experimental fork, but I fully
> intend
> > to as I agree entirely that the ability to generate things like linear
> pdfs
> > and other static views is utterly crucial. The fact that a notebook with
> > branches can generate a pdf that looks like it came from a linear
> notebook
> > (ie the "static article" view) is a *major* selling point/core feature of
> > what I'm trying to do with branching notebooks. It is key that people be
> > able to meet the needs they are meeting now; if we can't, meeting the
> more
> > nebulous needs they aren't isn't likely to save us (me) from irrelevance.
> >
> > Under my alternate description of the computational model described above
> > nbconvert will behave pretty much as it always has: step through the
> > notebook and process the cells in order into whatever format you are
> > targeting. The one exception is the cells processing their children, but
> the
> > scale of this change is not particularly large for the specific types of
> > nesting I'm going for.
> >
> > Tasks would likely simply render their children without making any mark
> > themselves in most cases, while altsets would do the same, but only for
> the
> > "active" branch. This involves a bit of looping and a bunch of calls to
> > existing code that knows how to transform the existing (core) cell types,
> > but really not much else.
> >
> >
> >
> >>
> >>  The other place this abstraction comes into play
> >> is in our keyboard shortcuts.  We are striving for the notebook to be
> >> usable for people who dont' touch the mouse (your traditional vi/emacs
> >> users).  Nesting makes that very difficult.
> >
> >
> > I admit this one is tougher, though I've done some small amount thinking
> > about it (currently hitting "down" on a container cell enters it while
> > hitting down on the last cell in a container navigates to the cell
> "after"
> > the container in my PoC).
> >
> > I think this is surmountable though, and worth the effort if it were the
> > only thing holding IPython notebook back from offering a
> > change/alternative/"fix" to how we talk about research and what we can do
> > with the documents we use to describe it.
> >
> > Wow that was a lot of text. Thanks for making it all the way to the end!
>
> I made it!
>
> Cheers,
>
> Brian
>
> > ~G
> >
> >
> >> <snip>
> >>
> >
> >
> > --
> > Gabriel Becker
> > Graduate Student
> > Statistics Department
> > University of California, Davis
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130713/85a723f8/attachment.html>

From ellisonbg at gmail.com  Sat Jul 13 19:40:27 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 13 Jul 2013 16:40:27 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CADwqtCOE2hoQ99Kdaym1c6w1nYd1T=7i8R+cPrgL=6JRx_OiPQ@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CADwqtCP-cJxFqde_9YwsvzdOSRfx_tkrVcTxaE69VxU+Ni43Rw@mail.gmail.com>
	<CAH4pYpT4TBgHgqgZXLEN48+GWBk3Lk-_y+NY-QboFC+LH0t=rg@mail.gmail.com>
	<CADwqtCOE2hoQ99Kdaym1c6w1nYd1T=7i8R+cPrgL=6JRx_OiPQ@mail.gmail.com>
Message-ID: <CAH4pYpRFcwRT=Gi6ZYTsXRKAkP6i=7X7f6SpSku43+JY-xwFKQ@mail.gmail.com>

> I will think about the rest of what you said and suggested and we can take
> it back up when you get back.

Great.

> Have a good trip/move

Thanks,

Brian

> ~G
>
>>
>>
>> > Note that sometimes the viewer is you coming back to an analysis after
>> > some
>> > span of time so that the reasoning behind your decisions is no longer
>> > fresh.
>>
>> Yes, this is an extremely common - if not the most common - usage case of
>> all...
>>
>> > As a practical/UI standpoint unselected branches can be hidden almost
>> > entirely (in theory, not currently in my PoC :p), resulting in a view
>> > equivalent to (any) the only view offered by a linear notebook. This
>> > means
>> > that from a viewer (and author since a straight line IS a DAG and
>> > nesting
>> > isn't forced) standpoint, what I'm describing is in essense a strict
>> > extension of what the notebook does now, rather than a change.
>>
>> I would be *more* interested in alt-cell approaches that present the
>> notebook as a linear entity in all cases, but that has the alt-cell
>> logic underneath.  For example, what about the following:
>>
>> * A user writes the different N alt cells in linear sequence
>> * The result is a purely linear notebook where one of the N cells should
>> be run.
>> * We write a JavaScript plugin for the notebook that does a couple of
>> things:
>>
>> 1. It provides a cell toolbar for marking those cells as members of an
>> alt-set.  This would simple modify the cell level metadata and allow
>> the author to provide titles of each alt-member.
>> 2. It provides the logic for building a UI for viewing one of the
>> alt-set members at a time.  It could be as simple as injecting a drop
>> down menu that shows one and hides the rest.
>>
>> * This plugin could simple walk the notebook cells and find all the
>> alt-cell sets and build this supplementary UI.
>> * This plugin could also have settings that allow the author to select
>> the "best" member of the alt-set.
>> * nbconvert Transformers could use the cell level metadata to export
>> the notebook in different formats.
>>
>> As I write about this - I think this would be extremely nice, and it
>> would not be difficult to write at all.  Because of how our JavaScript
>> plugins work, it could be developed outside IPython initially.  The
>> question of inclusion in the official code base could be handled
>> later.  Honestly, this approach should be much easier than the work
>> you have already done.
>>
>> Best of all the resulting notebooks would remain standard linear
>> notebooks that could be shared today on nbviewer, etc.  It would just
>> work.
>>
>> Are you interested in taking a shot at this?  I think that would be
>> awesome.
>>
>> >>
>> >> BUT, I completely agree that the notebook does not handle certain
>> >> types of branching very well.  Where the notebook starts to really
>> >> suck is for longer analyses that you want to repeat for differing
>> >> parameters or algorithms.  You talk more about this usage case below
>> >> and we have started to think about how we would handle this.  Here are
>> >> our current thoughts:
>> >>
>> >> It would be nice to write a long notebook and then add metadata to the
>> >> notebook that indicates that some variables are to be treated as
>> >> "templated" variables.  Then we would create tools that would enable a
>> >> user to run a notebook over a range of templates:
>> >>
>> >> for x in xvars:
>> >>   for y in yvars:
>> >>     for algo in myalgos
>> >>     run_notebook('MyCoolCode', x, y, algo)
>> >>
>> >> The result would be **something** that allows the user to explore the
>> >> parameter space represented.  A single notebook would be used as the
>> >> "source" for this analysis and the result would be the set of all
>> >> paths through the notebook.  We have even thought about using our
>> >> soon-to-be-designed interactive widget architecture to enable the
>> >> results to be explored using different UI controls (sliders, etc) for
>> >> the xvar, yvar, algos.  This way you could somehow "load" the
>> >> resulting analysis into another notebook and explore things
>> >> interactively - with all of the computations already done.
>> >>
>> >
>> > This is a very powerful and exciting use-case. In fact it is one I am
>> > investigating myself in the context of a different project unrelated to
>> > IPython notebook. I call the set of results generated by such repeated
>> > runs
>> > with different input sets (ie paths through the document) the
>> > "robustness
>> > set" of the notebook with respect to the particular output variable
>> > being
>> > investigated.
>>
>> Yes, this is a sort to batch mode for the notebook.
>>
>> > The key here is that the robustness we are talking about is not only
>> > with
>> > respect to data/tuning parameters, but also with respect to the
>> > decisions/choices made during the analysis process itself. These
>> > decisions
>> > are often the difference between valid and invalid conclusions, but are
>> > rarely talked about during discussions about reproducible
>> > research/science
>> > AFAIK (I'd love to be wrong about that, even if it would make me look
>> > silly/foolish here).
>> >
>> > The DAG conceptual model buys us a lot here too though. Instead of
>> > having to
>> > run the entire notebook, you can calculate all possible paths through
>> > the
>> > DAG for any arbitrary (connected) starting and ending points. So we can
>> > rerun only pieces of  large notebooks to investigate any variable/plot
>> > regardless of whether it constitutes a final result of the
>> > notebook/analsyis.
>>
>> Yes, this type of analysis could also be done by the JavaScript plugin
>> approach above.
>>
>> >
>> >>
>> >> We have other people interested in this type of workflow and it can
>> >> all be done within the context of our existing linear notebook model.
>> >> It is just assembling the existing abstractions in different ways.
>> >>
>> >
>> > That is a plus. There is what I consider to be a pretty major drawback
>> > to
>> > this approach though.
>> >
>> > It is easy to see how this would work in the case of variables
>> > representing
>> > individual number/string/boolean valued parameters without much
>> > perturbation
>> > of the code.
>> >
>> > Trying to write an analysis script that can graciously handle
>> > substantially
>> > dissimilar analysis methods, on the other hand, is more problematic. We
>> > can
>> > do it, of course, but at that point we are moving much more into the
>> > realm
>> > of a program rather than an analysis script.
>>
>> Yes, definitely.
>>
>> > Consider the example of classifying new data based on a training set via
>> > KNN, SVM, and GLM approaches. These approaches all need different sets
>> > of
>> > parameters, return different types of objects as the output of the
>> > fitting
>> > function, may have subtley different behaviour when being used for
>> > prediction, etc.
>>
>> Yep, that is the big challenge with the branching idea in general.  It
>> is not always true that the members of the alt sets can be swapped
>> out.
>>
>> > The abstractions necessary to deal with these differences are likely in
>> > my
>> > opinion to be highly costly in terms of how easy it is for readers of
>> > the
>> > notebook to follow and understand what the code is doing.
>> >
>> > With actual branching, the code in each branch is exactly the same as if
>> > it
>> > were in a normal linear notebook which implemented only that one branch,
>> > making it much more likely to be straightforward and easy to read.
>>
>> But I think the same issue exists with any approach to branching
>> right?  I am thinking the scripted notebook could have the same type
>> of API - the important point is that the templated variables, while
>> simple types, could trigger different code paths.
>>
>> algo = 0 # a template variable
>>
>> if algo == 0:
>>   # alt-cell #1
>> elif algo == 1:
>>   # alt-cell #3
>> ...
>>
>> This is not pretty but it would work...
>>
>> > One of my targeted use-cases is publications which can more accurately
>> > convey the research which was done while still able to offer the clarity
>> > of
>> > focus of what we do now, so I think that is quite important. YMMV.
>> >
>> > And now the sticking point.
>> >>
>> >>
>> >> <snip>
>> >>
>> >> Q: does the new feature violate important abstractions we have in
>> >> place.
>> >>
>> >> If the answer is no, then we do our normal job of considering the
>> >> costs of adding the feature versus the benefits.
>> >>
>> >> If the answer is yes, then we *stop*.
>> >
>> >
>> > I really do appreciate the IPython team's position. I think there is
>> > some
>> > relevant nuance involved in this particular case, however, which makes
>> > the
>> > does it change? yes:no test overly coarse. I attempt to make my case for
>> > this below.
>> >
>> > I think the answer to the questions "does this new feature violate
>> > important
>> > abstractions?" and "is it impossible/burdensomely difficult to alter
>> > important existing abstractions in a way that supports this feature
>> > without
>> > affecting the uses of the abstraction?" , may be different here, despite
>> > being the same in the overwhelming majority of cases.  And I would argue
>> > the
>> > second test offers identical protections as the first against the
>> > various
>> > pitfalls of making major changes to large projects willie-nillie (which
>> > i
>> > assure you I do understand are very real).
>> >
>> > I'm not advocating a dramatic about-face on the issue complete with
>> > parade
>> > and skywriting that "IPython is pursuing an exciting new thing starting
>> > today!". I do, however,  think it is perhapsworth consideration at a
>> > somewhat narrower and more immediate scale than it would be otherwise.
>>
>> I hope you can see that I really like the general idea and think the
>> usage cases you are describing are really important.  I think I can
>> speak for the project in saying that we want the notebook to be useful
>> for things like this.  But I think our abstractions are important
>> enough that we make every attempt to see how we can do these while
>> leveraging our existing abstractions.  This is partially a question
>> about implementation, but also partly a question about how the new
>> features are thought about.  The reason we don't like to break
>> abstractions for new features is that we have found an interesting
>> relationship between abstraction breaking and new features.  We have
>> found that when a new feature/idea breaks a core abstraction that we
>> have thought about very carefully, it is usually because the feature
>> has not been fully understood.  Time and time again, we have found
>> that when we take the time to fully understand the feature, it usually
>> fits within our abstractions beautifully and is even much better that
>> we ever imagined it could be.
>>
>> The plugin idea above is a perfect example of this.  By preserving the
>> abstractions the new feature itself a multiplication of even new
>> functionality:
>>
>> * The resulting notebooks can still be version controlled.  This means
>> that the different alt-cell can be thrown into git and when we develop
>> a visual diff tool for notebooks, they will *just work*.
>> * The notebooks can immediately leverage the abstractions we have put
>> into place for converting notebooks to different formats.  You could
>> write custom transformers to present the notebook in a reveal.js
>> giving alt-cells special treatment.
>> * All of this can be done, and into the hands of user, without going
>> through those overly conservative IPython developers ;-)
>> * It will just work with nbviewer as well.
>> * It provides a cleanly abstracted foundation for other people to build
>> upon
>>
>> In summary, we are trying to build an architecture that allows a few
>> simple abstractions (we actually don't have that many!) to combine in
>> boundless ways to create features we never planned on, but that "just
>> work".
>>
>> The upside of this, is that when we have encountered features that are
>> important to us that really do require us to break or re-vision core
>> abstractions - we gladly undertake this work.  Mainly because we feel
>> that the new abstractions will be even more powerful.
>>
>> >>
>> >> <snip>
>> >>
>> >>
>> >> Thinking about your proposed feature from this perspective: both the
>> >> task cells and alt cells introduce hierarchy and nesting into the
>> >> notebook.  This breaks our core abstraction that cells are not nested.
>> >>  In Jan-Feb our core development team had a discussion about this
>> >> abstraction exactly.  We decided that we definitely don't want to move
>> >> in the direction of allowing nesting in the notebook.  Because of this
>> >> we are in the process of removing the 1 level of nesting our notebook
>> >> format currently has, namely worksheets.  So for us, it is not just
>> >> about complexity - it is about breaking the abstractions.
>> >
>> >
>> > I do understand this position. I'd like to think I am bringing up points
>> > not
>> > raised during that meeting, but whether or not that is the case
>> > abstractions
>> > ARE important.
>> >
>> > I guess I am/was thinking about the abstraction in place in IPython
>> > notebook
>> > a bit differently than you are describing it.
>> >
>> > For the next few paragraphs: process == (render|transform|execute|*)
>> >
>> > In my mind the abstraction/computational model is that a notebook is an
>> > ordered set of cells and to process the notebook you simply go through
>> > the
>> > cells in order and process them. What process means is dependent on the
>> > type
>> > of cell, and there are various pieces of code in various places (mostly
>> > the
>> > frontends and nbconvert AFAIK) that know how to handle each cell type.
>> >
>> > Under this formulation the change in abstraction is actually pretty
>> > small.
>> > The only addition is the statement that code which processes cells is
>> > responsible for initiating/handling the processing of any child cells
>> > those
>> > cells contain. The easy easiest example of this is the execute method on
>> > my
>> > task cells, which simply loops through each of its children and (if
>> > applicable) calls their execute method.
>> >
>> > With this change we still have a notebook defined as an ordered set of
>> > (top
>> > level) cells, and we can still process a notebook by stepping through
>> > each
>> > of them in order and processing that cell.
>> >
>> > Some changes to the concept of next/previous cells and cell position
>> > (for
>> > positional insertion, etc) were required and cells must be aware of
>> > their
>> > direct parent (which will be either a cell or the notebook itself), but
>> > I
>> > would argue these aren't actually important attributes of the
>> > abstraction
>> > itself and the changes were actually fairly narrow and (AFAICS) pretty
>> > painless and straightforward after some careful though/planning.
>>
>> This is an interesting way of thinking about nesting that I had not
>> thought about.
>>
>> >
>> >>
>> >> The reason that these abstractions are so important is that they
>> >> provide powerful foundations for us to build on.  One place the
>> >> "notebook as a linear sequence of cell" abstraction comes into play is
>> >> in our work on nbconvert that will appear in 1.0 in the next few
>> >> weeks.  This allows to to convert notebooks very easily to a number of
>> >> different formats.
>> >
>> >
>> > I haven't tackled nbconvert yet on my experimental fork, but I fully
>> > intend
>> > to as I agree entirely that the ability to generate things like linear
>> > pdfs
>> > and other static views is utterly crucial. The fact that a notebook with
>> > branches can generate a pdf that looks like it came from a linear
>> > notebook
>> > (ie the "static article" view) is a *major* selling point/core feature
>> > of
>> > what I'm trying to do with branching notebooks. It is key that people be
>> > able to meet the needs they are meeting now; if we can't, meeting the
>> > more
>> > nebulous needs they aren't isn't likely to save us (me) from
>> > irrelevance.
>> >
>> > Under my alternate description of the computational model described
>> > above
>> > nbconvert will behave pretty much as it always has: step through the
>> > notebook and process the cells in order into whatever format you are
>> > targeting. The one exception is the cells processing their children, but
>> > the
>> > scale of this change is not particularly large for the specific types of
>> > nesting I'm going for.
>> >
>> > Tasks would likely simply render their children without making any mark
>> > themselves in most cases, while altsets would do the same, but only for
>> > the
>> > "active" branch. This involves a bit of looping and a bunch of calls to
>> > existing code that knows how to transform the existing (core) cell
>> > types,
>> > but really not much else.
>> >
>> >
>> >
>> >>
>> >>  The other place this abstraction comes into play
>> >> is in our keyboard shortcuts.  We are striving for the notebook to be
>> >> usable for people who dont' touch the mouse (your traditional vi/emacs
>> >> users).  Nesting makes that very difficult.
>> >
>> >
>> > I admit this one is tougher, though I've done some small amount thinking
>> > about it (currently hitting "down" on a container cell enters it while
>> > hitting down on the last cell in a container navigates to the cell
>> > "after"
>> > the container in my PoC).
>> >
>> > I think this is surmountable though, and worth the effort if it were the
>> > only thing holding IPython notebook back from offering a
>> > change/alternative/"fix" to how we talk about research and what we can
>> > do
>> > with the documents we use to describe it.
>> >
>> > Wow that was a lot of text. Thanks for making it all the way to the end!
>>
>> I made it!
>>
>> Cheers,
>>
>> Brian
>>
>> > ~G
>> >
>> >
>> >> <snip>
>> >>
>> >
>> >
>> > --
>> > Gabriel Becker
>> > Graduate Student
>> > Statistics Department
>> > University of California, Davis
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From tarun.gaba7 at gmail.com  Sun Jul 14 03:25:17 2013
From: tarun.gaba7 at gmail.com (TARUN GABA)
Date: Sun, 14 Jul 2013 12:55:17 +0530
Subject: [IPython-dev] container Javascript object not available
Message-ID: <CAHAono2b3EM4S3u2OfBxzNgdL4kXBj6sQ6dBzg=RWVwhT4_sCw@mail.gmail.com>

Hi,
I am trying to render some webgls in IPython notebook,
My code is on these lines ..

function Canvas(arg) { this.initialize = function() { container.show();
..... }}

then I call
var new_canvas = new Canvas(arg);
new_canvas.initialize();

I get this error ..
ReferenceError: container is not defined

Any Ideas what might be wrong.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130714/eaa50efb/attachment.html>

From aheilbut at gmail.com  Sun Jul 14 03:30:07 2013
From: aheilbut at gmail.com (Adrian Heilbut)
Date: Sun, 14 Jul 2013 03:30:07 -0400
Subject: [IPython-dev] container Javascript object not available
In-Reply-To: <CAHAono2b3EM4S3u2OfBxzNgdL4kXBj6sQ6dBzg=RWVwhT4_sCw@mail.gmail.com>
References: <CAHAono2b3EM4S3u2OfBxzNgdL4kXBj6sQ6dBzg=RWVwhT4_sCw@mail.gmail.com>
Message-ID: <-1087202440560448429@unknownmsgid>

Just an idea.. But container is not defined in the scope where it's called.

On Jul 14, 2013, at 3:25 AM, TARUN GABA <tarun.gaba7 at gmail.com> wrote:

Hi,
I am trying to render some webgls in IPython notebook,
My code is on these lines ..

function Canvas(arg) { this.initialize = function() { container.show();
..... }}

then I call
var new_canvas = new Canvas(arg);
new_canvas.initialize();

I get this error ..
ReferenceError: container is not defined

Any Ideas what might be wrong.

_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130714/ad96cd4f/attachment.html>

From nborwankar at gmail.com  Sun Jul 14 19:46:19 2013
From: nborwankar at gmail.com (Nitin Borwankar)
Date: Sun, 14 Jul 2013 16:46:19 -0700
Subject: [IPython-dev] =?windows-1252?q?Storing_data_=85?=
In-Reply-To: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>
References: <D7E17FC9-C828-44B8-9EB9-17F18681553A@lbl.gov>
Message-ID: <CAAhmvMKkfrYpZGM4PEpcDj5dLNkCp7yECg=OHDhQhVzHL5Gd5A@mail.gmail.com>

I use a git repo with directories off the root named

.../notebooks
.../notebooks/images
.../notebooks/data

Then inside the notebooks I can access the data with relative paths.
When someone else clones the git repo the paths remain valid.
I use github as the exchange point for sharing the repo with
notebooks+data+images ...

This has worked well so far.
the images and data dirs could also be off the root, but this scheme worked
better for me.

Nitin Borwankar




------------------------------------------------------------------
Nitin Borwankar
nborwankar at gmail.com


On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:

> Hi
>
> I'd like to associate data generated within the notebook with the actual
> notebook for that session.
>
> Right now I'm writing files to the file system, but that places the
> additional burden of having the manage the data storage.
>
> [1] How are people dealing with the task of storing data (which may be
> specific to that ipython notebook, or even better, specific to the
> notebook's session).
>
> [2] Is there something (existing or planned) in the API that allows me to
> store session level data in the ipynb?
>
> Thanks,
>
> F
>
> Francis Reyes
> Advanced Light Source
> 1 Cyclotron Road
> Berkeley, CA 94720
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130714/69e73280/attachment.html>

From jkibele at gmail.com  Sun Jul 14 21:36:33 2013
From: jkibele at gmail.com (Jared Kibele)
Date: Mon, 15 Jul 2013 13:36:33 +1200
Subject: [IPython-dev] Using upstart to start ipython notebook.
Message-ID: <CAK_h3epgDvfEbFYbodtdzjObNPzTv_uxwafh+g3NOn=_KOhf3w@mail.gmail.com>

Hi everyone,

I'm running iPython notebook on an Amazone EC2 instance. Everything works
fine when I start ipython from ssh but when I close the ssh connection, the
notebook server dies. It seems like it would be better to use upstart to
manage the notebook server but I can't quite get it working. I think I'm
having the problem mentioned
here<http://python.6.x6.nabble.com/Notebook-doesn-t-load-profile-when-started-on-boot-td4976770.html>.
I think upstart isn't starting as the correct user and therefore is failing
to find my ipython profile. Here's my upstart conf:

start on filesystem

stop on runlevel [!2345]

respawn
exec sudo -u ubuntu -i ipython notebook --profile=nbserver
--ipython-dir=/home/ubuntu


This is my first time attempting to use upstart and I'm finding the
documentation a bit overwhelming. Can anyone help set me straight and tell
me what's wrong with my conf file?

Thanks,
Jared Kibele
PhD Candidate
Leigh Marine Lab
University of Auckland
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130715/bf392af5/attachment.html>

From jkibele at gmail.com  Sun Jul 14 22:01:21 2013
From: jkibele at gmail.com (Jared Kibele)
Date: Mon, 15 Jul 2013 14:01:21 +1200
Subject: [IPython-dev] Using upstart to start ipython notebook.
In-Reply-To: <CAK_h3epgDvfEbFYbodtdzjObNPzTv_uxwafh+g3NOn=_KOhf3w@mail.gmail.com>
References: <CAK_h3epgDvfEbFYbodtdzjObNPzTv_uxwafh+g3NOn=_KOhf3w@mail.gmail.com>
Message-ID: <CAK_h3eqn-Wy0pg8Nzy3WvYiwtERJW3kgELVUdoWjDJ2dBT2YmQ@mail.gmail.com>

Hello again,

I found part of my problem (about 2 minutes after sending my previous
email). An earlier attempt at running a slightly different version of my
conf file resulted in the creation of another profile_nbserver directory.
That profile was being read instead of the one I intended. I deleted that
one, and then my conf file was able to start my notebook server and I was
able to connect.

I have a new (related?) problem now. When I connect, the IPython dashboard
looks different (no kernel tab) and none of my existing notebooks load.
When I create a new notebook, I'm unable to import scipy. My upstart log
file contains the following: "WARNING:root:Invalid cookie signature". Then
when I try to open the existing notebooks, it says: "WARNING:root:500 GET
/notebooks/3ae6-28e5-4ef3-a5e8-fae2f6c8373f?_=2761383137552
(ipaddress.removed): Unreadable JSON notebook."

Can anyone help me out? Sorry about any extra confusion from my initial
question.


On Mon, Jul 15, 2013 at 1:36 PM, Jared Kibele <jkibele at gmail.com> wrote:

> Hi everyone,
>
> I'm running iPython notebook on an Amazone EC2 instance. Everything works
> fine when I start ipython from ssh but when I close the ssh connection, the
> notebook server dies. It seems like it would be better to use upstart to
> manage the notebook server but I can't quite get it working. I think I'm
> having the problem mentioned here<http://python.6.x6.nabble.com/Notebook-doesn-t-load-profile-when-started-on-boot-td4976770.html>.
> I think upstart isn't starting as the correct user and therefore is failing
> to find my ipython profile. Here's my upstart conf:
>
> start on filesystem
>
> stop on runlevel [!2345]
>
> respawn
> exec sudo -u ubuntu -i ipython notebook --profile=nbserver
> --ipython-dir=/home/ubuntu
>
>
> This is my first time attempting to use upstart and I'm finding the
> documentation a bit overwhelming. Can anyone help set me straight and tell
> me what's wrong with my conf file?
>
> Thanks,
> Jared Kibele
> PhD Candidate
> Leigh Marine Lab
> University of Auckland
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130715/86af838f/attachment.html>

From takowl at gmail.com  Mon Jul 15 07:50:59 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 15 Jul 2013 12:50:59 +0100
Subject: [IPython-dev] Using upstart to start ipython notebook.
In-Reply-To: <CAK_h3eqn-Wy0pg8Nzy3WvYiwtERJW3kgELVUdoWjDJ2dBT2YmQ@mail.gmail.com>
References: <CAK_h3epgDvfEbFYbodtdzjObNPzTv_uxwafh+g3NOn=_KOhf3w@mail.gmail.com>
	<CAK_h3eqn-Wy0pg8Nzy3WvYiwtERJW3kgELVUdoWjDJ2dBT2YmQ@mail.gmail.com>
Message-ID: <CAOvn4qhBaYCUqqVoRsARPOSiLvdjv8wCfZet879gsubDXM4uxw@mail.gmail.com>

On 15 July 2013 03:01, Jared Kibele <jkibele at gmail.com> wrote:

> the IPython dashboard looks different ... Unreadable JSON notebook."
>

That sounds like you have an older version of IPython installed on your EC2
instance. It sounds like it's running Ubuntu - if you've installed IPython
through apt, then you can add this PPA and update to get the latest stable
release of IPython:
https://launchpad.net/~jtaylor/+archive/ipython

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130715/901fad17/attachment.html>

From jtaylor.debian at googlemail.com  Mon Jul 15 13:25:33 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Mon, 15 Jul 2013 19:25:33 +0200
Subject: [IPython-dev] Using upstart to start ipython notebook.
In-Reply-To: <CAOvn4qhBaYCUqqVoRsARPOSiLvdjv8wCfZet879gsubDXM4uxw@mail.gmail.com>
References: <CAK_h3epgDvfEbFYbodtdzjObNPzTv_uxwafh+g3NOn=_KOhf3w@mail.gmail.com>
	<CAK_h3eqn-Wy0pg8Nzy3WvYiwtERJW3kgELVUdoWjDJ2dBT2YmQ@mail.gmail.com>
	<CAOvn4qhBaYCUqqVoRsARPOSiLvdjv8wCfZet879gsubDXM4uxw@mail.gmail.com>
Message-ID: <51E4308D.7020001@googlemail.com>

On 15.07.2013 13:50, Thomas Kluyver wrote:
> On 15 July 2013 03:01, Jared Kibele <jkibele at gmail.com
> <mailto:jkibele at gmail.com>> wrote:
> 
>     the IPython dashboard looks different ... Unreadable JSON notebook."
> 
> 
> That sounds like you have an older version of IPython installed on your
> EC2 instance. It sounds like it's running Ubuntu - if you've installed
> IPython through apt, then you can add this PPA and update to get the
> latest stable release of IPython:
> https://launchpad.net/~jtaylor/+archive/ipython
> 

currently the ppa should not be necessary, all (desktop) supported
releases have ipython 0.13.2 in the backports repository.

e.g. for precise (12.04):
apt-get install ipython-notebook -t precise-backports



From jkibele at gmail.com  Mon Jul 15 19:02:28 2013
From: jkibele at gmail.com (Jared Kibele)
Date: Tue, 16 Jul 2013 11:02:28 +1200
Subject: [IPython-dev] Using upstart to start ipython notebook.
In-Reply-To: <51E4308D.7020001@googlemail.com>
References: <CAK_h3epgDvfEbFYbodtdzjObNPzTv_uxwafh+g3NOn=_KOhf3w@mail.gmail.com>
	<CAK_h3eqn-Wy0pg8Nzy3WvYiwtERJW3kgELVUdoWjDJ2dBT2YmQ@mail.gmail.com>
	<CAOvn4qhBaYCUqqVoRsARPOSiLvdjv8wCfZet879gsubDXM4uxw@mail.gmail.com>
	<51E4308D.7020001@googlemail.com>
Message-ID: <CAK_h3eqpgtBzQ3F+fYzt2d1md7C5ZvZDoyo0c0-eFvR2=YegQg@mail.gmail.com>

Thanks for the replies. I set up my EC2 instance following this tutorial:
https://gist.github.com/iamatypeofwalrus/5183133 so I'm using the Anaconda
python package. Everything is fine when I ssh in and start up my notebook
server as the ubuntu user. I think the problem with my upstart config is
that it's not really running as the ubuntu user. Specifically, I think it's
not running the ubuntu user's .bashrc file and is therefore not getting the
Anaconda python stuck on to the front of the PATH variable. I thought that
the '-i' part of 'sudo -u ubuntu -i' in my upstart conf file was supposed
to make sure that happened but maybe not. Any thoughts on how I can make
sure that upstart is starting my server as ubuntu rather than root?

-Jared


On Tue, Jul 16, 2013 at 5:25 AM, Julian Taylor <
jtaylor.debian at googlemail.com> wrote:

> On 15.07.2013 13:50, Thomas Kluyver wrote:
> > On 15 July 2013 03:01, Jared Kibele <jkibele at gmail.com
> > <mailto:jkibele at gmail.com>> wrote:
> >
> >     the IPython dashboard looks different ... Unreadable JSON notebook."
> >
> >
> > That sounds like you have an older version of IPython installed on your
> > EC2 instance. It sounds like it's running Ubuntu - if you've installed
> > IPython through apt, then you can add this PPA and update to get the
> > latest stable release of IPython:
> > https://launchpad.net/~jtaylor/+archive/ipython
> >
>
> currently the ppa should not be necessary, all (desktop) supported
> releases have ipython 0.13.2 in the backports repository.
>
> e.g. for precise (12.04):
> apt-get install ipython-notebook -t precise-backports
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130716/143cfdac/attachment.html>

From bussonniermatthias at gmail.com  Tue Jul 16 02:39:19 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 16 Jul 2013 08:39:19 +0200
Subject: [IPython-dev] IPython Drinkup in Paris ?
In-Reply-To: <871B3EE7-E239-459A-A9FF-A44203B198D9@gmail.com>
References: <85010AA8-5348-4777-86E2-3FE0625BBD51@gmail.com>
	<CAHAreOqP4qq-MDYSDsvUvfV9bNxkRfg_-gG5-ufStPU3iM7rCA@mail.gmail.com>
	<CAE-UAvQ4S933St1GBfE++rTr9zzO8XGcj1ss_4B9540S7MQ8dg@mail.gmail.com>
	<463206CE-A670-4C77-8B60-8AEC16D98A80@gmail.com>
	<CAHAreOp3FaO0kiGXidisqm6QfCwYXonVeHoBU7ZggY9zErw1vQ@mail.gmail.com>
	<CAP-uhDfbu-7vz9UjCqdm2C6fAhjRHW1R4WxfAzTeSJg1jupU2Q@mail.gmail.com>
	<3452DDA7-3ADA-4D4B-94BB-03B00474E082@gmail.com>
	<CAE-UAvS-qVX+mYTrZzP4t6w8A9-d=MfVFU_LY+eUD7HTX3HXzw@mail.gmail.com>
	<871B3EE7-E239-459A-A9FF-A44203B198D9@gmail.com>
Message-ID: <E6060EE9-3943-4F00-888C-4B78798CCEFE@gmail.com>

Hi all, 

Thanks to all those who came yesterday, apparently some peoples couldn't make it others tried to call us by phone,
unfortunately the cellular network appeared to be down around the bar and we received the messages only later.

Hope we can organize another thing like that later.

Cheers, 
-- 
Matthias

Le 13 juil. 2013 ? 00:47, Matthias BUSSONNIER a ?crit :

> 
> Le 13 juil. 2013 ? 00:41, Nelle Varoquaux a ?crit :
>> 
>> Around what time ?
> 
> I'll be there around 19h I guess. 
> That what's I wrote on doodle. 
> 
> Just in case, my phone number (in a non-machin readable form).
> 9661?70039 (with a leading 0, or +33)
> -- 
> Matthias



From dpsanders at gmail.com  Tue Jul 16 11:56:27 2013
From: dpsanders at gmail.com (David P. Sanders)
Date: Tue, 16 Jul 2013 10:56:27 -0500
Subject: [IPython-dev] First experiences and usability suggestions for
	notebook and nbconvert
Message-ID: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>

Hi,

I have been playing with the notebook and nbconvert from master, and have
some feedback on some basic usability issues, which I hope will be useful.

Note that I am not (yet) a developer, but am rather trying to get a feel
for things before possibly moving in that direction...

Please let me know if this is not the correct forum for such comments, and
if I should, for example, instead open issues.


1. Ipython command line invocation
- Would it be possible to get rid of the hyphens in --pylab and --profile
in the ipython invocation? I understand that this is to do with
implementation, but it is neither easy to remember, nor to motivate to
novices that they should type this!


- Is there a good (non-historical) reason to use 'nbconvert' instead of
just
ipython convert TARGET FILENAME?
Or even convertnb,  so that the verb and noun are in the natural order.

Or make nbconvert just a separate binary (which, I understand, it basically
is anyway, at least on unices):

nbconvert TARGET FILENAME?


-  Is it possible / easy to set up bash completion to press <TAB> after
'ipython' and have the options displayed?


- It would great to be able to type
ipython notebook odes.ipynb
and have the relevant ipynb open directly, instead of going via the
dashboard


2. nbconvert:
- It would be great (necessary, in my opinion) to have an 'Export as'
option from within the notebook to run the relevant nbconvert.


- PDF converter:
A direct PDF converter is necessary.
This is presumably trivial by just running
pdflatex after the LaTeX converter.
(Oh, if LaTeX is installed of course... I guess this is the problem?)


- html converter:
I tried
ipython nbconvert html odes.ipynb

but it doesn't work!

I strongly suggest to replace 'full_html' with just 'html'
and 'simple_html' could be 'basic_html'

- I was expecting that the HTML output would look exactly like on nbviewer,
which is also a static copy of the notebook, but it looks different
I guess this is a CSS thing -- is there a good solution to have something
exactly like on nbviewer?

- The nbconvert step produced an empty subdirectory 'odes_files' with html
export and some .txt files for LaTeX output. This seems unnecessary from
the user point of view.


- Folding:
In my notebook, the output of several cells were folded, in particular the
result of things like
help(odeint)
[I am planning on using this for a course, and wanted to record this step
in the notebook]
But in the result of nbconvert, the full output of the documentation is
shown.

This is very bad news -- how can the output folding be respected and not
shown?


- New LaTeX commands:
I defined various LaTeX shortcuts like
\newcommand{\yy}{\mathbf{y}}
inside $$...$$ in markdown cells.
(Is this the correct / canonical way to do this?)

Markdown seems to respect these in the rest of the document, and in the
HTML output, but running
pdflatex on the LaTeX output forgot about them immediately after they were
defined.


- Suppressing code cells in LaTeX output:
Towards the reproducible paper goal, how can code cells be suppressed in
the LaTeX output. I guess with the metadata somehow?


3. Notebook issues:
- Keyboard shortcuts:
I am a keyboard person: for me, all functionality which is accessible via
menus should be assigned a (preferably modifiable) keyboard shortcut.
In particular, it seems that split cell, merge cell, and close and halt are
currently missing shortcuts.

- Similarly, could there be toolbar icons for split and merge?

- Basic find and replace in the notebook:
Find and replace, at least a basic one, is a must-have feature for
refactoring.
This seems to already be available in CodeMirror

-  In-place file editing
It seems that by combining the %load and %%file magics,
one has a kludgy way of editing an arbitrary file.
Maybe this could be combined easily into an %%edit magic?

- Is there a way of deleting multiple input cells simultaneously, rather
than pressing Ctrl M-d several times or clicking the scissors icon several
times (which admittedly is easier in this case). This would be very nice to
have


4. Notebook styling:
- There seems to be much appetite for notebook styling, e.g. xkcdify.

Here is a gist I found, referenced from a blog post that Brian commented on
at some point, with a very basic interface. Something like this would be
great to have, with a user-contributed style database:

https://gist.github.com/aflaxman/5707238

Maybe this could just be via load_ext?

By the way, is there a list of the possible extensions available?

- Will Min's nbtoc extension be integrated?


Apologies for the long message, but these are the things that have been
bugging me for the last few days :)

Thanks and best wishes,
David.


-- 
**************************************************************************
Dr. David P. Sanders

Profesor Titular A / Associate Professor
Departamento de F?sica, Facultad de Ciencias
Universidad Nacional Aut?noma de M?xico (UNAM)

dpsanders at gmail.com
http://sistemas.fciencias.unam.mx/~dsanders

Cub?culo / office: #414
Tel.: +52 55 5622 4965
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130716/487ab363/attachment.html>

From dpsanders at gmail.com  Tue Jul 16 13:33:23 2013
From: dpsanders at gmail.com (David P. Sanders)
Date: Tue, 16 Jul 2013 12:33:23 -0500
Subject: [IPython-dev] First experiences and usability suggestions for
	notebook and nbconvert
In-Reply-To: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
References: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
Message-ID: <CAFT+uJkkNwPfLFR0x2O=EWxLAYKYzy+qaDgOpiixp8zDCTxtdQ@mail.gmail.com>

>
> - It would great to be able to type
> ipython notebook odes.ipynb
> and have the relevant ipynb open directly, instead of going via the
> dashboard
>
>
Further to this comment:

At the moment, my default behaviour is *not* usually to want a list of the
notebooks in the current directory, but simply to start writing stuff in
the notebook, just like I would if I ran 'ipython' from the command line.

Maybe there could be an explicit option

ipython notebook dashboard

or

ipython notebook list

to get the dashboard behaviour.


[--snip]


> - Folding:
> In my notebook, the output of several cells were folded, in particular the
> result of things like
> help(odeint)
> [I am planning on using this for a course, and wanted to record this step
> in the notebook]
> But in the result of nbconvert, the full output of the documentation is
> shown.
>
> This is very bad news -- how can the output folding be respected and not
> shown?
>


This seems to be a more general problem: if I hide the output of a cell in
the notebook, save and close, and reopen the same notebook, then the output
appears in full. This seems to me to be an actual bug. I will (look for
and) open an issue.

David.



>
>
> - New LaTeX commands:
> I defined various LaTeX shortcuts like
> \newcommand{\yy}{\mathbf{y}}
> inside $$...$$ in markdown cells.
> (Is this the correct / canonical way to do this?)
>
> Markdown seems to respect these in the rest of the document, and in the
> HTML output, but running
> pdflatex on the LaTeX output forgot about them immediately after they were
> defined.
>
>
> - Suppressing code cells in LaTeX output:
> Towards the reproducible paper goal, how can code cells be suppressed in
> the LaTeX output. I guess with the metadata somehow?
>
>
> 3. Notebook issues:
> - Keyboard shortcuts:
> I am a keyboard person: for me, all functionality which is accessible via
> menus should be assigned a (preferably modifiable) keyboard shortcut.
> In particular, it seems that split cell, merge cell, and close and halt are
> currently missing shortcuts.
>
> - Similarly, could there be toolbar icons for split and merge?
>
> - Basic find and replace in the notebook:
> Find and replace, at least a basic one, is a must-have feature for
> refactoring.
> This seems to already be available in CodeMirror
>
> -  In-place file editing
> It seems that by combining the %load and %%file magics,
> one has a kludgy way of editing an arbitrary file.
> Maybe this could be combined easily into an %%edit magic?
>
> - Is there a way of deleting multiple input cells simultaneously, rather
> than pressing Ctrl M-d several times or clicking the scissors icon several
> times (which admittedly is easier in this case). This would be very nice to
> have
>
>
> 4. Notebook styling:
> - There seems to be much appetite for notebook styling, e.g. xkcdify.
>
> Here is a gist I found, referenced from a blog post that Brian commented on
> at some point, with a very basic interface. Something like this would be
> great to have, with a user-contributed style database:
>
> https://gist.github.com/aflaxman/5707238
>
> Maybe this could just be via load_ext?
>
> By the way, is there a list of the possible extensions available?
>
> - Will Min's nbtoc extension be integrated?
>
>
> Apologies for the long message, but these are the things that have been
> bugging me for the last few days :)
>
> Thanks and best wishes,
> David.
>
>
> --
> **************************************************************************
> Dr. David P. Sanders
>
> Profesor Titular A / Associate Professor
> Departamento de F?sica, Facultad de Ciencias
> Universidad Nacional Aut?noma de M?xico (UNAM)
>
> dpsanders at gmail.com
> http://sistemas.fciencias.unam.mx/~dsanders
>
> Cub?culo / office: #414
> Tel.: +52 55 5622 4965
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/ipython-dev/attachments/20130716/487ab363/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> End of IPython-dev Digest, Vol 114, Issue 38
> ********************************************
>



-- 
**************************************************************************
Dr. David P. Sanders

Profesor Titular A / Associate Professor
Departamento de F?sica, Facultad de Ciencias
Universidad Nacional Aut?noma de M?xico (UNAM)

dpsanders at gmail.com
http://sistemas.fciencias.unam.mx/~dsanders

Cub?culo / office: #414
Tel.: +52 55 5622 4965
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130716/bb3c7c6e/attachment.html>

From jon.freder at gmail.com  Tue Jul 16 13:45:33 2013
From: jon.freder at gmail.com (Jonathan Frederic)
Date: Tue, 16 Jul 2013 10:45:33 -0700
Subject: [IPython-dev] First experiences and usability suggestions for
 notebook and nbconvert
In-Reply-To: <CAFT+uJkkNwPfLFR0x2O=EWxLAYKYzy+qaDgOpiixp8zDCTxtdQ@mail.gmail.com>
References: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
	<CAFT+uJkkNwPfLFR0x2O=EWxLAYKYzy+qaDgOpiixp8zDCTxtdQ@mail.gmail.com>
Message-ID: <CAAoBLw1uMGZM44R7Lvv5Xwc0bLEro9iZOPPGpOfn-O4+S3AVTA@mail.gmail.com>

Hi David,

The team and I are working on these issues right now.  There is a pull
request (PR), under review, that will alleviate some of the nbconvert
problems you mention (see https://github.com/ipython/ipython/pull/3607)

>From the PR, this is how nbconvert will be called:

"

   ipython nbconvert notebook*.ipynb
   ipython nbconvert notebook1.ipynb notebook2.ipynb
   ipython nbconvert # this will use the config file to fill in the notebooks

"

I don't know all of the details, but I can provide some comments to your
email in-line with regard to nbconvert (as seen below).

Thank you so much for spending the time to send some feedback,
Sincerely,

Jon


On Tue, Jul 16, 2013 at 8:56 AM, David P. Sanders <dpsanders at gmail.com>wrote:

> Hi,
>
> I have been playing with the notebook and nbconvert from master, and have
> some feedback on some basic usability issues, which I hope will be useful.
>
> Note that I am not (yet) a developer, but am rather trying to get a feel
> for things before possibly moving in that direction...
>
> Please let me know if this is not the correct forum for such comments, and
> if I should, for example, instead open issues.
>
>
> 1. Ipython command line invocation
> - Would it be possible to get rid of the hyphens in --pylab and --profile
> in the ipython invocation? I understand that this is to do with
> implementation, but it is neither easy to remember, nor to motivate to
> novices that they should type this!
>
>
Flags like this can be set in config files if you need them every time you
work in a particular environment.



>
> - Is there a good (non-historical) reason to use 'nbconvert' instead of
> just
> ipython convert TARGET FILENAME?
> Or even convertnb,  so that the verb and noun are in the natural order.
>
> Or make nbconvert just a separate binary (which, I understand, it
> basically is anyway, at least on unices):
>
>
It was like this for a long time time prior to being merged into master
recently.


> nbconvert TARGET FILENAME?
>
>
> -  Is it possible / easy to set up bash completion to press <TAB> after
> 'ipython' and have the options displayed?
>
>
> - It would great to be able to type
> ipython notebook odes.ipynb
> and have the relevant ipynb open directly, instead of going via the
> dashboard
>
>
> 2. nbconvert:
> - It would be great (necessary, in my opinion) to have an 'Export as'
> option from within the notebook to run the relevant nbconvert.
>
>
That is planned, but not for 1.0


>
> - PDF converter:
> A direct PDF converter is necessary.
> This is presumably trivial by just running
> pdflatex after the LaTeX converter.
> (Oh, if LaTeX is installed of course... I guess this is the problem?)
>
>
We were think of generating and writing a make file to the output directory
that would build the pdf.


>
> - html converter:
> I tried
> ipython nbconvert html odes.ipynb
>
> but it doesn't work!
>
> I strongly suggest to replace 'full_html' with just 'html'
> and 'simple_html' could be 'basic_html'
>

Ah, good suggestion, I'll see what the others think of this.


>
> - I was expecting that the HTML output would look exactly like on
> nbviewer, which is also a static copy of the notebook, but it looks
> different
> I guess this is a CSS thing -- is there a good solution to have something
> exactly like on nbviewer?
>
>
I don't know the answer to that, Matthias may know.  He is actively
developing nbviewer and I believe he wrote the html templates that exist
now...

- The nbconvert step produced an empty subdirectory 'odes_files' with html
> export and some .txt files for LaTeX output. This seems unnecessary from
> the user point of view.
>
>
This will change with the PR


> - Folding:
> In my notebook, the output of several cells were folded, in particular the
> result of things like
> help(odeint)
> [I am planning on using this for a course, and wanted to record this step
> in the notebook]
> But in the result of nbconvert, the full output of the documentation is
> shown.
>
> This is very bad news -- how can the output folding be respected and not
> shown?
>
>
I don't think a pre-canned feature exists to do this at the moment.  It can
be done by writing a transformer.


>
> - New LaTeX commands:
> I defined various LaTeX shortcuts like
> \newcommand{\yy}{\mathbf{y}}
> inside $$...$$ in markdown cells.
> (Is this the correct / canonical way to do this?)
>
> Markdown seems to respect these in the rest of the document, and in the
> HTML output, but running
> pdflatex on the LaTeX output forgot about them immediately after they were
> defined.
>
>
This will have to be looked into.  I don't know why the Latex isn't being
passed through into the template.


>
> - Suppressing code cells in LaTeX output:
> Towards the reproducible paper goal, how can code cells be suppressed in
> the LaTeX output. I guess with the metadata somehow?
>
>
A transformer or a custom template

______________________________
>
> _________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



On Tue, Jul 16, 2013 at 10:33 AM, David P. Sanders <dpsanders at gmail.com>wrote:

>
>
>> - It would great to be able to type
>> ipython notebook odes.ipynb
>> and have the relevant ipynb open directly, instead of going via the
>> dashboard
>>
>>
> Further to this comment:
>
> At the moment, my default behaviour is *not* usually to want a list of the
> notebooks in the current directory, but simply to start writing stuff in
> the notebook, just like I would if I ran 'ipython' from the command line.
>
> Maybe there could be an explicit option
>
> ipython notebook dashboard
>
> or
>
> ipython notebook list
>
> to get the dashboard behaviour.
>
>
> [--snip]
>
>
>> - Folding:
>> In my notebook, the output of several cells were folded, in particular the
>> result of things like
>> help(odeint)
>> [I am planning on using this for a course, and wanted to record this step
>> in the notebook]
>> But in the result of nbconvert, the full output of the documentation is
>> shown.
>>
>> This is very bad news -- how can the output folding be respected and not
>> shown?
>>
>
>
> This seems to be a more general problem: if I hide the output of a cell in
> the notebook, save and close, and reopen the same notebook, then the output
> appears in full. This seems to me to be an actual bug. I will (look for
> and) open an issue.
>
> David.
>
>
>
>>
>>
>> - New LaTeX commands:
>> I defined various LaTeX shortcuts like
>> \newcommand{\yy}{\mathbf{y}}
>> inside $$...$$ in markdown cells.
>> (Is this the correct / canonical way to do this?)
>>
>> Markdown seems to respect these in the rest of the document, and in the
>> HTML output, but running
>> pdflatex on the LaTeX output forgot about them immediately after they were
>> defined.
>>
>>
>> - Suppressing code cells in LaTeX output:
>> Towards the reproducible paper goal, how can code cells be suppressed in
>> the LaTeX output. I guess with the metadata somehow?
>>
>>
>> 3. Notebook issues:
>> - Keyboard shortcuts:
>> I am a keyboard person: for me, all functionality which is accessible via
>> menus should be assigned a (preferably modifiable) keyboard shortcut.
>> In particular, it seems that split cell, merge cell, and close and halt
>> are
>> currently missing shortcuts.
>>
>> - Similarly, could there be toolbar icons for split and merge?
>>
>> - Basic find and replace in the notebook:
>> Find and replace, at least a basic one, is a must-have feature for
>> refactoring.
>> This seems to already be available in CodeMirror
>>
>> -  In-place file editing
>> It seems that by combining the %load and %%file magics,
>> one has a kludgy way of editing an arbitrary file.
>> Maybe this could be combined easily into an %%edit magic?
>>
>> - Is there a way of deleting multiple input cells simultaneously, rather
>> than pressing Ctrl M-d several times or clicking the scissors icon several
>> times (which admittedly is easier in this case). This would be very nice
>> to
>> have
>>
>>
>> 4. Notebook styling:
>> - There seems to be much appetite for notebook styling, e.g. xkcdify.
>>
>> Here is a gist I found, referenced from a blog post that Brian commented
>> on
>> at some point, with a very basic interface. Something like this would be
>> great to have, with a user-contributed style database:
>>
>> https://gist.github.com/aflaxman/5707238
>>
>> Maybe this could just be via load_ext?
>>
>> By the way, is there a list of the possible extensions available?
>>
>> - Will Min's nbtoc extension be integrated?
>>
>>
>> Apologies for the long message, but these are the things that have been
>> bugging me for the last few days :)
>>
>> Thanks and best wishes,
>> David.
>>
>>
>> --
>> **************************************************************************
>> Dr. David P. Sanders
>>
>> Profesor Titular A / Associate Professor
>> Departamento de F?sica, Facultad de Ciencias
>> Universidad Nacional Aut?noma de M?xico (UNAM)
>>
>> dpsanders at gmail.com
>> http://sistemas.fciencias.unam.mx/~dsanders
>>
>> Cub?culo / office: #414
>>
>> Tel.: +52 55 5622 4965
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.scipy.org/pipermail/ipython-dev/attachments/20130716/487ab363/attachment-0001.html
>>
>> ------------------------------
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>> End of IPython-dev Digest, Vol 114, Issue 38
>> ********************************************
>>
>
>
>
> --
> **************************************************************************
> Dr. David P. Sanders
>
> Profesor Titular A / Associate Professor
> Departamento de F?sica, Facultad de Ciencias
> Universidad Nacional Aut?noma de M?xico (UNAM)
>
> dpsanders at gmail.com
> http://sistemas.fciencias.unam.mx/~dsanders
>
> Cub?culo / office: #414
> Tel.: +52 55 5622 4965
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130716/dab86307/attachment.html>

From takowl at gmail.com  Tue Jul 16 13:58:46 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 16 Jul 2013 18:58:46 +0100
Subject: [IPython-dev] First experiences and usability suggestions for
 notebook and nbconvert
In-Reply-To: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
References: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
Message-ID: <CAOvn4qh+7sPYv=wYQSrLH8RhbQyJrQWrMHiOUZpRkO_9yi6p9A@mail.gmail.com>

On 16 July 2013 16:56, David P. Sanders <dpsanders at gmail.com> wrote:

> 1. Ipython command line invocation
> - Would it be possible to get rid of the hyphens in --pylab and --profile
> in the ipython invocation? I understand that this is to do with
> implementation, but it is neither easy to remember, nor to motivate to
> novices that they should type this!
>

It's not so much the implementation, as

- --foo is the normal way to pass command line arguments to many different
programs.
- You can also run a file with a command like 'ipython somescript', so
using the dashes means that we can have lots of options without clashing
with files people might want to run.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130716/92bb5cc3/attachment.html>

From damianavila at gmail.com  Tue Jul 16 16:22:24 2013
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Tue, 16 Jul 2013 17:22:24 -0300
Subject: [IPython-dev] First experiences and usability suggestions for
 notebook and nbconvert
In-Reply-To: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
References: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
Message-ID: <51E5AB80.8020603@gmail.com>

El 16/07/13 12:56, David P. Sanders escribi?:
> 3. Notebook issues:
> - Keyboard shortcuts:
> I am a keyboard person: for me, all functionality which is accessible 
> via menus should be assigned a (preferably modifiable) keyboard shortcut.
> In particular, it seems that split cell, merge cell, and close and 
> halt are currently missing shortcuts.
>
> - Similarly, could there be toolbar icons for split and merge?

Hi David,

Jonathan as already answered some nbconvert-related things...

Regarding the splitting issue... We have a large discussion some days 
ago about that, you can see the details here: 
https://github.com/ipython/ipython/pull/3571

To resume, the split shortcut is was merged (yesterday) in the current 
master. But not the merge shortcuts, maybe after the land of the 
dual-mode editor we can discuss again about that.

In the mean time you can use custom javascript files to add icons for 
the split and merge function to the toolbar: examples of custom 
extensions of this nature lives here: 
https://github.com/ipython-contrib/IPython-notebook-extensions

Hope it helps!

Saludos ;-)

Dami?n.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130716/0847802c/attachment.html>

From gager at ilsb.tuwien.ac.at  Wed Jul 17 02:11:08 2013
From: gager at ilsb.tuwien.ac.at (Jakob Gager)
Date: Wed, 17 Jul 2013 08:11:08 +0200
Subject: [IPython-dev] First experiences and usability suggestions for
 notebook and nbconvert
In-Reply-To: <CAAoBLw1uMGZM44R7Lvv5Xwc0bLEro9iZOPPGpOfn-O4+S3AVTA@mail.gmail.com>
References: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
	<CAFT+uJkkNwPfLFR0x2O=EWxLAYKYzy+qaDgOpiixp8zDCTxtdQ@mail.gmail.com>
	<CAAoBLw1uMGZM44R7Lvv5Xwc0bLEro9iZOPPGpOfn-O4+S3AVTA@mail.gmail.com>
Message-ID: <51E6357C.1000200@ilsb.tuwien.ac.at>

Hi David,

you did a great list of issues! May I comment on this one...

>
>     - New LaTeX commands:
>     I defined various LaTeX shortcuts like
>     \newcommand{\yy}{\mathbf{y}}
>     inside $$...$$ in markdown cells.
>     (Is this the correct / canonical way to do this?)
>
>     Markdown seems to respect these in the rest of the document, and in the HTML output, but running
>     pdflatex on the LaTeX output forgot about them immediately after they were defined.

The problem here, is that mathjax allows to embed the \newcommand call within a math environment 
(otherwise it wouldn't be recognized by mathjax at all!). However, original LaTeX does not allow such 
calls to be placed with in math environments. The markdown2latex filter does what it should do and 
converts plain markdown text to plain LaTeX text and math environments into math environments.

To get your tex files to compile simply remove the math environment around the \newcommand (in the .tex 
or the notebook) and it should work fine.

I'll add this to my github issue list #3603 of "upcomming issues with nbconvert".

Cheers
Jakob


From bussonniermatthias at gmail.com  Wed Jul 17 03:46:19 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 17 Jul 2013 09:46:19 +0200
Subject: [IPython-dev] First experiences and usability suggestions for
	notebook and nbconvert
In-Reply-To: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
References: <CAFT+uJmf7709NQvy0ZMaNN1nMaMrsixUV6ubL7hgN3k1n23eGQ@mail.gmail.com>
Message-ID: <431B23B4-0E13-4481-88B7-BD57CC664817@gmail.com>

Hi David, 

Le 16 juil. 2013 ? 17:56, David P. Sanders a ?crit :

> Hi,
> 
> I have been playing with the notebook and nbconvert from master, and have some feedback on some basic usability issues, which I hope will be useful.

It is always insightful to get fresh feedback, thanks a lot for that we really appreciate.


> Note that I am not (yet) a developer, but am rather trying to get a feel for things before possibly moving in that direction...
> 
> Please let me know if this is not the correct forum for such comments, and if I should, for example, instead open issues.
> 
> 
> 1. Ipython command line invocation
> - Would it be possible to get rid of the hyphens in --pylab and --profile in the ipython invocation? I understand that this is to do with implementation, but it is neither easy to remember, nor to motivate to novices that they should type this!

Thomas already answered , --pylab and  --profile can be placed anywhere and it is pretty common in unix world
to have long option with a double dash. Of course you can set them permanently in you profile as Jon said, but there 
is a particularity of --pylab. 

I would suggest not to start IPython in --pylab mode but to use the %pylab magic at top of your notebooks. 
First because un-pylabing a kernel is not possible
Second because your readers do not know you started with pylab (with this wa can say that --pylab make the notebook not-reproductible).




> 
> - Is there a good (non-historical) reason to use 'nbconvert' instead of just 
> ipython convert TARGET FILENAME?
> Or even convertnb,  so that the verb and noun are in the natural order.

Not really, but I would say that `convert` itself would be too confusing in discussion. 
I think there are already confusion due to the `notebook` term used for the server, the file, the application
 the js-client... etc etc. I'd like to avoid this with convert.


> Or make nbconvert just a separate binary (which, I understand, it basically is anyway, at least on unices):
> nbconvert TARGET FILENAME?

I would have prefered that , for various reason it has been decided not to. 
you could make a bash alias.


> -  Is it possible / easy to set up bash completion to press <TAB> after 'ipython' and have the options displayed?

Debian package does it. we should have an example script somewhere in the source tree.
IIRC it is a 2 step process that parses the output of `ipython --help-all` to get all the option
and gen generate the ipython completion file. 


> 
> - It would great to be able to type
> ipython notebook odes.ipynb
> and have the relevant ipynb open directly, instead of going via the dashboard

Won't enter into detail, but 1 not possible right now. 
2 , does not always make sense as the server can be on another computer.
It is (kind of) doable as a separate project. We will encourage anyone outside 
of the core team to write a CLI tool that control the notebook. 


> I strongly suggest to replace 'full_html' with just 'html'
> and 'simple_html' could be 'basic_html'

Yes, It will come in time. 

> 
> - I was expecting that the HTML output would look exactly like on nbviewer, which is also a static copy of the notebook, but it looks different
> I guess this is a CSS thing -- is there a good solution to have something exactly like on nbviewer?

We are polishing nbconvert API, after that I will use it on nbviewer. So the style will converge. 
Question of time. 


> - Folding:
> In my notebook, the output of several cells were folded, in particular the result of things like 
> help(odeint)
> [I am planning on using this for a course, and wanted to record this step in the notebook]
> But in the result of nbconvert, the full output of the documentation is shown.
> 
> This is very bad news -- how can the output folding be respected and not shown?

Might need js-extension in the notebook.
With custom templates and js.
Not too hard, just take time to do.

The question of having it in the core is different.

> 
> - Suppressing code cells in LaTeX output:
> Towards the reproducible paper goal, how can code cells be suppressed in the LaTeX output. I guess with the metadata somehow?

Pretty trivial 4 line template inheriting.
	- inherit the template you wish (1 line)
        - 1 blank line
       -  overwrite {block cell_input}{end block} to be empty
       -  overwrite {block input_prompt}{end block} to be empty
Use you template.


> 
> 
> 3. Notebook issues:
> - Keyboard shortcuts:
> I am a keyboard person: for me, all functionality which is accessible via menus should be assigned a (preferably modifiable) keyboard shortcut.
> In particular, it seems that split cell, merge cell, and close and halt are currently missing shortcuts.

We don't have enough available keys, we will have to make it configurable.

> 
> - Similarly, could there be toolbar icons for split and merge?

DIY, easy, ping if need help:
http://elacave.lmdb.eu/~carreau/yui/classes/IPython.customjs.html

> 
> - Basic find and replace in the notebook:
> Find and replace, at least a basic one, is a must-have feature for refactoring.
> This seems to already be available in CodeMirror.

SHoudl be doable with custom-js , if yo achieve to make an extension that does it, please PR.

> -  In-place file editing
> It seems that by combining the %load and %%file magics, 
> one has a kludgy way of editing an arbitrary file.
> Maybe this could be combined easily into an %%edit magic?

Difficult with current architecture, will come later.

> 
> - Is there a way of deleting multiple input cells simultaneously, rather than pressing Ctrl M-d several times or clicking the scissors icon several times (which admittedly is easier in this case). This would be very nice to have

Not yet, but we will go toward multi-cell selection.

> 
> 
> 4. Notebook styling:
> - There seems to be much appetite for notebook styling, e.g. xkcdify.
> 
> Here is a gist I found, referenced from a blog post that Brian commented on at some point, with a very basic interface. Something like this would be great to have, with a user-contributed style database:
> 
> https://gist.github.com/aflaxman/5707238
> 
> Maybe this could just be via load_ext?

You might have seen my comment on xkcd issue.


> By the way, is there a list of the possible extensions available?

On the wiki, 
also ,
I'm trying to regroup some js-extensions here : http://bit.ly/14Y7jul
(contribution welcomed)

> 
> - Will Min's nbtoc extension be integrated?

Not yet.



From markbak at gmail.com  Wed Jul 17 06:56:08 2013
From: markbak at gmail.com (Mark Bakker)
Date: Wed, 17 Jul 2013 12:56:08 +0200
Subject: [IPython-dev] 1.0 release plans
Message-ID: <CAEX=yaacWBYX-sBGNzNPC6yJvQUFGjg43TXLs8MU4fbERC3PRw@mail.gmail.com>

Hello List,

Sorry for this easy question (just joined the list).

When can we expect the 1.0 release? It was listed as July 15.

I really want to use the 1.0 version to teach an introductory programming
class, which starts September 2,

Thanks,

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130717/71dca7d9/attachment.html>

From bussonniermatthias at gmail.com  Wed Jul 17 07:02:32 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 17 Jul 2013 13:02:32 +0200
Subject: [IPython-dev] 1.0 release plans
In-Reply-To: <CAEX=yaacWBYX-sBGNzNPC6yJvQUFGjg43TXLs8MU4fbERC3PRw@mail.gmail.com>
References: <CAEX=yaacWBYX-sBGNzNPC6yJvQUFGjg43TXLs8MU4fbERC3PRw@mail.gmail.com>
Message-ID: <8513CBE6-AD9B-412E-93B2-5EEDD70D12C7@gmail.com>


Le 17 juil. 2013 ? 12:56, Mark Bakker a ?crit :

> Hello List,
> 
> Sorry for this easy question (just joined the list).
> 
> When can we expect the 1.0 release? It was listed as July 15.

We'll try to have an alpha on Friday. (not sure if we can)
We will probably discuss this on live hangout Thursday at 10 PST on youtube. 
Link to hangout are usually posted on twitter/g+/hipchat.
Once the alpha is released I would count 2 to 3 weeks for the final 1.0.
-- 
Matthias

> I really want to use the 1.0 version to teach an introductory programming class, which starts September 2,
> 
> Thanks,
> 
> Mark




From dinov at microsoft.com  Wed Jul 17 23:13:36 2013
From: dinov at microsoft.com (Dino Viehland)
Date: Thu, 18 Jul 2013 03:13:36 +0000
Subject: [IPython-dev] Splitting inputs, cell inputs in an IPython client
Message-ID: <c471cdc72a29427cbadd25f36ad1054b@BY2PR03MB596.namprd03.prod.outlook.com>

We've recently noticed an issue when hosting IPython in PTVS - we split the 
inputs differently, which means when you have an input with multiple 
statements that build up a plot, we end up outputting the individual portions 
of the plot rather than one final plot at the end.

I've looked at how IPython is splitting the inputs and noticed a couple of things.  
First there's this doc string in the InputSplitter code:

        Return whether a block of interactive input can accept more input.
 
        This method is meant to be used by line-oriented frontends, who need to
        guess whether a block is complete or not based solely on prior and
        current input lines.  The InputSplitter considers it has a complete
        interactive block and will not accept more input only when either a
        SyntaxError is raised, or *all* of the following are true:
 
        1. The input compiles to a complete statement.
        
        2. The indentation level is flush-left (because if we are indented,
           like inside a function definition or for loop, we need to keep
           reading new input).
          
        3. There is one extra line consisting only of whitespace.
 
        Because of condition #3, this method should be used only by
        *line-oriented* frontends, since it means that intermediate blank lines
        are not allowed in function definitions (or any other indented block).
 
        If the current input produces a syntax error, this method immediately
        returns False but does *not* raise the syntax error exception, as
        typically clients will want to send invalid syntax to an execution
        backend which might convert the invalid syntax into valid Python via
        one of the dynamic IPython mechanisms.

Then there's this comment (*'s added for emphasis):
 ?????? # If we already have complete input and we're flush left, the answer
??????? # depends.? In line mode, if there hasn't been any indentation,
??????? # that's it. If we've come back from some indentation, we need
??????? # the blank final line to finish.
??????? # In cell mode, we need to check how many blocks the input so far
??????? # compiles into, because if there's already more than one full
??????? # independent block of input, then the client has entered full
??????? # 'cell' mode and is feeding lines that each is complete.? In this
??????? # case we should then keep accepting. The Qt terminal-like console
??????? # does precisely this, *to provide the convenience of terminal-like
??????? # input of single expressions, but allowing the user (with a
??????? # separate keystroke) to switch to 'cell' mode and type multiple
??????? # expressions in one shot*.

So my question is then - what do you think the best way to map this into 
our REPL running in VS is?  We already have logic for detecting if a statement 
is complete.  It's similar, but we never allow multiple complete statements in 
a row ('cell' mode I guess).  

We could go about supporting this a couple of different ways.  #1 is to simply 
model this behavior within PTVS.  We already have a parser, we're already 
looking for complete statements, we just need to tweak our behavior around 
rules #2 and #3.  
 
Or, option 2 we could send the text across the wire to IPython, and ask if IPython 
thinks that the input is complete.  That would allow IPython to be in charge, and if 
you evolve this behavior, or there's some way for users to provide their own line 
splitter, then we'd get the latest and greatest behavior.

Orthogonal to either of those choices is if we should expose a button somewhere 
which lets you quickly switch between 'line' and 'cell' mode, or if we should just
default to cell mode in our REPL when running in IPython mode.  And w/ approach
#2 we'd need to send the mode change over somehow too.
 
Do you guys have any thoughts on what you think we should do here?  I think #1 is 
probably actually easier to implement for us and result in a better user experience.  For 
example if the remote IPython process has become unresponsive or crashed the user 
will have a slow response time because we're querying for the complete input on the 
UI thread.  But option 2 might evolve nicer if this parsing ever changes.


Thoughts?




From takowl at gmail.com  Thu Jul 18 05:24:45 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 18 Jul 2013 10:24:45 +0100
Subject: [IPython-dev] Splitting inputs, cell inputs in an IPython client
In-Reply-To: <c471cdc72a29427cbadd25f36ad1054b@BY2PR03MB596.namprd03.prod.outlook.com>
References: <c471cdc72a29427cbadd25f36ad1054b@BY2PR03MB596.namprd03.prod.outlook.com>
Message-ID: <CAOvn4qhUWG3z190dsKnzTvtkDc=ZCFJEkp-5rzJSUhFBih296A@mail.gmail.com>

On 18 July 2013 04:13, Dino Viehland <dinov at microsoft.com> wrote:

> So my question is then - what do you think the best way to map this into
> our REPL running in VS is?  We already have logic for detecting if a
> statement
> is complete.  It's similar, but we never allow multiple complete
> statements in
> a row ('cell' mode I guess).
>

The two different modes are not as significant as they used to be. Both can
allow multiple complete statements in a single block of input. Of our own
clients:

- In the terminal, as you enter each line, it tries to determine whether
the block of input is complete. If it is, it sends it for execution, and if
not, it gives you a continuation prompt.
- In the Qt console, hitting return at the end of a line works like in the
terminal, but you can override it with shift-return/ctrl-return to force a
newline or force execution. At present, the completeness checking is done
by running the InputSplitter code in the frontend, but see below.
- In the notebook, none of this happens, and you need to use an explicit
shortcut to execute a cell.


>  Or, option 2 we could send the text across the wire to IPython, and ask
> if IPython
> thinks that the input is complete.  That would allow IPython to be in
> charge, and if
> you evolve this behavior, or there's some way for users to provide their
> own line
> splitter, then we'd get the latest and greatest behavior.
>

I do intend to add this to the protocol, and make the Qt console use it.
Part of the rationale is that we want to add a web console that's neatly
integrated with the Notebook, and our current InputSplitter code can only
be used by Python-based frontends.

> I think #1 is
> probably actually easier to implement for us and result in a better user
experience.  For
> example if the remote IPython process has become unresponsive or crashed
the user
> will have a slow response time because we're querying for the complete
input on the
> UI thread.

It will need some care, but I think it should be doable. We already do tab
completion by exchanging messages with the kernel.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130718/9d0bf028/attachment.html>

From mdroe at stsci.edu  Thu Jul 18 09:42:42 2013
From: mdroe at stsci.edu (Michael Droettboom)
Date: Thu, 18 Jul 2013 09:42:42 -0400
Subject: [IPython-dev] Results of matplotlib user survey 2013
Message-ID: <51E7F0D2.8040807@stsci.edu>

We have had 508 responses to the matplotlib user survey.  Quite a nice 
turnout!

You can view the results here:

https://docs.google.com/spreadsheet/viewanalytics?key=0AjrPjlTMRTwTdHpQS25pcTZIRWdqX0pNckNSU01sMHc&gridId=0#chart

and from there, you can access the complete raw results.

I will be doing more analysis of the results over the coming days and 
weeks, including dedup'ing some of the responses and converting some of 
the free-form responses into github issues etc.  Volunteers to help with 
this are of course welcome!

Cheers,
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130718/2ea4215f/attachment.html>

From mdroe at stsci.edu  Thu Jul 18 10:20:00 2013
From: mdroe at stsci.edu (Michael Droettboom)
Date: Thu, 18 Jul 2013 10:20:00 -0400
Subject: [IPython-dev] Results of matplotlib user survey 2013
In-Reply-To: <51E7F0D2.8040807@stsci.edu>
References: <51E7F0D2.8040807@stsci.edu>
Message-ID: <51E7F990.3060307@stsci.edu>

Apologies: I didn't realize the link to the raw results only exists for 
users with edit permissions.  The public URL for the raw results is:

https://docs.google.com/spreadsheet/ccc?key=0AjrPjlTMRTwTdHpQS25pcTZIRWdqX0pNckNSU01sMHc&usp=sharing

Mike

On 07/18/2013 09:42 AM, Michael Droettboom wrote:
> We have had 508 responses to the matplotlib user survey.  Quite a nice 
> turnout!
>
> You can view the results here:
>
> https://docs.google.com/spreadsheet/viewanalytics?key=0AjrPjlTMRTwTdHpQS25pcTZIRWdqX0pNckNSU01sMHc&gridId=0#chart
>
> and from there, you can access the complete raw results.
>
> I will be doing more analysis of the results over the coming days and 
> weeks, including dedup'ing some of the responses and converting some 
> of the free-form responses into github issues etc.  Volunteers to help 
> with this are of course welcome!
>
> Cheers,
> Mike

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130718/b47ef7f5/attachment.html>

From fperez.net at gmail.com  Thu Jul 18 17:33:08 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 18 Jul 2013 14:33:08 -0700
Subject: [IPython-dev] UC Berkeley open house with the development team, 7/24,
	4pm
Message-ID: <CAHAreOp-BaPJzgP5pHmiaDik_qjgZE=VxpJKeHkX3CH_hYhkug@mail.gmail.com>

Hi all,

[ while this is a local Berkeley event, I know this list reaches
campus folks as well who may not otherwise hear about it].

As part of our ongoing work, we're having an all-hands development
meeting during the week of July 22-26, including overseas members of
our team and partners in related projects from industry and academia
(including the Julia language team).

While the bulk of that meeting will focus on technical issues, on
Wednesday the 24th, at 4pm, we'll hold an 'open house' event.  Come
for snacks, meet the team, ask questions about IPython, discuss ideas
and suggestions, or demo your own work that uses IPython. We are as
eager to learn from what the community is doing with IPython as we are
to answer your own questions.

Event: IPython open house
Date: Wednesday, July 24th, 4pm.
Location: Tolman Hall 5101.

More details: https://github.com/ipython/ipython/wiki/Dev:-Meeting,-July-2013

Please forward this to interested colleagues on campus.

Cheers,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From dinov at microsoft.com  Fri Jul 19 12:38:53 2013
From: dinov at microsoft.com (Dino Viehland)
Date: Fri, 19 Jul 2013 16:38:53 +0000
Subject: [IPython-dev] Splitting inputs, cell inputs in an IPython client
In-Reply-To: <CAOvn4qhUWG3z190dsKnzTvtkDc=ZCFJEkp-5rzJSUhFBih296A@mail.gmail.com>
References: <c471cdc72a29427cbadd25f36ad1054b@BY2PR03MB596.namprd03.prod.outlook.com>
	<CAOvn4qhUWG3z190dsKnzTvtkDc=ZCFJEkp-5rzJSUhFBih296A@mail.gmail.com>
Message-ID: <27ac082a793444f4852b422bc5244ce3@BY2PR03MB596.namprd03.prod.outlook.com>


Thomas wrote:

On 18 July 2013 04:13, Dino Viehland <dinov at microsoft.com<mailto:dinov at microsoft.com>> wrote:
So my question is then - what do you think the best way to map this into
our REPL running in VS is?  We already have logic for detecting if a statement
is complete.  It's similar, but we never allow multiple complete statements in
a row ('cell' mode I guess).

The two different modes are not as significant as they used to be. Both can allow multiple complete statements in a single block of input. Of our own clients:
- In the terminal, as you enter each line, it tries to determine whether the block of input is complete. If it is, it sends it for execution, and if not, it gives you a continuation prompt.
- In the Qt console, hitting return at the end of a line works like in the terminal, but you can override it with shift-return/ctrl-return to force a newline or force execution. At present, the completeness checking is done by running the InputSplitter code in the frontend, but see below.
- In the notebook, none of this happens, and you need to use an explicit shortcut to execute a cell.
[Dino Viehland] Ok, we do also have explicit line break commands and submit commands available, I'll look into how we can best map this in our console.

Or, option 2 we could send the text across the wire to IPython, and ask if IPython
thinks that the input is complete.  That would allow IPython to be in charge, and if
you evolve this behavior, or there's some way for users to provide their own line
splitter, then we'd get the latest and greatest behavior.

I do intend to add this to the protocol, and make the Qt console use it. Part of the rationale is that we want to add a web console that's neatly integrated with the Notebook, and our current InputSplitter code can only be used by Python-based frontends.
[Dino Viehland] Do you know when this will happen?  We can certainly do our own parsing for now and then switch over whenever this becomes available.

> I think #1 is
> probably actually easier to implement for us and result in a better user experience.  For
> example if the remote IPython process has become unresponsive or crashed the user
> will have a slow response time because we're querying for the complete input on the
> UI thread.
It will need some care, but I think it should be doable. We already do tab completion by exchanging messages with the kernel.
[Dino Viehland] Good point, we do the same, and those happen a lot more.  Thanks for the feedback!
Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130719/b248bb47/attachment.html>

From takowl at gmail.com  Fri Jul 19 13:03:03 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 19 Jul 2013 18:03:03 +0100
Subject: [IPython-dev] Splitting inputs, cell inputs in an IPython client
In-Reply-To: <27ac082a793444f4852b422bc5244ce3@BY2PR03MB596.namprd03.prod.outlook.com>
References: <c471cdc72a29427cbadd25f36ad1054b@BY2PR03MB596.namprd03.prod.outlook.com>
	<CAOvn4qhUWG3z190dsKnzTvtkDc=ZCFJEkp-5rzJSUhFBih296A@mail.gmail.com>
	<27ac082a793444f4852b422bc5244ce3@BY2PR03MB596.namprd03.prod.outlook.com>
Message-ID: <CAOvn4qi5pQLD-VxqijfntJmKLnsmnjsc0E0UNa0D=ar8uTm5kg@mail.gmail.com>

On 19 July 2013 17:38, Dino Viehland <dinov at microsoft.com> wrote:

> Or, option 2 we could send the text across the wire to IPython, and ask if
> IPython
> thinks that the input is complete.  That would allow IPython to be in
> charge, and if
> you evolve this behavior, or there's some way for users to provide their
> own line
> splitter, then we'd get the latest and greatest behavior.****
>
>  ** **
>
> I do intend to add this to the protocol, and make the Qt console use it.
> Part of the rationale is that we want to add a web console that's neatly
> integrated with the Notebook, and our current InputSplitter code can only
> be used by Python-based frontends.****
>
> *[Dino Viehland] Do you know when this will happen?  We can certainly do
> our own parsing for now and then switch over whenever this becomes
> available.*
>

I expect it will miss 1.0 now. I think a web console would be an important
addition to the notebook, so hopefully it will make the next release
(planned around Christmas time). But once I'm being paid to work on
IPython, it might depend on whether others agree with me about that being a
priority.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130719/8a00a20e/attachment.html>

From fperez.net at gmail.com  Fri Jul 19 18:30:05 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 19 Jul 2013 15:30:05 -0700
Subject: [IPython-dev] Splitting inputs, cell inputs in an IPython client
In-Reply-To: <CAOvn4qi5pQLD-VxqijfntJmKLnsmnjsc0E0UNa0D=ar8uTm5kg@mail.gmail.com>
References: <c471cdc72a29427cbadd25f36ad1054b@BY2PR03MB596.namprd03.prod.outlook.com>
	<CAOvn4qhUWG3z190dsKnzTvtkDc=ZCFJEkp-5rzJSUhFBih296A@mail.gmail.com>
	<27ac082a793444f4852b422bc5244ce3@BY2PR03MB596.namprd03.prod.outlook.com>
	<CAOvn4qi5pQLD-VxqijfntJmKLnsmnjsc0E0UNa0D=ar8uTm5kg@mail.gmail.com>
Message-ID: <CAHAreOqZ4eLwuGR=Rme3JDcb7Tw97tsw7C_b50qyzFPT=J4V4g@mail.gmail.com>

We'll begin thinking about some of these questions as the work on the
julia kernel progresses:

https://github.com/JuliaLang/julia-ipython

Having a robust Julia kernel that works with all IPython clients will
force us to flush some of these design issues. One important point to
keep in mind is that we really need to think now about these protocols
as being 100% language-agnostic and as enablers of a broad range of
use cases.  What a web console will look like is something we'll need
to work on, but I'm pretty sure that we will end up needing one before
too long.
Cheers,

f


From benjaminrk at gmail.com  Sun Jul 21 21:43:47 2013
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 21 Jul 2013 18:43:47 -0700
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
Message-ID: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>

First alpha of IPython 1.0.0 is now available, please give it a test drive.

http://archive.ipython.org/testing/1.0.0

Probably the quickest way to install:

pip install http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz

Lots of new stuff, especially in the notebook. Also significant performance
improvements in IPython.parallel and qtconsole, and a revamped input
transform system that should be much better behaved than ever before. Plus
loads of bugfixes all over the place, of course.

For a quick summary, see what's
new<http://ipython.org/ipython-doc/dev/whatsnew/version1.0.html>
.

The most significant addition is nbconvert, accessible now as ipython
nbconvert [mynotebook.ipynb].

nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
so we hope to hear from early adopters, particularly those using it to
integrate IPython with blogging engines or writing papers or documentation.
We full expect that by 2.0 there will need to be enough changes that some
APIs will not survive from 1.0 to 2.0, so you should be aware of that.

There will still be a few more adjustments to nbconvert,
and more fleshing out of documentation before 1.0 is released.

Please report any installation issues you encounter, that's the most
important part of this first alpha.

We are happy to get this out before our big all hands dev meeting this week,
and only six days behind schedule (who said that July 15 was the target for
*final* release of 1.0?)

Thanks for everyone's contributions in getting us this far!

-MinRK et al. IPython Devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130721/aec51018/attachment.html>

From asmeurer at gmail.com  Sun Jul 21 23:23:05 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 21 Jul 2013 22:23:05 -0500
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
Message-ID: <CAKgW=6+4XGLL-Wq4Wxs2bFN=hTAGH0-j292pjLsf0w__wy-wFQ@mail.gmail.com>

I get

/Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
UserWarning: The top-level `frontend` package has been deprecated. All
its subpackages have been moved to the top `IPython` level.
  warn("The top-level `frontend` package has been deprecated. "

when I start IPython. Is it maybe an issue with my configuration?

Aaron Meurer


On Sun, Jul 21, 2013 at 8:43 PM, MinRK <benjaminrk at gmail.com> wrote:
> First alpha of IPython 1.0.0 is now available, please give it a test drive.
>
> http://archive.ipython.org/testing/1.0.0
>
> Probably the quickest way to install:
>
> pip install http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz
>
> Lots of new stuff, especially in the notebook. Also significant performance
> improvements in IPython.parallel and qtconsole, and a revamped input
> transform system that should be much better behaved than ever before. Plus
> loads of bugfixes all over the place, of course.
>
> For a quick summary, see what's new.
>
> The most significant addition is nbconvert, accessible now as ipython
> nbconvert [mynotebook.ipynb].
>
> nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
> so we hope to hear from early adopters, particularly those using it to
> integrate IPython with blogging engines or writing papers or documentation.
> We full expect that by 2.0 there will need to be enough changes that some
> APIs will not survive from 1.0 to 2.0, so you should be aware of that.
>
> There will still be a few more adjustments to nbconvert,
> and more fleshing out of documentation before 1.0 is released.
>
> Please report any installation issues you encounter, that's the most
> important part of this first alpha.
>
> We are happy to get this out before our big all hands dev meeting this week,
> and only six days behind schedule (who said that July 15 was the target for
> final release of 1.0?)
>
> Thanks for everyone's contributions in getting us this far!
>
> -MinRK et al. IPython Devs
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From benjaminrk at gmail.com  Sun Jul 21 23:29:42 2013
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 21 Jul 2013 20:29:42 -0700
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <CAKgW=6+4XGLL-Wq4Wxs2bFN=hTAGH0-j292pjLsf0w__wy-wFQ@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
	<CAKgW=6+4XGLL-Wq4Wxs2bFN=hTAGH0-j292pjLsf0w__wy-wFQ@mail.gmail.com>
Message-ID: <CAHNn8BV-odoSwvxESLXSSjgcbryQ378vwP5VorJbi5hWPgKMCA@mail.gmail.com>

Do you have anything in your config files that might import from frontend
(could be such an import in sympy printing)? Does it happen with an empty
profile (`ipython --profile empty`), or in a fresh install in a clean venv?
 It's possible some of the entry points weren't updated, but I don't think
that's the case.


On Sun, Jul 21, 2013 at 8:23 PM, Aaron Meurer <asmeurer at gmail.com> wrote:

> I get
>
>
> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
> UserWarning: The top-level `frontend` package has been deprecated. All
> its subpackages have been moved to the top `IPython` level.
>   warn("The top-level `frontend` package has been deprecated. "
>
> when I start IPython. Is it maybe an issue with my configuration?
>
> Aaron Meurer
>
>
> On Sun, Jul 21, 2013 at 8:43 PM, MinRK <benjaminrk at gmail.com> wrote:
> > First alpha of IPython 1.0.0 is now available, please give it a test
> drive.
> >
> > http://archive.ipython.org/testing/1.0.0
> >
> > Probably the quickest way to install:
> >
> > pip install
> http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz
> >
> > Lots of new stuff, especially in the notebook. Also significant
> performance
> > improvements in IPython.parallel and qtconsole, and a revamped input
> > transform system that should be much better behaved than ever before.
> Plus
> > loads of bugfixes all over the place, of course.
> >
> > For a quick summary, see what's new.
> >
> > The most significant addition is nbconvert, accessible now as ipython
> > nbconvert [mynotebook.ipynb].
> >
> > nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
> > so we hope to hear from early adopters, particularly those using it to
> > integrate IPython with blogging engines or writing papers or
> documentation.
> > We full expect that by 2.0 there will need to be enough changes that some
> > APIs will not survive from 1.0 to 2.0, so you should be aware of that.
> >
> > There will still be a few more adjustments to nbconvert,
> > and more fleshing out of documentation before 1.0 is released.
> >
> > Please report any installation issues you encounter, that's the most
> > important part of this first alpha.
> >
> > We are happy to get this out before our big all hands dev meeting this
> week,
> > and only six days behind schedule (who said that July 15 was the target
> for
> > final release of 1.0?)
> >
> > Thanks for everyone's contributions in getting us this far!
> >
> > -MinRK et al. IPython Devs
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130721/a980d64b/attachment.html>

From asmeurer at gmail.com  Sun Jul 21 23:33:57 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 21 Jul 2013 22:33:57 -0500
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <CAHNn8BV-odoSwvxESLXSSjgcbryQ378vwP5VorJbi5hWPgKMCA@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
	<CAKgW=6+4XGLL-Wq4Wxs2bFN=hTAGH0-j292pjLsf0w__wy-wFQ@mail.gmail.com>
	<CAHNn8BV-odoSwvxESLXSSjgcbryQ378vwP5VorJbi5hWPgKMCA@mail.gmail.com>
Message-ID: <CAKgW=6Lb88b7kSkoGCb9Z3MOFy+hX9L_x+x9+tDL9QhP6hFfLA@mail.gmail.com>

This is actually coming from a conda package I built
(https://github.com/ContinuumIO/conda-recipes/tree/master/ipython).

Here is the empty profile:

$ipython --profile empty
/Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
UserWarning: The top-level `frontend` package has been deprecated. All
its subpackages have been moved to the top `IPython` level.
  warn("The top-level `frontend` package has been deprecated. "
/Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/utils/path.py:468:
UserWarning: Found old IPython config file
u'/Users/aaronmeurer/.ipython/ipythonrc' (modified by user)
  warnings.warn("Found old IPython config file %r (modified by user)"%f)
/Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/utils/path.py:480:
UserWarning:
  The IPython configuration system has changed as of 0.11, and these files will
  be ignored. See http://ipython.github.com/ipython-doc/dev/config for details
  of the new config system.
  To start configuring IPython, do `ipython profile create`, and edit
  `ipython_config.py` in <ipython_dir>/profile_default.
  If you need to leave the old config files in place for an older version of
  IPython and want to suppress this warning message, set
  `c.InteractiveShellApp.ignore_old_config=True` in the new config.
  `c.InteractiveShellApp.ignore_old_config=True` in the new config.""")
Python 2.7.5 |Continuum Analytics, Inc.| (default, Jun 28 2013, 22:20:13)
Type "copyright", "credits" or "license" for more information.

IPython 1.0.0a1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

IPython profile: empty

In [1]:

Is there a way to just ignore the profile stuff (like emacs -Q)?

Aaron Meurer

On Sun, Jul 21, 2013 at 10:29 PM, MinRK <benjaminrk at gmail.com> wrote:
> Do you have anything in your config files that might import from frontend
> (could be such an import in sympy printing)? Does it happen with an empty
> profile (`ipython --profile empty`), or in a fresh install in a clean venv?
> It's possible some of the entry points weren't updated, but I don't think
> that's the case.
>
>
> On Sun, Jul 21, 2013 at 8:23 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>>
>> I get
>>
>>
>> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
>> UserWarning: The top-level `frontend` package has been deprecated. All
>> its subpackages have been moved to the top `IPython` level.
>>   warn("The top-level `frontend` package has been deprecated. "
>>
>> when I start IPython. Is it maybe an issue with my configuration?
>>
>> Aaron Meurer
>>
>>
>> On Sun, Jul 21, 2013 at 8:43 PM, MinRK <benjaminrk at gmail.com> wrote:
>> > First alpha of IPython 1.0.0 is now available, please give it a test
>> > drive.
>> >
>> > http://archive.ipython.org/testing/1.0.0
>> >
>> > Probably the quickest way to install:
>> >
>> > pip install
>> > http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz
>> >
>> > Lots of new stuff, especially in the notebook. Also significant
>> > performance
>> > improvements in IPython.parallel and qtconsole, and a revamped input
>> > transform system that should be much better behaved than ever before.
>> > Plus
>> > loads of bugfixes all over the place, of course.
>> >
>> > For a quick summary, see what's new.
>> >
>> > The most significant addition is nbconvert, accessible now as ipython
>> > nbconvert [mynotebook.ipynb].
>> >
>> > nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
>> > so we hope to hear from early adopters, particularly those using it to
>> > integrate IPython with blogging engines or writing papers or
>> > documentation.
>> > We full expect that by 2.0 there will need to be enough changes that
>> > some
>> > APIs will not survive from 1.0 to 2.0, so you should be aware of that.
>> >
>> > There will still be a few more adjustments to nbconvert,
>> > and more fleshing out of documentation before 1.0 is released.
>> >
>> > Please report any installation issues you encounter, that's the most
>> > important part of this first alpha.
>> >
>> > We are happy to get this out before our big all hands dev meeting this
>> > week,
>> > and only six days behind schedule (who said that July 15 was the target
>> > for
>> > final release of 1.0?)
>> >
>> > Thanks for everyone's contributions in getting us this far!
>> >
>> > -MinRK et al. IPython Devs
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From bussonniermatthias at gmail.com  Sun Jul 21 23:36:58 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 21 Jul 2013 20:36:58 -0700
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
Message-ID: <CA2DBCA5-4242-43E4-AC99-5C3AEBE410C3@gmail.com>


> For a quick summary, see what's new.

https://github.com/ipython/ipython/blob/master/docs/source/whatsnew/version1.0.txt#L215

The -- ```python --
block is rendered in rst with the backslash to escape the `

Not sure how to fix.

-- 
M




From bussonniermatthias at gmail.com  Sun Jul 21 23:40:05 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 21 Jul 2013 20:40:05 -0700
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <CAKgW=6Lb88b7kSkoGCb9Z3MOFy+hX9L_x+x9+tDL9QhP6hFfLA@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
	<CAKgW=6+4XGLL-Wq4Wxs2bFN=hTAGH0-j292pjLsf0w__wy-wFQ@mail.gmail.com>
	<CAHNn8BV-odoSwvxESLXSSjgcbryQ378vwP5VorJbi5hWPgKMCA@mail.gmail.com>
	<CAKgW=6Lb88b7kSkoGCb9Z3MOFy+hX9L_x+x9+tDL9QhP6hFfLA@mail.gmail.com>
Message-ID: <56A82ED3-1539-44AB-B8DD-71EA00FE7B22@gmail.com>


Le 21 juil. 2013 ? 20:33, Aaron Meurer a ?crit :

> This is actually coming from a conda package I built
> (https://github.com/ContinuumIO/conda-recipes/tree/master/ipython).
> 
> Here is the empty profile:

be carefull many files still have references to frontend.

like meta.yaml
build:
  entry_points:
    - ipython3 = IPython.frontend.terminal.ipapp:launch_new_instance  [py3k]
    - ipython = IPython.frontend.terminal.ipapp:launch_new_instance

> 
> $ipython --profile empty
> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
> UserWarning: The top-level `frontend` package has been deprecated. All
> its subpackages have been moved to the top `IPython` level.
>  warn("The top-level `frontend` package has been deprecated. "
> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/utils/path.py:468:
> UserWarning: Found old IPython config file
> u'/Users/aaronmeurer/.ipython/ipythonrc' (modified by user)
>  warnings.warn("Found old IPython config file %r (modified by user)"%f)
> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/utils/path.py:480:
> UserWarning:
>  The IPython configuration system has changed as of 0.11, and these files will
>  be ignored. See http://ipython.github.com/ipython-doc/dev/config for details
>  of the new config system.
>  To start configuring IPython, do `ipython profile create`, and edit
>  `ipython_config.py` in <ipython_dir>/profile_default.
>  If you need to leave the old config files in place for an older version of
>  IPython and want to suppress this warning message, set
>  `c.InteractiveShellApp.ignore_old_config=True` in the new config.
>  `c.InteractiveShellApp.ignore_old_config=True` in the new config.""")
> Python 2.7.5 |Continuum Analytics, Inc.| (default, Jun 28 2013, 22:20:13)
> Type "copyright", "credits" or "license" for more information.
> 
> IPython 1.0.0a1 -- An enhanced Interactive Python.
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object', use 'object??' for extra details.
> 
> IPython profile: empty
> 
> In [1]:
> 
> Is there a way to just ignore the profile stuff (like emacs -Q)?
> 
> Aaron Meurer
> 
> On Sun, Jul 21, 2013 at 10:29 PM, MinRK <benjaminrk at gmail.com> wrote:
>> Do you have anything in your config files that might import from frontend
>> (could be such an import in sympy printing)? Does it happen with an empty
>> profile (`ipython --profile empty`), or in a fresh install in a clean venv?
>> It's possible some of the entry points weren't updated, but I don't think
>> that's the case.
>> 
>> 
>> On Sun, Jul 21, 2013 at 8:23 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>>> 
>>> I get
>>> 
>>> 
>>> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
>>> UserWarning: The top-level `frontend` package has been deprecated. All
>>> its subpackages have been moved to the top `IPython` level.
>>>  warn("The top-level `frontend` package has been deprecated. "
>>> 
>>> when I start IPython. Is it maybe an issue with my configuration?
>>> 
>>> Aaron Meurer
>>> 
>>> 
>>> On Sun, Jul 21, 2013 at 8:43 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>> First alpha of IPython 1.0.0 is now available, please give it a test
>>>> drive.
>>>> 
>>>> http://archive.ipython.org/testing/1.0.0
>>>> 
>>>> Probably the quickest way to install:
>>>> 
>>>> pip install
>>>> http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz
>>>> 
>>>> Lots of new stuff, especially in the notebook. Also significant
>>>> performance
>>>> improvements in IPython.parallel and qtconsole, and a revamped input
>>>> transform system that should be much better behaved than ever before.
>>>> Plus
>>>> loads of bugfixes all over the place, of course.
>>>> 
>>>> For a quick summary, see what's new.
>>>> 
>>>> The most significant addition is nbconvert, accessible now as ipython
>>>> nbconvert [mynotebook.ipynb].
>>>> 
>>>> nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
>>>> so we hope to hear from early adopters, particularly those using it to
>>>> integrate IPython with blogging engines or writing papers or
>>>> documentation.
>>>> We full expect that by 2.0 there will need to be enough changes that
>>>> some
>>>> APIs will not survive from 1.0 to 2.0, so you should be aware of that.
>>>> 
>>>> There will still be a few more adjustments to nbconvert,
>>>> and more fleshing out of documentation before 1.0 is released.
>>>> 
>>>> Please report any installation issues you encounter, that's the most
>>>> important part of this first alpha.
>>>> 
>>>> We are happy to get this out before our big all hands dev meeting this
>>>> week,
>>>> and only six days behind schedule (who said that July 15 was the target
>>>> for
>>>> final release of 1.0?)
>>>> 
>>>> Thanks for everyone's contributions in getting us this far!
>>>> 
>>>> -MinRK et al. IPython Devs
>>>> 
>>>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> 
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From asmeurer at gmail.com  Sun Jul 21 23:44:13 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 21 Jul 2013 22:44:13 -0500
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <56A82ED3-1539-44AB-B8DD-71EA00FE7B22@gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
	<CAKgW=6+4XGLL-Wq4Wxs2bFN=hTAGH0-j292pjLsf0w__wy-wFQ@mail.gmail.com>
	<CAHNn8BV-odoSwvxESLXSSjgcbryQ378vwP5VorJbi5hWPgKMCA@mail.gmail.com>
	<CAKgW=6Lb88b7kSkoGCb9Z3MOFy+hX9L_x+x9+tDL9QhP6hFfLA@mail.gmail.com>
	<56A82ED3-1539-44AB-B8DD-71EA00FE7B22@gmail.com>
Message-ID: <CAKgW=6+KZgup_dkrqM_9NfVWL24LL0y-M3tiQM8+RW=So943CA@mail.gmail.com>

Oh, yeah, my fault then. I removed it and it went away.

Aaron Meurer

On Sun, Jul 21, 2013 at 10:40 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 21 juil. 2013 ? 20:33, Aaron Meurer a ?crit :
>
>> This is actually coming from a conda package I built
>> (https://github.com/ContinuumIO/conda-recipes/tree/master/ipython).
>>
>> Here is the empty profile:
>
> be carefull many files still have references to frontend.
>
> like meta.yaml
> build:
>   entry_points:
>     - ipython3 = IPython.frontend.terminal.ipapp:launch_new_instance  [py3k]
>     - ipython = IPython.frontend.terminal.ipapp:launch_new_instance
>
>>
>> $ipython --profile empty
>> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
>> UserWarning: The top-level `frontend` package has been deprecated. All
>> its subpackages have been moved to the top `IPython` level.
>>  warn("The top-level `frontend` package has been deprecated. "
>> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/utils/path.py:468:
>> UserWarning: Found old IPython config file
>> u'/Users/aaronmeurer/.ipython/ipythonrc' (modified by user)
>>  warnings.warn("Found old IPython config file %r (modified by user)"%f)
>> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/utils/path.py:480:
>> UserWarning:
>>  The IPython configuration system has changed as of 0.11, and these files will
>>  be ignored. See http://ipython.github.com/ipython-doc/dev/config for details
>>  of the new config system.
>>  To start configuring IPython, do `ipython profile create`, and edit
>>  `ipython_config.py` in <ipython_dir>/profile_default.
>>  If you need to leave the old config files in place for an older version of
>>  IPython and want to suppress this warning message, set
>>  `c.InteractiveShellApp.ignore_old_config=True` in the new config.
>>  `c.InteractiveShellApp.ignore_old_config=True` in the new config.""")
>> Python 2.7.5 |Continuum Analytics, Inc.| (default, Jun 28 2013, 22:20:13)
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 1.0.0a1 -- An enhanced Interactive Python.
>> ?         -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help      -> Python's own help system.
>> object?   -> Details about 'object', use 'object??' for extra details.
>>
>> IPython profile: empty
>>
>> In [1]:
>>
>> Is there a way to just ignore the profile stuff (like emacs -Q)?
>>
>> Aaron Meurer
>>
>> On Sun, Jul 21, 2013 at 10:29 PM, MinRK <benjaminrk at gmail.com> wrote:
>>> Do you have anything in your config files that might import from frontend
>>> (could be such an import in sympy printing)? Does it happen with an empty
>>> profile (`ipython --profile empty`), or in a fresh install in a clean venv?
>>> It's possible some of the entry points weren't updated, but I don't think
>>> that's the case.
>>>
>>>
>>> On Sun, Jul 21, 2013 at 8:23 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>>>>
>>>> I get
>>>>
>>>>
>>>> /Users/aaronmeurer/anaconda/envs/ipython1.0/lib/python2.7/site-packages/IPython/frontend.py:30:
>>>> UserWarning: The top-level `frontend` package has been deprecated. All
>>>> its subpackages have been moved to the top `IPython` level.
>>>>  warn("The top-level `frontend` package has been deprecated. "
>>>>
>>>> when I start IPython. Is it maybe an issue with my configuration?
>>>>
>>>> Aaron Meurer
>>>>
>>>>
>>>> On Sun, Jul 21, 2013 at 8:43 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>>> First alpha of IPython 1.0.0 is now available, please give it a test
>>>>> drive.
>>>>>
>>>>> http://archive.ipython.org/testing/1.0.0
>>>>>
>>>>> Probably the quickest way to install:
>>>>>
>>>>> pip install
>>>>> http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz
>>>>>
>>>>> Lots of new stuff, especially in the notebook. Also significant
>>>>> performance
>>>>> improvements in IPython.parallel and qtconsole, and a revamped input
>>>>> transform system that should be much better behaved than ever before.
>>>>> Plus
>>>>> loads of bugfixes all over the place, of course.
>>>>>
>>>>> For a quick summary, see what's new.
>>>>>
>>>>> The most significant addition is nbconvert, accessible now as ipython
>>>>> nbconvert [mynotebook.ipynb].
>>>>>
>>>>> nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
>>>>> so we hope to hear from early adopters, particularly those using it to
>>>>> integrate IPython with blogging engines or writing papers or
>>>>> documentation.
>>>>> We full expect that by 2.0 there will need to be enough changes that
>>>>> some
>>>>> APIs will not survive from 1.0 to 2.0, so you should be aware of that.
>>>>>
>>>>> There will still be a few more adjustments to nbconvert,
>>>>> and more fleshing out of documentation before 1.0 is released.
>>>>>
>>>>> Please report any installation issues you encounter, that's the most
>>>>> important part of this first alpha.
>>>>>
>>>>> We are happy to get this out before our big all hands dev meeting this
>>>>> week,
>>>>> and only six days behind schedule (who said that July 15 was the target
>>>>> for
>>>>> final release of 1.0?)
>>>>>
>>>>> Thanks for everyone's contributions in getting us this far!
>>>>>
>>>>> -MinRK et al. IPython Devs
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From lchaplin13 at gmail.com  Mon Jul 22 01:37:02 2013
From: lchaplin13 at gmail.com (Lee Chaplin)
Date: Mon, 22 Jul 2013 17:37:02 +1200
Subject: [IPython-dev] No code in the exported HTML
Message-ID: <CAJ_5dtr+dtz9cd0L_aP4YDSnrdJhpRRRH_cevcx+Q6OFG2RDtA@mail.gmail.com>

Hi all,

I am a bit lost here, any help will be very appreciated.
I would like to save a static HTML page from an IPy Notebook containing no
code, and no "In [1]:" or "Out [1]:" divs.
In other words, I would like that the resulted page to contain no traces of
python/ipython code, so leaving only the comments, texts, images, latex in
place.
I reckon one option would be to eliminate by hand the HTML divs that
contain the code - or by XSLT. The second option seems be to use nbconvert
- and I don't know yet how to use it.

Any idea what is the simplest/straight forward way to achieve this?

Thank you,
Lee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130722/1e6a71d8/attachment.html>

From bussonniermatthias at gmail.com  Mon Jul 22 02:39:14 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 21 Jul 2013 23:39:14 -0700
Subject: [IPython-dev] No code in the exported HTML
In-Reply-To: <CAJ_5dtr+dtz9cd0L_aP4YDSnrdJhpRRRH_cevcx+Q6OFG2RDtA@mail.gmail.com>
References: <CAJ_5dtr+dtz9cd0L_aP4YDSnrdJhpRRRH_cevcx+Q6OFG2RDtA@mail.gmail.com>
Message-ID: <54706C6F-9FA7-4668-89E0-054C664166FA@gmail.com>

Hi Lee, 

IPython 1.0.alpha has beed released a few hours ago, 
This means that you can try it and report bug if you wish, but be carefull, it is still an alpha. 

there are not a lot of doc available for now in nbconvert, but what I would suggest is to write a
simple template that inherit full_html and overwrite the "block input" to be empty, like so

{%- extends 'fullhtml.tpl' -%}

{% block input %}
{%- endblock input %}


You should be able to use the 2 following option to tell nbconvert to use your template

FullHTMLExporter.template_file=<Unicode>

I'm not sure there is a CL flag to tell nbconvert where to search theses templates, so you might ave to put your template in IPython installation directory for now or use the Programatic API. 
--
Matthias


Le 21 juil. 2013 ? 22:37, Lee Chaplin a ?crit :

> Hi all,
> 
> I am a bit lost here, any help will be very appreciated.
> I would like to save a static HTML page from an IPy Notebook containing no code, and no "In [1]:" or "Out [1]:" divs.
> In other words, I would like that the resulted page to contain no traces of python/ipython code, so leaving only the comments, texts, images, latex in place.
> I reckon one option would be to eliminate by hand the HTML divs that contain the code - or by XSLT. The second option seems be to use nbconvert - and I don't know yet how to use it.
> 
> Any idea what is the simplest/straight forward way to achieve this?
> 
> Thank you,
> Lee
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From maximilian.albert at gmail.com  Mon Jul 22 07:49:50 2013
From: maximilian.albert at gmail.com (Maximilian Albert)
Date: Mon, 22 Jul 2013 12:49:50 +0100
Subject: [IPython-dev] [ANN] IPython 1.0.0a1
In-Reply-To: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
Message-ID: <CAGA_dmgqxWRK3AKXJCaH-11HyiX6YcxWOvkwy3icjS8qzX1V6Q@mail.gmail.com>

Hi all,

first of all, congratulations for getting the release candidate out
and thanks for everyone's amazing work and efforts!

Just a few minor issues/comments related to my first attempts playing
with the new nbconvert:

(i) When using the LaTeX converter on one of my sample notebooks and
running pdflatex on the resulting .tex file, it complained about a
missing file 'adjustbox.sty'. Obviously, fixing this is just a matter
of installing the 'adjustbox' package, but Ubuntu 12.04 LTS (which I'm
still using) doesn't provide this in any of its packages (however, as
of 12.10 it's included in the 'texlive-pictures' package). It is no
big deal to install 'adjustbox' manually from CTAN, but I guess some
users might appreciate not having to do that if there is an easy
workaround within nbconvert that avoids using this package.

(ii) I noticed that nbconvert creates a subdirectory called
'nbconvert_build' in which all output is placed. It took me a while to
figure out how to configure the output directory (namely, using the
option '--FilesWriter.build_directory', which was only mentioned when
I used the --help-all switch). Since I expect configuring the output
directory to be a common requirement, I'd suggest adding a simpler
command line switch such as '--output-dir', and also mention this in
the basic help text.

As a side note, without any options I would have expected the 'main'
output file (e.g., foo.tex when using the LaTeX converter) to be
placed in the same directory as the input .ipynb file and only
auxiliary files (images etc.) to be put in the 'nbconvert_build'
subdirectory - which I'd personally find a bit cleaner. No big deal,
though. (And I guess there are reasons for doing it the other way.)

(iii) I believe this was mentioned in a different thread already, but
I'd suggest renaming the format 'full_html' to 'html' as the latter is
much more intuitive to guess.

Thanks for your time and best wishes,
Max


2013/7/22 MinRK <benjaminrk at gmail.com>:
> First alpha of IPython 1.0.0 is now available, please give it a test drive.
>
> http://archive.ipython.org/testing/1.0.0
>
> Probably the quickest way to install:
>
> pip install http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz
>
> Lots of new stuff, especially in the notebook. Also significant performance
> improvements in IPython.parallel and qtconsole, and a revamped input
> transform system that should be much better behaved than ever before. Plus
> loads of bugfixes all over the place, of course.
>
> For a quick summary, see what's new.
>
> The most significant addition is nbconvert, accessible now as ipython
> nbconvert [mynotebook.ipynb].
>
> nbconvert is going to be decidedly in ?tech preview? mode for 1.0,
> so we hope to hear from early adopters, particularly those using it to
> integrate IPython with blogging engines or writing papers or documentation.
> We full expect that by 2.0 there will need to be enough changes that some
> APIs will not survive from 1.0 to 2.0, so you should be aware of that.
>
> There will still be a few more adjustments to nbconvert,
> and more fleshing out of documentation before 1.0 is released.
>
> Please report any installation issues you encounter, that's the most
> important part of this first alpha.
>
> We are happy to get this out before our big all hands dev meeting this week,
> and only six days behind schedule (who said that July 15 was the target for
> final release of 1.0?)
>
> Thanks for everyone's contributions in getting us this far!
>
> -MinRK et al. IPython Devs
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From lchaplin13 at gmail.com  Tue Jul 23 22:26:59 2013
From: lchaplin13 at gmail.com (Lee Chaplin)
Date: Wed, 24 Jul 2013 14:26:59 +1200
Subject: [IPython-dev] No code in the exported HTML
Message-ID: <CAJ_5dtqD0wb83oNiETnvthQOrccVaOej6VU_h9wd5Znj9OaXig@mail.gmail.com>

Thanks Matthias, I will have to study more about nbconvert...
For the moment (and for others that might look for a solution) I managed to
find a quick and dirty solution in Python using BeautifulSoup, something
like:
soup = BeautifulSoup(data_read) # data_read = the IPyNB static saved page
for divin in soup.find_all("div", { "class" : "input hbox" }):
    divin.extract()
for divout in soup.find_all("div", { "class" : "prompt output_prompt" }):
    divout.string.replace_with("")
data_write = soup.renderContents()

Lee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130724/251771df/attachment.html>

From tarun.gaba7 at gmail.com  Wed Jul 24 12:49:13 2013
From: tarun.gaba7 at gmail.com (TARUN GABA)
Date: Wed, 24 Jul 2013 22:19:13 +0530
Subject: [IPython-dev] IPython's Javascript Test framework
Message-ID: <CAHAono0AmAPwXg5F4fgw7+kTg5fgquuCGi7qCWV4ZPBDwoKy7g@mail.gmail.com>

Hi
I was wondering which Javascript testing module/framework is being used for
IPython's Javascripts?

Also could anyone suggest some good headless(without browser) Javascript
testing frameworks.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130724/dfd62984/attachment.html>

From bussonniermatthias at gmail.com  Wed Jul 24 13:21:05 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 24 Jul 2013 10:21:05 -0700
Subject: [IPython-dev] IPython's Javascript Test framework
In-Reply-To: <CAHAono0AmAPwXg5F4fgw7+kTg5fgquuCGi7qCWV4ZPBDwoKy7g@mail.gmail.com>
References: <CAHAono0AmAPwXg5F4fgw7+kTg5fgquuCGi7qCWV4ZPBDwoKy7g@mail.gmail.com>
Message-ID: <549354B5-2009-4C21-9EE2-982F5E9C7979@gmail.com>


Le 24 juil. 2013 ? 09:49, TARUN GABA a ?crit :

> Hi
> I was wondering which Javascript testing module/framework is being used for IPython's Javascripts?

None for now, we are considering Casper.js to test UI, saw a lice demo yesterday.

modularize js to be testable in node (haven't looked at framework to that yet)


> Also could anyone suggest some good headless(without browser) Javascript testing frameworks.
> 
> Thanks
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From david at wolever.net  Wed Jul 24 13:23:27 2013
From: david at wolever.net (David Wolever)
Date: Wed, 24 Jul 2013 10:23:27 -0700
Subject: [IPython-dev] IPython's Javascript Test framework
In-Reply-To: <CAHAono0AmAPwXg5F4fgw7+kTg5fgquuCGi7qCWV4ZPBDwoKy7g@mail.gmail.com>
References: <CAHAono0AmAPwXg5F4fgw7+kTg5fgquuCGi7qCWV4ZPBDwoKy7g@mail.gmail.com>
Message-ID: <7E9FC6E7-75C3-48BB-92D6-DDCCD1A5EC9B@wolever.net>


On 2013-07-24, at 9:49 AM, TARUN GABA <tarun.gaba7 at gmail.com> wrote:

> Also could anyone suggest some good headless(without browser) Javascript testing frameworks.

This StackOverflow question does a good job of covering the popular tools: http://stackoverflow.com/q/300855/71522

Basically, the three levels you need to consider are:
- Do you need full DOM interaction in different browsers? If so, you need Selenium (and you can use SauceLabs to run your tests across every browser ever)
- Do you need some DOM interaction in a more or less browser-agnostic way? Then you need PhantomJS.
- Are you just testing pure JavaScript? In that case, go for QUnit + Node. You could follow the example of one of my libraries, here: https://github.com/wolever/remora/tree/master/test
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130724/29654b48/attachment.html>

From vbraun.name at gmail.com  Wed Jul 24 19:17:03 2013
From: vbraun.name at gmail.com (Volker Braun)
Date: Wed, 24 Jul 2013 16:17:03 -0700 (PDT)
Subject: [IPython-dev] InputSplitter and SyntaxError
Message-ID: <1374707823022-5025938.post@n6.nabble.com>

In IPython 0.13.2, if the input_splitter.push() method raises an exception
then it is not caught, terminating the IPython process. Sage defines a
special syntax for some mathematical objects and we would like to have a way
to tell the user about misbalanced brackets, but I don't see how that is
supposed to work. Is there a way?

I see that the new IPython 1.0 reworks the input splitter framework so
perhaps it is fixed,  but looking at the github sources I don't see any
additional catching of exceptions. Though I haven't actually tried with the
new IPython alpha since Sage still needs some work to use it.

More details at http://trac.sagemath.org/14961



--
View this message in context: http://python.6.x6.nabble.com/InputSplitter-and-SyntaxError-tp5025938.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From jason-sage at creativetrax.com  Wed Jul 24 22:42:25 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 24 Jul 2013 19:42:25 -0700
Subject: [IPython-dev] InputSplitter and SyntaxError
In-Reply-To: <1374707823022-5025938.post@n6.nabble.com>
References: <1374707823022-5025938.post@n6.nabble.com>
Message-ID: <51F09091.70001@creativetrax.com>

On 7/24/13 4:17 PM, Volker Braun wrote:
> In IPython 0.13.2, if the input_splitter.push() method raises an exception
> then it is not caught, terminating the IPython process. Sage defines a
> special syntax for some mathematical objects and we would like to have a way
> to tell the user about misbalanced brackets, but I don't see how that is
> supposed to work. Is there a way?
>
> I see that the new IPython 1.0 reworks the input splitter framework so
> perhaps it is fixed,  but looking at the github sources I don't see any
> additional catching of exceptions. Though I haven't actually tried with the
> new IPython alpha since Sage still needs some work to use it.
>


FYI Volker: I posted patches for Sage to work with IPython 1.0 (as of a 
few weeks ago, anyway):

http://trac.sagemath.org/ticket/14713

Thanks,

Jason



From drrt at cs.utexas.edu  Thu Jul 25 02:04:58 2013
From: drrt at cs.utexas.edu (drrt)
Date: Wed, 24 Jul 2013 23:04:58 -0700 (PDT)
Subject: [IPython-dev] A couple of tests don't pass when using Python 3.3
In-Reply-To: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
Message-ID: <1374732298031-5025954.post@n6.nabble.com>

I installed this alpha version of IPython using Python 3.3 and everything
else from scratch using MacPorts.
I would appreciate if you can tell me what the following errors mean? and
how can I fix them?
Thanks

This is one of the tests that failed


and the other one is an issue with matplotlib





--
View this message in context: http://python.6.x6.nabble.com/ANN-IPython-1-0-0a1-tp5025537p5025954.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From martin.zmk at gmail.com  Thu Jul 25 07:15:12 2013
From: martin.zmk at gmail.com (lecast)
Date: Thu, 25 Jul 2013 04:15:12 -0700 (PDT)
Subject: [IPython-dev] Extensions arguments and cell types
Message-ID: <1374750912235-5025980.post@n6.nabble.com>

1. Is it possible to pass arguments to extensions on %(re)load_ext or do I
have to create a line/cell magic for that purpose only?

2. Can an extension add a cell type in the notebook? I want to write an
extension that would work as if it was a Markdown cell (i.e. after execute
you see only the output). Alternatively can I, through an extension, add css
classes to cells using cell magics from that extension AND add on load
css/js that collapses those cells. 




--
View this message in context: http://python.6.x6.nabble.com/Extensions-arguments-and-cell-types-tp5025980.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From bussonniermatthias at gmail.com  Thu Jul 25 10:35:02 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 25 Jul 2013 07:35:02 -0700
Subject: [IPython-dev] Extensions arguments and cell types
In-Reply-To: <1374750912235-5025980.post@n6.nabble.com>
References: <1374750912235-5025980.post@n6.nabble.com>
Message-ID: <4176C83F-60D5-46C7-9847-20C59A1EB641@gmail.com>


Le 25 juil. 2013 ? 04:15, lecast a ?crit :

> 1. Is it possible to pass arguments to extensions on %(re)load_ext or do I
> have to create a line/cell magic for that purpose only?

I'm not sure, what would be the use case ?

> 
> 2. Can an extension add a cell type in the notebook?

No.

> I want to write an
> extension that would work as if it was a Markdown cell (i.e. after execute
> you see only the output). Alternatively can I, through an extension, add css
> classes to cells using cell magics from that extension AND add on load
> css/js that collapses those cells. 

Those look a lot more that JS extensions you could attach arbitrary info to cell, 
and with a custom javascript with a js extension you could add css to those cells
at load time. The cell toolbar UI should help you to tag cell to do that.

have a look in IPython/html/static/notebook/js/celltoolbarpresets

-- 
Matthias




> --
> View this message in context: http://python.6.x6.nabble.com/Extensions-arguments-and-cell-types-tp5025980.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From fereyes at lbl.gov  Thu Jul 25 10:43:51 2013
From: fereyes at lbl.gov (Francis Reyes)
Date: Thu, 25 Jul 2013 07:43:51 -0700
Subject: [IPython-dev] ipython 1.0.0a1: ImportError: No module named
	IPython.kernel.zmq.kernelapp
Message-ID: <603C6C88-58FD-47C3-AE8D-A5229899762F@lbl.gov>

Using (tag) 1.0.0a1, I cannot seem to get ipython to work correctly.  Granted I might have a special case (Python is from a ContinuumIO installation in my home directory). 

The kernel crashes immediately when I enter text into any code cell. 

appending the ipython directory to $PYTHONPATH remedies the situation as now the modules are all accessible to python.


------ log ----


07:37:55-francisreyes at kenya:$ which python
/Users/francisreyes/anaconda/bin/python
07:38:13-francisreyes at kenya:$ python -m IPython notebook --profile=foo
2013-07-25 07:38:19.695 [NotebookApp] Using existing profile dir: u'/Users/francisreyes/.ipython/profile_foo'
2013-07-25 07:38:19.794 [NotebookApp] Using MathJax from CDN: http://cdn.mathjax.org/mathjax/latest/MathJax.js
2013-07-25 07:38:19.809 [NotebookApp] Serving notebooks from local directory: /Users/francisreyes/notebooks
2013-07-25 07:38:19.809 [NotebookApp] The IPython Notebook is running at: http://127.0.0.1:8888/
2013-07-25 07:38:19.809 [NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/osascript) is code signed with entitlements
2013-07-25 07:38:24.945 [NotebookApp] Connecting to: tcp://127.0.0.1:54245
2013-07-25 07:38:24.947 [NotebookApp] Kernel started: d39fb48f-03e2-47d0-892c-ffbbc2a43ab9
2013-07-25 07:38:24.958 [NotebookApp] Connecting to: tcp://127.0.0.1:54242
2013-07-25 07:38:24.958 [NotebookApp] Connecting to: tcp://127.0.0.1:54244
2013-07-25 07:38:24.959 [NotebookApp] Connecting to: tcp://127.0.0.1:54243
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named IPython.kernel.zmq.kernelapp
2013-07-25 07:38:27.945 [NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel d39fb48f-03e2-47d0-892c-ffbbc2a43ab9 restarted
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named IPython.kernel.zmq.kernelapp


Francis Reyes
Advanced Light Source
1 Cyclotron Road
Berkeley, CA 94720





From martin.zmk at gmail.com  Thu Jul 25 10:53:18 2013
From: martin.zmk at gmail.com (lecast)
Date: Thu, 25 Jul 2013 07:53:18 -0700 (PDT)
Subject: [IPython-dev] Extensions arguments and cell types
In-Reply-To: <4176C83F-60D5-46C7-9847-20C59A1EB641@gmail.com>
References: <1374750912235-5025980.post@n6.nabble.com>
	<4176C83F-60D5-46C7-9847-20C59A1EB641@gmail.com>
Message-ID: <1374763998546-5026027.post@n6.nabble.com>

Use case would be like in http://nbviewer.ipython.org/6080008/ipyBibtex.ipynb

Where I want to do 

%load_ext ipyBibtex path/to/file.bib 3

Instead of:
%load_ext ipyBibtex
%etal 3
%bibFile path/to/file.bib

As for js extensions are you referring me to
https://github.com/ipython-contrib/IPython-notebook-extensions ? And can
%load_ext be used to load the bundle (py+js)?  



--
View this message in context: http://python.6.x6.nabble.com/Extensions-arguments-and-cell-types-tp5025980p5026027.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From benjaminrk at gmail.com  Thu Jul 25 13:26:12 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 25 Jul 2013 10:26:12 -0700
Subject: [IPython-dev] ipython 1.0.0a1: ImportError: No module named
	IPython.kernel.zmq.kernelapp
In-Reply-To: <603C6C88-58FD-47C3-AE8D-A5229899762F@lbl.gov>
References: <603C6C88-58FD-47C3-AE8D-A5229899762F@lbl.gov>
Message-ID: <CAHNn8BWU3tPLNZXLOBNnB4ptQDaVNO9whtZEAwKTQbhKfxu9rw@mail.gmail.com>

If that file doesn?t exist, then you aren?t using IPython 1.0. You may have
multiple IPython installs, and an old one showing up ahead on sys.path.

For instance, where did the latest IPython install say it went? And if you
just do:

python -c "import IPython; print IPython"

Does it point to the same place?

-MinRK


On Thu, Jul 25, 2013 at 7:43 AM, Francis Reyes <fereyes at lbl.gov> wrote:

> Using (tag) 1.0.0a1, I cannot seem to get ipython to work correctly.
>  Granted I might have a special case (Python is from a ContinuumIO
> installation in my home directory).
>
> The kernel crashes immediately when I enter text into any code cell.
>
> appending the ipython directory to $PYTHONPATH remedies the situation as
> now the modules are all accessible to python.
>
>
> ------ log ----
>
>
> 07:37:55-francisreyes at kenya:$ which python
> /Users/francisreyes/anaconda/bin/python
> 07:38:13-francisreyes at kenya:$ python -m IPython notebook --profile=foo
> 2013-07-25 07:38:19.695 [NotebookApp] Using existing profile dir:
> u'/Users/francisreyes/.ipython/profile_foo'
> 2013-07-25 07:38:19.794 [NotebookApp] Using MathJax from CDN:
> http://cdn.mathjax.org/mathjax/latest/MathJax.js
> 2013-07-25 07:38:19.809 [NotebookApp] Serving notebooks from local
> directory: /Users/francisreyes/notebooks
> 2013-07-25 07:38:19.809 [NotebookApp] The IPython Notebook is running at:
> http://127.0.0.1:8888/
> 2013-07-25 07:38:19.809 [NotebookApp] Use Control-C to stop this server
> and shut down all kernels (twice to skip confirmation).
> dyld: DYLD_ environment variables being ignored because main executable
> (/usr/bin/osascript) is code signed with entitlements
> 2013-07-25 07:38:24.945 [NotebookApp] Connecting to: tcp://127.0.0.1:54245
> 2013-07-25 07:38:24.947 [NotebookApp] Kernel started:
> d39fb48f-03e2-47d0-892c-ffbbc2a43ab9
> 2013-07-25 07:38:24.958 [NotebookApp] Connecting to: tcp://127.0.0.1:54242
> 2013-07-25 07:38:24.958 [NotebookApp] Connecting to: tcp://127.0.0.1:54244
> 2013-07-25 07:38:24.959 [NotebookApp] Connecting to: tcp://127.0.0.1:54243
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
> ImportError: No module named IPython.kernel.zmq.kernelapp
> 2013-07-25 07:38:27.945 [NotebookApp] KernelRestarter: restarting kernel
> (1/5)
> WARNING:root:kernel d39fb48f-03e2-47d0-892c-ffbbc2a43ab9 restarted
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
> ImportError: No module named IPython.kernel.zmq.kernelapp
>
>
> Francis Reyes
> Advanced Light Source
> 1 Cyclotron Road
> Berkeley, CA 94720
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130725/b2e8421a/attachment.html>

From sychan at lbl.gov  Thu Jul 25 13:39:53 2013
From: sychan at lbl.gov (Stephen Chan)
Date: Thu, 25 Jul 2013 10:39:53 -0700
Subject: [IPython-dev] ipython 1.0.0a1: ImportError: No module named
	IPython.kernel.zmq.kernelapp
In-Reply-To: <CAHNn8BWU3tPLNZXLOBNnB4ptQDaVNO9whtZEAwKTQbhKfxu9rw@mail.gmail.com>
References: <603C6C88-58FD-47C3-AE8D-A5229899762F@lbl.gov>
	<CAHNn8BWU3tPLNZXLOBNnB4ptQDaVNO9whtZEAwKTQbhKfxu9rw@mail.gmail.com>
Message-ID: <CA+n9YfoqWAGSv27WDRrXWoZ8W02uwaXjAY8Yixzo+WYBwbVbnA@mail.gmail.com>

   I came across a similar problem the other day when loading the 1.0dev
IPython and trying to run IPython.test().
   If you clone the repo, cd into it and bring up python and then import
IPython, it will pickup the new IPython modules from your cwd (
IPython.__file__ pointed where I expected). But at least when I tried to
run IPython.test(), it no longer used my cwd as part of the module load
path and it defaulted to the IPython I had installed systemwide. If I
explicitly set an absolute path in PYTHONPATH, things would work.

   This sounds like what is happening to Francis.

   Steve


On Thu, Jul 25, 2013 at 10:26 AM, MinRK <benjaminrk at gmail.com> wrote:

> If that file doesn?t exist, then you aren?t using IPython 1.0. You may
> have multiple IPython installs, and an old one showing up ahead on sys.path.
>
> For instance, where did the latest IPython install say it went? And if you
> just do:
>
> python -c "import IPython; print IPython"
>
> Does it point to the same place?
>
> -MinRK
>
>
> On Thu, Jul 25, 2013 at 7:43 AM, Francis Reyes <fereyes at lbl.gov> wrote:
>
>> Using (tag) 1.0.0a1, I cannot seem to get ipython to work correctly.
>>  Granted I might have a special case (Python is from a ContinuumIO
>> installation in my home directory).
>>
>> The kernel crashes immediately when I enter text into any code cell.
>>
>> appending the ipython directory to $PYTHONPATH remedies the situation as
>> now the modules are all accessible to python.
>>
>>
>> ------ log ----
>>
>>
>> 07:37:55-francisreyes at kenya:$ which python
>> /Users/francisreyes/anaconda/bin/python
>> 07:38:13-francisreyes at kenya:$ python -m IPython notebook --profile=foo
>> 2013-07-25 07:38:19.695 [NotebookApp] Using existing profile dir:
>> u'/Users/francisreyes/.ipython/profile_foo'
>> 2013-07-25 07:38:19.794 [NotebookApp] Using MathJax from CDN:
>> http://cdn.mathjax.org/mathjax/latest/MathJax.js
>> 2013-07-25 07:38:19.809 [NotebookApp] Serving notebooks from local
>> directory: /Users/francisreyes/notebooks
>> 2013-07-25 07:38:19.809 [NotebookApp] The IPython Notebook is running at:
>> http://127.0.0.1:8888/
>> 2013-07-25 <http://127.0.0.1:8888/2013-07-25> 07:38:19.809 [NotebookApp]
>> Use Control-C to stop this server and shut down all kernels (twice to skip
>> confirmation).
>> dyld: DYLD_ environment variables being ignored because main executable
>> (/usr/bin/osascript) is code signed with entitlements
>> 2013-07-25 07:38:24.945 [NotebookApp] Connecting to: tcp://
>> 127.0.0.1:54245
>> 2013-07-25 07:38:24.947 [NotebookApp] Kernel started:
>> d39fb48f-03e2-47d0-892c-ffbbc2a43ab9
>> 2013-07-25 07:38:24.958 [NotebookApp] Connecting to: tcp://
>> 127.0.0.1:54242
>> 2013-07-25 07:38:24.958 [NotebookApp] Connecting to: tcp://
>> 127.0.0.1:54244
>> 2013-07-25 07:38:24.959 [NotebookApp] Connecting to: tcp://
>> 127.0.0.1:54243
>> Traceback (most recent call last):
>>   File "<string>", line 1, in <module>
>> ImportError: No module named IPython.kernel.zmq.kernelapp
>> 2013-07-25 07:38:27.945 [NotebookApp] KernelRestarter: restarting kernel
>> (1/5)
>> WARNING:root:kernel d39fb48f-03e2-47d0-892c-ffbbc2a43ab9 restarted
>> Traceback (most recent call last):
>>   File "<string>", line 1, in <module>
>> ImportError: No module named IPython.kernel.zmq.kernelapp
>>
>>
>> Francis Reyes
>> Advanced Light Source
>> 1 Cyclotron Road
>> Berkeley, CA 94720
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130725/15397ed4/attachment.html>

From tarun.gaba7 at gmail.com  Thu Jul 25 14:43:22 2013
From: tarun.gaba7 at gmail.com (TARUN GABA)
Date: Fri, 26 Jul 2013 00:13:22 +0530
Subject: [IPython-dev] Regarding Import for a Package
Message-ID: <CAHAono3bJBpJFFt_ZUayx0ABKBsY+RLfb_gCXEoMcf5pSiUkpA@mail.gmail.com>

Hi,
I am developing a Package under PyDy,  for GSoC.
This is my directory structure.

pydy/
--------- __init__.py
---------pydy_viz/
                    -----------------__init__.py
                    --------------shapes.py
                    --------------tests\
                                    -----------__init__.py
                                    -----------test_shapes.py

Now I am trying to import pydy_viz.shapes from inside tests, i.e from
test_shapes.py

I have tried both approaches.
from pydy_viz.shapes import *
Error: No module named pydy_viz.shapes
and relatively ..

from ..shapes import *
Error: tried relative imports from non-package.

but I am not able to import it.
Any ideas what I might be doing wrong.
I dont have a setup.py file in place for now.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130726/2fd78be5/attachment.html>

From ntezak at stanford.edu  Thu Jul 25 14:52:12 2013
From: ntezak at stanford.edu (Nikolas Tezak)
Date: Thu, 25 Jul 2013 11:52:12 -0700
Subject: [IPython-dev] Regarding Import for a Package
In-Reply-To: <CAHAono3bJBpJFFt_ZUayx0ABKBsY+RLfb_gCXEoMcf5pSiUkpA@mail.gmail.com>
References: <CAHAono3bJBpJFFt_ZUayx0ABKBsY+RLfb_gCXEoMcf5pSiUkpA@mail.gmail.com>
Message-ID: <E256C50C-B6CD-4A3B-940C-6DEA1E9EFF17@stanford.edu>

Hi!

If the directory containing the top-level pydy module is already in your sys.path variable, the smartest thing IMHO is to use absolute imports including the whole tree of modules.

from pydy.pydy_viz.shapes import *

To add it to the sys.path, you modify your system's  PYTHONPATH environment variable. 
Since I don't know any details of your runtime setup, here is a way to find out more ;)
https://www.google.com/search?q=setting+pythonpath

Best,

Nik


On Jul 25, 2013, at 11:43 AM, TARUN GABA wrote:

> Hi,
> I am developing a Package under PyDy,  for GSoC.
> This is my directory structure.
> 
> pydy/
> --------- __init__.py
> ---------pydy_viz/
>                     -----------------__init__.py
>                     --------------shapes.py
>                     --------------tests\
>                                     -----------__init__.py
>                                     -----------test_shapes.py
> 
> Now I am trying to import pydy_viz.shapes from inside tests, i.e from test_shapes.py
> 
> I have tried both approaches.
> from pydy_viz.shapes import *
> Error: No module named pydy_viz.shapes
> and relatively ..
> 
> from ..shapes import *
> Error: tried relative imports from non-package.
> 
> but I am not able to import it.
> Any ideas what I might be doing wrong.
> I dont have a setup.py file in place for now.
> 
> Thanks
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Thu Jul 25 18:45:04 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 25 Jul 2013 23:45:04 +0100
Subject: [IPython-dev] A couple of tests don't pass when using Python 3.3
In-Reply-To: <1374732298031-5025954.post@n6.nabble.com>
References: <CAHNn8BUBw+56QT2zhgK8g_2yCzLX7AH5OSkdg5xZxGz5rBi+7g@mail.gmail.com>
	<1374732298031-5025954.post@n6.nabble.com>
Message-ID: <CAOvn4qhnHgr31DgZtqSDDNX+Y0ddo5dAA4mgy6tfiA-fUrOh9A@mail.gmail.com>

On 25 July 2013 07:04, drrt <drrt at cs.utexas.edu> wrote:

> I would appreciate if you can tell me what the following errors mean? and
> how can I fix them?
>

The Pylab errors should be fixed by PR #3722:
https://github.com/ipython/ipython/pull/3722/files

Not sure about the pexpect issue. If it's consistent, please file an issue
for it.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130725/e0a28b7a/attachment.html>

From takowl at gmail.com  Thu Jul 25 18:50:37 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 25 Jul 2013 23:50:37 +0100
Subject: [IPython-dev] Extensions arguments and cell types
In-Reply-To: <1374763998546-5026027.post@n6.nabble.com>
References: <1374750912235-5025980.post@n6.nabble.com>
	<4176C83F-60D5-46C7-9847-20C59A1EB641@gmail.com>
	<1374763998546-5026027.post@n6.nabble.com>
Message-ID: <CAOvn4qgOMiwNKmbagcU2ZdW=Ye1H7_YU5icXC7mNEZ_wzatMHQ@mail.gmail.com>

On 25 July 2013 15:53, lecast <martin.zmk at gmail.com> wrote:

> Where I want to do
>
> %load_ext ipyBibtex path/to/file.bib 3
>

I don't think we're going to support passing arguments when loading an
extension. You could make a magic command like '%bibtex path/to/file.bib
3', and configure it to load the extension on startup, if you want to save
typing.


> As for js extensions are you referring me to
> https://github.com/ipython-contrib/IPython-notebook-extensions ? And can
> %load_ext be used to load the bundle (py+js)?
>

At present, %load_ext is only for .py extensions. We plan to work out a
good way to have py+js extensions, but it's not there yet.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130725/45623fac/attachment.html>

From jason-sage at creativetrax.com  Sat Jul 27 19:58:37 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 27 Jul 2013 16:58:37 -0700
Subject: [IPython-dev] manipulates/interacts
Message-ID: <51F45EAD.3010803@creativetrax.com>

I see on the sprint planning page [1] that there was a 
manipulates/interact conversation planned.  Could someone summarize the 
discussions on a blog or the wiki or something?

It'd be great to have a summary of the rest of the meeting as well.  I 
realize, of course, that many people are still traveling back today, so 
no hurry, of course.  The rest of us are just really interested.

Thanks,

Jason


From raymond.yee at gmail.com  Sat Jul 27 20:03:25 2013
From: raymond.yee at gmail.com (Raymond Yee)
Date: Sat, 27 Jul 2013 17:03:25 -0700
Subject: [IPython-dev] manipulates/interacts
In-Reply-To: <51F45EAD.3010803@creativetrax.com>
References: <51F45EAD.3010803@creativetrax.com>
Message-ID: <51F45FCD.5050301@gmail.com>

Until  fuller written narratives arise, take a look at hackpad:

https://hackpad.com/IPython-Summer-2013-Development-Meeting-D1UR23usGnA

-Raymond


On 7/27/13 4:58 PM, Jason Grout wrote:
> I see on the sprint planning page [1] that there was a 
> manipulates/interact conversation planned.  Could someone summarize the 
> discussions on a blog or the wiki or something?
>
> It'd be great to have a summary of the rest of the meeting as well.  I 
> realize, of course, that many people are still traveling back today, so 
> no hurry, of course.  The rest of us are just really interested.
>
> Thanks,
>
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From anfedorov at gmail.com  Sun Jul 28 18:52:39 2013
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Sun, 28 Jul 2013 15:52:39 -0700
Subject: [IPython-dev] Setting up ipython environment when notebook starts up
Message-ID: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>

I'd like to run an ipython notebook server with a custom env setup script
which runs when a kernel starts up that allows it to access the Google App
Engine data store (mimicking the
remote_api_shell.py<https://googleappengine.googlecode.com/svn/trunk/python/google/appengine/tools/remote_api_shell.py>tool).
It needs to set some authentication variables and and
modify the namespace it runs in.

Is there functionality to customize a notebook installation this way? If
not, could someone point me to where in the code the ipython kernel is
created when a new notebook is made?

Cheers,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130728/e51e96d5/attachment.html>

From anfedorov at gmail.com  Sun Jul 28 20:05:49 2013
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Sun, 28 Jul 2013 17:05:49 -0700
Subject: [IPython-dev] Setting up ipython environment when notebook
	starts up
In-Reply-To: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
Message-ID: <CAESh9jUuWbELDDybUjdTYKKLYpOktEv58AcoduCXeRKG4UMGCw@mail.gmail.com>

Looks like throwing together a startup script in

  ~/.ipython/profile_default/startup/

did the trick. Sorry for the bother!


On Sun, Jul 28, 2013 at 3:52 PM, Andrey Fedorov <anfedorov at gmail.com> wrote:

> I'd like to run an ipython notebook server with a custom env setup script
> which runs when a kernel starts up that allows it to access the Google App
> Engine data store (mimicking the remote_api_shell.py<https://googleappengine.googlecode.com/svn/trunk/python/google/appengine/tools/remote_api_shell.py>tool). It needs to set some authentication variables and and
> modify the namespace it runs in.
>
> Is there functionality to customize a notebook installation this way? If
> not, could someone point me to where in the code the ipython kernel is
> created when a new notebook is made?
>
> Cheers,
> Andrey
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130728/7ae6d123/attachment.html>

From gvwilson at third-bit.com  Mon Jul 29 11:13:17 2013
From: gvwilson at third-bit.com (Greg Wilson)
Date: Mon, 29 Jul 2013 11:13:17 -0400
Subject: [IPython-dev] prose, notebooks,
	and thrashing (and a re-direct to another list)
In-Reply-To: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
Message-ID: <51F6868D.1050903@third-bit.com>

Hi,
I posted the message below to the Software Carpentry discussion list a 
few minutes ago.  I'd appreciate input from the IPython Notebook 
community as well.
Thanks,
Greg
p.s. if anyone has a good way to manage discussion that spans two 
independent mailing lists, I'm all ears... :-)

---------------------------------------------------------------------------------------------------

Hi,

I'm trying to figure out how to satisfy four sets of needs for 
instructional material using IPython Notebooks in Software Carpentry. I 
feel like I'm thrashing, so I'd appreciate your help.

My four use cases are:

* instructor's guide: to help instructors prepare for a bootcamp
* lecture notes: what instructors use during a bootcamp
* workbook: what learners are given to use during a bootcamp
* textbook: what learners use on their own outside a bootcamp

To make this more concrete, have a look at 
http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html, which 
describes how I teach the shell:

* 
http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:instructors 
is a high-level intro for instructors.  Learners might find it useful, 
but it's not really aimed at them.

* Each topic opens with objectives (see 
http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:objectives 
for an example).  This belongs in the instructor's guide ("Here's what 
you're supposed to teach"), possibly the lecture notes ("Hi class, 
here's what I'm about to teach"), probably not the workbook, and 
definitely the textbook.

* Each topic ends with key points and challenge exercises: see 
http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:keypoints, 
and 
http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:challenges 
for examples.  Again, both belong in the instructor's guide and the 
textbook; they probably belong in the lecture notes, and probably also 
in the workbook (as a starting point for learners' own notes, and so 
that they know what to work on during practicals).

Problem #1: how do we manage material that ought to appear in multiple 
documents?  Right now, each snippet is a separate HTML file that is 
%include'd to assemble the entire document (see 
https://raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-shell/tutorial.html). 
That works OK for HTML pages, but not for notebooks: as far as I know, 
there's no way to %include file X in notebook Y.

Problem #2: what exactly ought to be in the lecture notes?  For example, 
compare 
http://swcarpentry.github.io/bc/lessons/guide-pyblocks/tutorial.html#s:pyblocks:logic:lesson 
(the introduction to "if/else" in the incomplete instructor's guide to 
basic Python programming using IPythonBlocks) to 
http://nbviewer.ipython.org/urls/raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-pyblocks/pyblocks-logic.ipynb 
(an IPython Notebook with the same handful of examples).  The former is 
much too verbose to put in front of learners in a classroom setting.  I 
feel the latter is too sparse, but when I add point-form notes and then 
lecture over the result, it feels clumsy:

* If the item I'm currently talking about is centered on the screen, 
learners can see my next points on screen below it.  This is distracting 
--- so distracting that tools like PowerPoint and other slideshow tools 
were invented in part to prevent it happening.  If the notebook 
supported progressive reveal of pre-made cells, that problem would lessen.

* Even with that, though, one of the most compelling reasons to use the 
notebook as a lecture tool is lost.  Learners really like us typing code 
in as we teach: it slows us down, which makes it more likely that we'll 
walk through the code at a comprehensible pace, and it makes it a lot 
easier for us to extemporize in response to their questions or 
confusion.  We lose both benefits if we start using "reveal next", 
though we're still better off than if we use slides.

Problem #3: what ought to be in the workbook (the notebook we give 
students as a starting point)?  Option A is "nothing": they should type 
things in as the lecture proceeds.  This forces them to learn hands-on 
(which is good), but greatly increases the risk that they'll fall behind:

* They're more likely to mis-type something than the instructor, so 
their actual lines-per-second is (much) lower.
* They want to take notes, which slows them down even further.

Option B is to give learners a notebook with notes and code already in 
it, but that gets us back to them (a) being distracted by material they 
haven't seen yet, but which is already on their screen, and (b) not 
having to be hands-on while learning.  On the other hand, these are 
grownups, and there's only so much we can do with existing tools.

So here's what I think I've come to:

1. For now, we create two documents: an instructor's guide that doubles 
as a textbook (no harm in learners reading notes for instructors, and 
possibly some benefit), and a notebook that interleaves point-form notes 
with code examples that instructors will use for lecturing, and learners 
will be given as a starting point.

2. We build a tool to extract point-form notes from an IPython Notebook 
to create the "objectives.html" and "keypoints.html" sections that are 
%include'd in #1 above.  This will help keep the notebook and the guide 
in step; whoever changes the notebook will still have to read the 
section of the guide to make sure, but it's a step.

3. The code cells in our notebooks have markers for the instructors and 
the learners to fill in.  For example, instead of giving people:

     for char in "GATTACA":
         if char == "A":
             print "found an A"

we have:

     for ________ in "GATTACA":
         if ________:
             print "found an A"

Filling in the ________'s in the code snippets forces the instructor and 
the learners to do some typing. As a bonus, if chosen judiciously they 
will also encourage both to focus on the key features of the example.

But there are problems.  (You knew there would be...)

* (medium-sized) In order to extract stuff from notebooks for use 
elsewhere, we need a way to tag the cells that contain objectives, notes 
for the key point summary, etc., so that our tool can find and 
differentiate them.  I *think* this can be done via a plugin: the 
notebook has an extensible metadata field per cell, which we've already 
exploited for another tool, so adding something that allows "right click 
and select tag" should be straightforward. It would be nice if this info 
was available as a class on the HTML element containing the material, so 
that it could be styled via CSS, but that's a bonus.

* (large) Literal ________'s in our code will make it invalid, so our 
notebooks either won't have any output (because we didn't execute the 
code) or will be littered with error messages (because we did).  The 
former's not bad, but if we do that, we need two version of each 
notebook: one with the blanks, and one with them filled in.  As always, 
keeping variants of a single document in sync will be a pain.

If the code in cells was stored as HTML, we could put something like 
<span class="fillin">char == "A"</span> in the code, then use Javascript 
to display ________, erase it when the learner started typing, or fill 
it in with the right answer when the learner just wanted to catch up or 
see the answer. Problem is, the code *isn't* stored as HTML; it's stored 
as JSON that looks like this:

     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "import math\n",
       "print \"pi is\", math.pi\n",
       "print \"square root of 5 is\", math.sqrt(5)"
      ],
      "language": "python",
      "metadata": {},
      "outputs": [
       {
        "output_type": "stream",
        "stream": "stdout",
        "text": [
         "pi is 3.14159265359\n",
         "square root of 5 is 2.2360679775\n"
        ]
       }
      ],
      "prompt_number": 1
     }

We could use character offsets to identify subsections of the code, then 
intercept rendering to insert span elements, but that way lies madness...

So as I said at the outset, I feel like I'm thrashing here, and would 
welcome suggestions on how to proceed.

Thanks,
Greg



From bussonniermatthias at gmail.com  Mon Jul 29 12:06:06 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 29 Jul 2013 18:06:06 +0200
Subject: [IPython-dev] prose, notebooks,
	and thrashing (and a re-direct to another list)
In-Reply-To: <51F6868D.1050903@third-bit.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
Message-ID: <D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>

Hi Greg, 

Le 29 juil. 2013 ? 17:13, Greg Wilson a ?crit :

> Hi,
> I posted the message below to the Software Carpentry discussion list a 
> few minutes ago.  I'd appreciate input from the IPython Notebook 
> community as well.
> Thanks,
> Greg
> p.s. if anyone has a good way to manage discussion that spans two 
> independent mailing lists, I'm all ears... :-)

I'll be carefull to click reply all :-)

Most of the point you are making were discussed during last week meeting.
TL DR, depending on the case , it already exist, or you will have to wait for 2.0.

Answer inline

> My four use cases are:
> 
> * instructor's guide: to help instructors prepare for a bootcamp
> * lecture notes: what instructors use during a bootcamp
> * workbook: what learners are given to use during a bootcamp
> * textbook: what learners use on their own outside a bootcamp
> 


> Problem #1: how do we manage material that ought to appear in multiple 
> documents?  Right now, each snippet is a separate HTML file that is 
> %include'd to assemble the entire document (see 
> https://raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-shell/tutorial.html). 
> That works OK for HTML pages, but not for notebooks: as far as I know, 
> there's no way to %include file X in notebook Y.

2.0 and probably will need dexy. (http://www.dexy.it/)
right now you can import one notebook as a python package 
We will add the possibility to add cell tags (~css .class selector) and cell name (~ html #id)
to refer to cell in conversion proccess.

[NDR, you can already attach arbitrary json to cells, it's just a matter of defining an expected structure.
Thoses data are already accessible in nbconvert if you want some cells not to appear in converted output.
]

> * If the item I'm currently talking about is centered on the screen, 
> learners can see my next points on screen below it.  This is distracting 
> --- so distracting that tools like PowerPoint and other slideshow tools 
> were invented in part to prevent it happening.  If the notebook 
> supported progressive reveal of pre-made cells, that problem would lessen.

... Have you seen slideshow mode ?
Have a look at F. Perez, B Granger and P. Ivanov talk ant las scipy.

Show cell one by one, skip some cells... etc
this is still a live notebook so you can execute cells live.
(won't provide the link as it is still in dev but google is your friend)


> 
> * Even with that, though, one of the most compelling reasons to use the 
> notebook as a lecture tool is lost.  Learners really like us typing code 
> in as we teach: it slows us down, which makes it more likely that we'll 
> walk through the code at a comprehensible pace, and it makes it a lot 
> easier for us to extemporize in response to their questions or 
> confusion.  We lose both benefits if we start using "reveal next", 
> though we're still better off than if we use slides.

I had a POC of revealing code cell by blocks.

> ...
> 
> * (medium-sized) In order to extract stuff from notebooks for use 
> elsewhere, we need a way to tag the cells that contain objectives, notes 
> for the key point summary, etc., so that our tool can find and 
> differentiate them.  I *think* this can be done via a plugin: the 
> notebook has an extensible metadata field per cell, which we've already 
> exploited for another tool, so adding something that allows "right click 
> and select tag" should be straightforward. It would be nice if this info 
> was available as a class on the HTML element containing the material, so 
> that it could be styled via CSS, but that's a bonus.

Yes, that's on my todo list. This would require SC to ship notebook with custom profiles.
There is just some edge cases at loading time.

> 
> * (large) Literal ________'s in our code will make it invalid, so our 
> notebooks either won't have any output (because we didn't execute the 
> code) or will be littered with error messages (because we did).  The 
> former's not bad, but if we do that, we need two version of each 
> notebook: one with the blanks, and one with them filled in.  As always, 
> keeping variants of a single document in sync will be a pain.

You could store the "answers" into the metadata of each cell that maps the Nth ____
to it's value. We can come up with something.

> 
> If the code in cells was stored as HTML, we could put something like 
> <span class="fillin">char == "A"</span> in the code, then use Javascript 
> to display ________, erase it when the learner started typing, or fill 
> it in with the right answer when the learner just wanted to catch up or 
> see the answer. Problem is, the code *isn't* stored as HTML; it's stored 
> as JSON that looks like this:

If you are diving into implementation I would have something like notebook creator typing:

 for ?char? in "GATTACA":
        if ?char == "A"?:
            print "found an A"

and a cell toolbar button that store this in the json MD, and at load time you can live replace by ____
or strip the ?. it should even be possible to add a execute hook to code cell that strip the ? before running the cell
which would be convenient.

Not sure on how to live replace ___ if learner start typing though.


> We could use character offsets to identify subsections of the code, then 
> intercept rendering to insert span elements, but that way lies madness...
> 
> So as I said at the outset, I feel like I'm thrashing here, and would 
> welcome suggestions on how to proceed.

Will gladly try to help, most of what you ask is often requested, and I would love to have more time to help.
Sadly PhD (and code IPython first), but I'll try to describe in a blog post how most of thoses things can be done.
It will just take me some time as I wish to do them well instead of just publishing a hawkish POC.

-- 
Matthias

From bussonniermatthias at gmail.com  Mon Jul 29 12:16:32 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 29 Jul 2013 18:16:32 +0200
Subject: [IPython-dev] [FW from User list] NB convert?
In-Reply-To: <CAAaQZevKdDkhWwLQ+QO4pCRS5rkbPeC+MmK5=6ATjDnAWOdq6w@mail.gmail.com>
References: <CAAaQZevKdDkhWwLQ+QO4pCRS5rkbPeC+MmK5=6ATjDnAWOdq6w@mail.gmail.com>
Message-ID: <7F9A7204-4C75-4E50-900F-39ACF7FFFF30@gmail.com>

From User-List, response here as we are trying to deprecate ipython-user.
Will send link to this thread to OP. Voluntarily  not cc-ing -user and OP 
to push them into unsubscribing -user and subscribe -dev.


Le 26 juil. 2013 ? 17:57, Tom Brander a ?crit :

> Just installed 1 beta on Ubuntu 12.10(64) Very smooth process.
> but can't seem to get nb convert going, initially it complained about no sphinx which I installed the I get the following:
> tom at tom-sam:~$ ipython nbconvert Heatherwood.ipynb
> >>>>>Last few lines of traceback below:
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/utils/pandoc.py", line 51, in pandoc
>     stdin=subprocess.PIPE, stdout=subprocess.PIPE
>   File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
>     errread, errwrite)
>   File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
>     raise child_exception
> OSError: [Errno 2] No such file or directory
> 
> Not too sure what it is looking for?

Error message should now be more explicit on master.
You need to install pandoc I think.
-- 
Matt

From ellisonbg at gmail.com  Mon Jul 29 13:31:57 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 29 Jul 2013 10:31:57 -0700
Subject: [IPython-dev] prose, notebooks,
 and thrashing (and a re-direct to another list)
In-Reply-To: <51F6868D.1050903@third-bit.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
Message-ID: <CAH4pYpR192nBzEc6CLsB=obo0dYDEsPEnwAUwfKXZdD7Gr+J_w@mail.gmail.com>

Greg,

Here is what I would do *today*.

* I would author a single notebook that has all of the content.
* I would use cell level metadata to add to the tags list.  At the dev
meeting we decided to add a top level tags key that is a list of tags
to the cell level metadata.
* I would write a custom Transformer for nbconvert that filters
notebook cells based on these tags.
* I would run run nbconvert multiple times to produce the different notebooks.

Sorry I am so brief - we are moving this week so I am mostly offline.
Other people can help with the details of these things.

Cheers,

Brian

On Mon, Jul 29, 2013 at 8:13 AM, Greg Wilson <gvwilson at third-bit.com> wrote:
> Hi,
> I posted the message below to the Software Carpentry discussion list a
> few minutes ago.  I'd appreciate input from the IPython Notebook
> community as well.
> Thanks,
> Greg
> p.s. if anyone has a good way to manage discussion that spans two
> independent mailing lists, I'm all ears... :-)
>
> ---------------------------------------------------------------------------------------------------
>
> Hi,
>
> I'm trying to figure out how to satisfy four sets of needs for
> instructional material using IPython Notebooks in Software Carpentry. I
> feel like I'm thrashing, so I'd appreciate your help.
>
> My four use cases are:
>
> * instructor's guide: to help instructors prepare for a bootcamp
> * lecture notes: what instructors use during a bootcamp
> * workbook: what learners are given to use during a bootcamp
> * textbook: what learners use on their own outside a bootcamp
>
> To make this more concrete, have a look at
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html, which
> describes how I teach the shell:
>
> *
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:instructors
> is a high-level intro for instructors.  Learners might find it useful,
> but it's not really aimed at them.
>
> * Each topic opens with objectives (see
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:objectives
> for an example).  This belongs in the instructor's guide ("Here's what
> you're supposed to teach"), possibly the lecture notes ("Hi class,
> here's what I'm about to teach"), probably not the workbook, and
> definitely the textbook.
>
> * Each topic ends with key points and challenge exercises: see
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:keypoints,
> and
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:challenges
> for examples.  Again, both belong in the instructor's guide and the
> textbook; they probably belong in the lecture notes, and probably also
> in the workbook (as a starting point for learners' own notes, and so
> that they know what to work on during practicals).
>
> Problem #1: how do we manage material that ought to appear in multiple
> documents?  Right now, each snippet is a separate HTML file that is
> %include'd to assemble the entire document (see
> https://raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-shell/tutorial.html).
> That works OK for HTML pages, but not for notebooks: as far as I know,
> there's no way to %include file X in notebook Y.
>
> Problem #2: what exactly ought to be in the lecture notes?  For example,
> compare
> http://swcarpentry.github.io/bc/lessons/guide-pyblocks/tutorial.html#s:pyblocks:logic:lesson
> (the introduction to "if/else" in the incomplete instructor's guide to
> basic Python programming using IPythonBlocks) to
> http://nbviewer.ipython.org/urls/raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-pyblocks/pyblocks-logic.ipynb
> (an IPython Notebook with the same handful of examples).  The former is
> much too verbose to put in front of learners in a classroom setting.  I
> feel the latter is too sparse, but when I add point-form notes and then
> lecture over the result, it feels clumsy:
>
> * If the item I'm currently talking about is centered on the screen,
> learners can see my next points on screen below it.  This is distracting
> --- so distracting that tools like PowerPoint and other slideshow tools
> were invented in part to prevent it happening.  If the notebook
> supported progressive reveal of pre-made cells, that problem would lessen.
>
> * Even with that, though, one of the most compelling reasons to use the
> notebook as a lecture tool is lost.  Learners really like us typing code
> in as we teach: it slows us down, which makes it more likely that we'll
> walk through the code at a comprehensible pace, and it makes it a lot
> easier for us to extemporize in response to their questions or
> confusion.  We lose both benefits if we start using "reveal next",
> though we're still better off than if we use slides.
>
> Problem #3: what ought to be in the workbook (the notebook we give
> students as a starting point)?  Option A is "nothing": they should type
> things in as the lecture proceeds.  This forces them to learn hands-on
> (which is good), but greatly increases the risk that they'll fall behind:
>
> * They're more likely to mis-type something than the instructor, so
> their actual lines-per-second is (much) lower.
> * They want to take notes, which slows them down even further.
>
> Option B is to give learners a notebook with notes and code already in
> it, but that gets us back to them (a) being distracted by material they
> haven't seen yet, but which is already on their screen, and (b) not
> having to be hands-on while learning.  On the other hand, these are
> grownups, and there's only so much we can do with existing tools.
>
> So here's what I think I've come to:
>
> 1. For now, we create two documents: an instructor's guide that doubles
> as a textbook (no harm in learners reading notes for instructors, and
> possibly some benefit), and a notebook that interleaves point-form notes
> with code examples that instructors will use for lecturing, and learners
> will be given as a starting point.
>
> 2. We build a tool to extract point-form notes from an IPython Notebook
> to create the "objectives.html" and "keypoints.html" sections that are
> %include'd in #1 above.  This will help keep the notebook and the guide
> in step; whoever changes the notebook will still have to read the
> section of the guide to make sure, but it's a step.
>
> 3. The code cells in our notebooks have markers for the instructors and
> the learners to fill in.  For example, instead of giving people:
>
>      for char in "GATTACA":
>          if char == "A":
>              print "found an A"
>
> we have:
>
>      for ________ in "GATTACA":
>          if ________:
>              print "found an A"
>
> Filling in the ________'s in the code snippets forces the instructor and
> the learners to do some typing. As a bonus, if chosen judiciously they
> will also encourage both to focus on the key features of the example.
>
> But there are problems.  (You knew there would be...)
>
> * (medium-sized) In order to extract stuff from notebooks for use
> elsewhere, we need a way to tag the cells that contain objectives, notes
> for the key point summary, etc., so that our tool can find and
> differentiate them.  I *think* this can be done via a plugin: the
> notebook has an extensible metadata field per cell, which we've already
> exploited for another tool, so adding something that allows "right click
> and select tag" should be straightforward. It would be nice if this info
> was available as a class on the HTML element containing the material, so
> that it could be styled via CSS, but that's a bonus.
>
> * (large) Literal ________'s in our code will make it invalid, so our
> notebooks either won't have any output (because we didn't execute the
> code) or will be littered with error messages (because we did).  The
> former's not bad, but if we do that, we need two version of each
> notebook: one with the blanks, and one with them filled in.  As always,
> keeping variants of a single document in sync will be a pain.
>
> If the code in cells was stored as HTML, we could put something like
> <span class="fillin">char == "A"</span> in the code, then use Javascript
> to display ________, erase it when the learner started typing, or fill
> it in with the right answer when the learner just wanted to catch up or
> see the answer. Problem is, the code *isn't* stored as HTML; it's stored
> as JSON that looks like this:
>
>      {
>       "cell_type": "code",
>       "collapsed": false,
>       "input": [
>        "import math\n",
>        "print \"pi is\", math.pi\n",
>        "print \"square root of 5 is\", math.sqrt(5)"
>       ],
>       "language": "python",
>       "metadata": {},
>       "outputs": [
>        {
>         "output_type": "stream",
>         "stream": "stdout",
>         "text": [
>          "pi is 3.14159265359\n",
>          "square root of 5 is 2.2360679775\n"
>         ]
>        }
>       ],
>       "prompt_number": 1
>      }
>
> We could use character offsets to identify subsections of the code, then
> intercept rendering to insert span elements, but that way lies madness...
>
> So as I said at the outset, I feel like I'm thrashing here, and would
> welcome suggestions on how to proceed.
>
> Thanks,
> Greg
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From gvwilson at third-bit.com  Mon Jul 29 14:53:02 2013
From: gvwilson at third-bit.com (Greg Wilson)
Date: Mon, 29 Jul 2013 14:53:02 -0400
Subject: [IPython-dev] prose, notebooks,
 and thrashing (and a re-direct to another list)
In-Reply-To: <CAH4pYpR192nBzEc6CLsB=obo0dYDEsPEnwAUwfKXZdD7Gr+J_w@mail.gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<CAH4pYpR192nBzEc6CLsB=obo0dYDEsPEnwAUwfKXZdD7Gr+J_w@mail.gmail.com>
Message-ID: <51F6BA0E.8020808@third-bit.com>

Thanks for the feedback, Brian. Tagging each cell with one or more of 
"instructor's guide", "workbook", etc., and then slicing 'em to produce 
different notebooks sounds like a good plan to me.  Can someone tell me 
(a) whether the UI elements for adding tags exist yet, (b) if not, 
whether they're in process, (c) if not, whether they can be done as a 
plugin?
Thanks,
Greg

On 2013-07-29 1:31 PM, Brian Granger wrote:
> Greg,
>
> Here is what I would do *today*.
>
> * I would author a single notebook that has all of the content.
> * I would use cell level metadata to add to the tags list.  At the dev
> meeting we decided to add a top level tags key that is a list of tags
> to the cell level metadata.
> * I would write a custom Transformer for nbconvert that filters
> notebook cells based on these tags.
> * I would run run nbconvert multiple times to produce the different notebooks.
>
> Sorry I am so brief - we are moving this week so I am mostly offline.
> Other people can help with the details of these things.
>
> Cheers,
>
> Brian
>
> On Mon, Jul 29, 2013 at 8:13 AM, Greg Wilson <gvwilson at third-bit.com> wrote:
>> Hi,
>> I posted the message below to the Software Carpentry discussion list a
>> few minutes ago.  I'd appreciate input from the IPython Notebook
>> community as well.
>> Thanks,
>> Greg
>> p.s. if anyone has a good way to manage discussion that spans two
>> independent mailing lists, I'm all ears... :-)
>>
>> ---------------------------------------------------------------------------------------------------
>>
>> Hi,
>>
>> I'm trying to figure out how to satisfy four sets of needs for
>> instructional material using IPython Notebooks in Software Carpentry. I
>> feel like I'm thrashing, so I'd appreciate your help.
>>
>> My four use cases are:
>>
>> * instructor's guide: to help instructors prepare for a bootcamp
>> * lecture notes: what instructors use during a bootcamp
>> * workbook: what learners are given to use during a bootcamp
>> * textbook: what learners use on their own outside a bootcamp
>>
>> To make this more concrete, have a look at
>> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html, which
>> describes how I teach the shell:
>>
>> *
>> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:instructors
>> is a high-level intro for instructors.  Learners might find it useful,
>> but it's not really aimed at them.
>>
>> * Each topic opens with objectives (see
>> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:objectives
>> for an example).  This belongs in the instructor's guide ("Here's what
>> you're supposed to teach"), possibly the lecture notes ("Hi class,
>> here's what I'm about to teach"), probably not the workbook, and
>> definitely the textbook.
>>
>> * Each topic ends with key points and challenge exercises: see
>> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:keypoints,
>> and
>> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:challenges
>> for examples.  Again, both belong in the instructor's guide and the
>> textbook; they probably belong in the lecture notes, and probably also
>> in the workbook (as a starting point for learners' own notes, and so
>> that they know what to work on during practicals).
>>
>> Problem #1: how do we manage material that ought to appear in multiple
>> documents?  Right now, each snippet is a separate HTML file that is
>> %include'd to assemble the entire document (see
>> https://raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-shell/tutorial.html).
>> That works OK for HTML pages, but not for notebooks: as far as I know,
>> there's no way to %include file X in notebook Y.
>>
>> Problem #2: what exactly ought to be in the lecture notes?  For example,
>> compare
>> http://swcarpentry.github.io/bc/lessons/guide-pyblocks/tutorial.html#s:pyblocks:logic:lesson
>> (the introduction to "if/else" in the incomplete instructor's guide to
>> basic Python programming using IPythonBlocks) to
>> http://nbviewer.ipython.org/urls/raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-pyblocks/pyblocks-logic.ipynb
>> (an IPython Notebook with the same handful of examples).  The former is
>> much too verbose to put in front of learners in a classroom setting.  I
>> feel the latter is too sparse, but when I add point-form notes and then
>> lecture over the result, it feels clumsy:
>>
>> * If the item I'm currently talking about is centered on the screen,
>> learners can see my next points on screen below it.  This is distracting
>> --- so distracting that tools like PowerPoint and other slideshow tools
>> were invented in part to prevent it happening.  If the notebook
>> supported progressive reveal of pre-made cells, that problem would lessen.
>>
>> * Even with that, though, one of the most compelling reasons to use the
>> notebook as a lecture tool is lost.  Learners really like us typing code
>> in as we teach: it slows us down, which makes it more likely that we'll
>> walk through the code at a comprehensible pace, and it makes it a lot
>> easier for us to extemporize in response to their questions or
>> confusion.  We lose both benefits if we start using "reveal next",
>> though we're still better off than if we use slides.
>>
>> Problem #3: what ought to be in the workbook (the notebook we give
>> students as a starting point)?  Option A is "nothing": they should type
>> things in as the lecture proceeds.  This forces them to learn hands-on
>> (which is good), but greatly increases the risk that they'll fall behind:
>>
>> * They're more likely to mis-type something than the instructor, so
>> their actual lines-per-second is (much) lower.
>> * They want to take notes, which slows them down even further.
>>
>> Option B is to give learners a notebook with notes and code already in
>> it, but that gets us back to them (a) being distracted by material they
>> haven't seen yet, but which is already on their screen, and (b) not
>> having to be hands-on while learning.  On the other hand, these are
>> grownups, and there's only so much we can do with existing tools.
>>
>> So here's what I think I've come to:
>>
>> 1. For now, we create two documents: an instructor's guide that doubles
>> as a textbook (no harm in learners reading notes for instructors, and
>> possibly some benefit), and a notebook that interleaves point-form notes
>> with code examples that instructors will use for lecturing, and learners
>> will be given as a starting point.
>>
>> 2. We build a tool to extract point-form notes from an IPython Notebook
>> to create the "objectives.html" and "keypoints.html" sections that are
>> %include'd in #1 above.  This will help keep the notebook and the guide
>> in step; whoever changes the notebook will still have to read the
>> section of the guide to make sure, but it's a step.
>>
>> 3. The code cells in our notebooks have markers for the instructors and
>> the learners to fill in.  For example, instead of giving people:
>>
>>       for char in "GATTACA":
>>           if char == "A":
>>               print "found an A"
>>
>> we have:
>>
>>       for ________ in "GATTACA":
>>           if ________:
>>               print "found an A"
>>
>> Filling in the ________'s in the code snippets forces the instructor and
>> the learners to do some typing. As a bonus, if chosen judiciously they
>> will also encourage both to focus on the key features of the example.
>>
>> But there are problems.  (You knew there would be...)
>>
>> * (medium-sized) In order to extract stuff from notebooks for use
>> elsewhere, we need a way to tag the cells that contain objectives, notes
>> for the key point summary, etc., so that our tool can find and
>> differentiate them.  I *think* this can be done via a plugin: the
>> notebook has an extensible metadata field per cell, which we've already
>> exploited for another tool, so adding something that allows "right click
>> and select tag" should be straightforward. It would be nice if this info
>> was available as a class on the HTML element containing the material, so
>> that it could be styled via CSS, but that's a bonus.
>>
>> * (large) Literal ________'s in our code will make it invalid, so our
>> notebooks either won't have any output (because we didn't execute the
>> code) or will be littered with error messages (because we did).  The
>> former's not bad, but if we do that, we need two version of each
>> notebook: one with the blanks, and one with them filled in.  As always,
>> keeping variants of a single document in sync will be a pain.
>>
>> If the code in cells was stored as HTML, we could put something like
>> <span class="fillin">char == "A"</span> in the code, then use Javascript
>> to display ________, erase it when the learner started typing, or fill
>> it in with the right answer when the learner just wanted to catch up or
>> see the answer. Problem is, the code *isn't* stored as HTML; it's stored
>> as JSON that looks like this:
>>
>>       {
>>        "cell_type": "code",
>>        "collapsed": false,
>>        "input": [
>>         "import math\n",
>>         "print \"pi is\", math.pi\n",
>>         "print \"square root of 5 is\", math.sqrt(5)"
>>        ],
>>        "language": "python",
>>        "metadata": {},
>>        "outputs": [
>>         {
>>          "output_type": "stream",
>>          "stream": "stdout",
>>          "text": [
>>           "pi is 3.14159265359\n",
>>           "square root of 5 is 2.2360679775\n"
>>          ]
>>         }
>>        ],
>>        "prompt_number": 1
>>       }
>>
>> We could use character offsets to identify subsections of the code, then
>> intercept rendering to insert span elements, but that way lies madness...
>>
>> So as I said at the outset, I feel like I'm thrashing here, and would
>> welcome suggestions on how to proceed.
>>
>> Thanks,
>> Greg
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



From gvwilson at third-bit.com  Mon Jul 29 15:00:03 2013
From: gvwilson at third-bit.com (Greg Wilson)
Date: Mon, 29 Jul 2013 15:00:03 -0400
Subject: [IPython-dev] prose, notebooks,
 and thrashing (and a re-direct to another list)
In-Reply-To: <D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
Message-ID: <51F6BBB3.4070509@third-bit.com>

Thanks for your feedback, Matthias; replies inline.
>> Problem #1: how do we manage material that ought to appear in multiple
>> documents?
> 2.0 and probably will need dexy. (http://www.dexy.it/)
Brian Granger has suggested going the other way: build one notebook with 
everything in it, tag each cell with the sub-documents it ought to 
appear in, and write a transformer to create those sub-documents.  Seems 
like that'll work right now (i.e., once the tagging UI plugin is built, 
the transformer is written, etc.) --- does that seem like a good way 
forward?
> ... Have you seen slideshow mode ?
Yup; it solves the "progressive reveal" problem, but doesn't force 
instructors and learners to do a bit of typing as they go along (which 
is very beneficial pedagogically).
>> * (medium-sized) In order to extract stuff from notebooks for use
>> elsewhere, we need a way to tag the cells that contain objectives, notes
>> for the key point summary, etc., so that our tool can find and
>> differentiate them.
> Yes, that's on my todo list. This would require SC to ship notebook with custom profiles.
> There is just some edge cases at loading time.
Is this the same as the tagging Brian Granger mentioned?
>> * (large) Literal ________'s in our code will make it invalid...
> You could store the "answers" into the metadata of each cell that maps the Nth ____
> to it's value. We can come up with something.
Rather than messing with character offsets and the like, can we use a 
plugin to prevent execution of any cell that contains a marker like 
________ ?  In most cases, cells will only have two or three of these, 
so it's not unreasonable to ask people to delete 'em and replace 'em 
manually, rather than trying to do something clever with the UI.

Thanks again,
Greg


From bussonniermatthias at gmail.com  Mon Jul 29 15:02:10 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 29 Jul 2013 21:02:10 +0200
Subject: [IPython-dev] [Discuss]  prose, notebooks,
	and thrashing (and a re-direct to another list)
In-Reply-To: <51F6BA0E.8020808@third-bit.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<CAH4pYpR192nBzEc6CLsB=obo0dYDEsPEnwAUwfKXZdD7Gr+J_w@mail.gmail.com>
	<51F6BA0E.8020808@third-bit.com>
Message-ID: <8276231C-AB9A-4748-8660-706CE44ECBBB@gmail.com>


Le 29 juil. 2013 ? 20:53, Greg Wilson a ?crit :

> Thanks for the feedback, Brian. Tagging each cell with one or more of "instructor's guide", "workbook", etc., and then slicing 'em to produce different notebooks sounds like a good plan to me.

Apparently my mail might have not reached you :-)

>  Can someone tell me

> (a) whether the UI elements for adding tags exist yet,

No

> (b) if not, whether they're in process,

Yes, I am assigned.

> (c) if not, whether they can be done as a plugin?

Yes, the hard part is the UI for tags.

Have a look at 
IPython/html/static/notebook/js/celltoolbarpresets/example.js

You should have example for checkbox-boolean field and simple-select.
-- 
Matthias

From gmbecker at ucdavis.edu  Mon Jul 29 15:07:22 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Mon, 29 Jul 2013 12:07:22 -0700
Subject: [IPython-dev] prose, notebooks,
 and thrashing (and a re-direct to another list)
In-Reply-To: <51F6BA0E.8020808@third-bit.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<CAH4pYpR192nBzEc6CLsB=obo0dYDEsPEnwAUwfKXZdD7Gr+J_w@mail.gmail.com>
	<51F6BA0E.8020808@third-bit.com>
Message-ID: <CADwqtCMaBr4rh086wCeBmRFQF6tDd-ndnFOPpAJoC3vP+0gRog@mail.gmail.com>

Hey guys,

I'm not looking to get into another huge debate at this juncture, but
nonsequential notebooks with branching are a natural fit for different
treatments or views of the same or related content, of which this is a
special case. (greg and Software carpentry see
here<http://python.6.x6.nabble.com/Some-new-cell-types-for-describing-data-analyses-in-IPy-Notebook-td5023238.html>for
the very long discussion we had on ipython dev about this previously).

In that case, you would still slice the student notebook out, but the
"tagging" would be essentially automatic and the instructor would have a
natural way of having access to all the content in an intelligible and
executable (incl. headless/full notebook execution) format within a single
notebook .

~G

P.S. Hope the move is going well Brian.


On Mon, Jul 29, 2013 at 11:53 AM, Greg Wilson <gvwilson at third-bit.com>wrote:

> Thanks for the feedback, Brian. Tagging each cell with one or more of
> "instructor's guide", "workbook", etc., and then slicing 'em to produce
> different notebooks sounds like a good plan to me.  Can someone tell me
> (a) whether the UI elements for adding tags exist yet, (b) if not,
> whether they're in process, (c) if not, whether they can be done as a
> plugin?
> Thanks,
> Greg
>
> On 2013-07-29 1:31 PM, Brian Granger wrote:
> > Greg,
> >
> > Here is what I would do *today*.
> >
> > * I would author a single notebook that has all of the content.
> > * I would use cell level metadata to add to the tags list.  At the dev
> > meeting we decided to add a top level tags key that is a list of tags
> > to the cell level metadata.
> > * I would write a custom Transformer for nbconvert that filters
> > notebook cells based on these tags.
> > * I would run run nbconvert multiple times to produce the different
> notebooks.
> >
> > Sorry I am so brief - we are moving this week so I am mostly offline.
> > Other people can help with the details of these things.
> >
> > Cheers,
> >
> > Brian
> >
> > On Mon, Jul 29, 2013 at 8:13 AM, Greg Wilson <gvwilson at third-bit.com>
> wrote:
> >> Hi,
> >> I posted the message below to the Software Carpentry discussion list a
> >> few minutes ago.  I'd appreciate input from the IPython Notebook
> >> community as well.
> >> Thanks,
> >> Greg
> >> p.s. if anyone has a good way to manage discussion that spans two
> >> independent mailing lists, I'm all ears... :-)
> >>
> >>
> ---------------------------------------------------------------------------------------------------
> >>
> >> Hi,
> >>
> >> I'm trying to figure out how to satisfy four sets of needs for
> >> instructional material using IPython Notebooks in Software Carpentry. I
> >> feel like I'm thrashing, so I'd appreciate your help.
> >>
> >> My four use cases are:
> >>
> >> * instructor's guide: to help instructors prepare for a bootcamp
> >> * lecture notes: what instructors use during a bootcamp
> >> * workbook: what learners are given to use during a bootcamp
> >> * textbook: what learners use on their own outside a bootcamp
> >>
> >> To make this more concrete, have a look at
> >> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html,
> which
> >> describes how I teach the shell:
> >>
> >> *
> >>
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:instructors
> >> is a high-level intro for instructors.  Learners might find it useful,
> >> but it's not really aimed at them.
> >>
> >> * Each topic opens with objectives (see
> >>
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:objectives
> >> for an example).  This belongs in the instructor's guide ("Here's what
> >> you're supposed to teach"), possibly the lecture notes ("Hi class,
> >> here's what I'm about to teach"), probably not the workbook, and
> >> definitely the textbook.
> >>
> >> * Each topic ends with key points and challenge exercises: see
> >>
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:keypoints
> ,
> >> and
> >>
> http://swcarpentry.github.io/bc/lessons/guide-shell/tutorial.html#s:shell:filedir:challenges
> >> for examples.  Again, both belong in the instructor's guide and the
> >> textbook; they probably belong in the lecture notes, and probably also
> >> in the workbook (as a starting point for learners' own notes, and so
> >> that they know what to work on during practicals).
> >>
> >> Problem #1: how do we manage material that ought to appear in multiple
> >> documents?  Right now, each snippet is a separate HTML file that is
> >> %include'd to assemble the entire document (see
> >>
> https://raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-shell/tutorial.html
> ).
> >> That works OK for HTML pages, but not for notebooks: as far as I know,
> >> there's no way to %include file X in notebook Y.
> >>
> >> Problem #2: what exactly ought to be in the lecture notes?  For example,
> >> compare
> >>
> http://swcarpentry.github.io/bc/lessons/guide-pyblocks/tutorial.html#s:pyblocks:logic:lesson
> >> (the introduction to "if/else" in the incomplete instructor's guide to
> >> basic Python programming using IPythonBlocks) to
> >>
> http://nbviewer.ipython.org/urls/raw.github.com/swcarpentry/bc/gh-pages/lessons/guide-pyblocks/pyblocks-logic.ipynb
> >> (an IPython Notebook with the same handful of examples).  The former is
> >> much too verbose to put in front of learners in a classroom setting.  I
> >> feel the latter is too sparse, but when I add point-form notes and then
> >> lecture over the result, it feels clumsy:
> >>
> >> * If the item I'm currently talking about is centered on the screen,
> >> learners can see my next points on screen below it.  This is distracting
> >> --- so distracting that tools like PowerPoint and other slideshow tools
> >> were invented in part to prevent it happening.  If the notebook
> >> supported progressive reveal of pre-made cells, that problem would
> lessen.
> >>
> >> * Even with that, though, one of the most compelling reasons to use the
> >> notebook as a lecture tool is lost.  Learners really like us typing code
> >> in as we teach: it slows us down, which makes it more likely that we'll
> >> walk through the code at a comprehensible pace, and it makes it a lot
> >> easier for us to extemporize in response to their questions or
> >> confusion.  We lose both benefits if we start using "reveal next",
> >> though we're still better off than if we use slides.
> >>
> >> Problem #3: what ought to be in the workbook (the notebook we give
> >> students as a starting point)?  Option A is "nothing": they should type
> >> things in as the lecture proceeds.  This forces them to learn hands-on
> >> (which is good), but greatly increases the risk that they'll fall
> behind:
> >>
> >> * They're more likely to mis-type something than the instructor, so
> >> their actual lines-per-second is (much) lower.
> >> * They want to take notes, which slows them down even further.
> >>
> >> Option B is to give learners a notebook with notes and code already in
> >> it, but that gets us back to them (a) being distracted by material they
> >> haven't seen yet, but which is already on their screen, and (b) not
> >> having to be hands-on while learning.  On the other hand, these are
> >> grownups, and there's only so much we can do with existing tools.
> >>
> >> So here's what I think I've come to:
> >>
> >> 1. For now, we create two documents: an instructor's guide that doubles
> >> as a textbook (no harm in learners reading notes for instructors, and
> >> possibly some benefit), and a notebook that interleaves point-form notes
> >> with code examples that instructors will use for lecturing, and learners
> >> will be given as a starting point.
> >>
> >> 2. We build a tool to extract point-form notes from an IPython Notebook
> >> to create the "objectives.html" and "keypoints.html" sections that are
> >> %include'd in #1 above.  This will help keep the notebook and the guide
> >> in step; whoever changes the notebook will still have to read the
> >> section of the guide to make sure, but it's a step.
> >>
> >> 3. The code cells in our notebooks have markers for the instructors and
> >> the learners to fill in.  For example, instead of giving people:
> >>
> >>       for char in "GATTACA":
> >>           if char == "A":
> >>               print "found an A"
> >>
> >> we have:
> >>
> >>       for ________ in "GATTACA":
> >>           if ________:
> >>               print "found an A"
> >>
> >> Filling in the ________'s in the code snippets forces the instructor and
> >> the learners to do some typing. As a bonus, if chosen judiciously they
> >> will also encourage both to focus on the key features of the example.
> >>
> >> But there are problems.  (You knew there would be...)
> >>
> >> * (medium-sized) In order to extract stuff from notebooks for use
> >> elsewhere, we need a way to tag the cells that contain objectives, notes
> >> for the key point summary, etc., so that our tool can find and
> >> differentiate them.  I *think* this can be done via a plugin: the
> >> notebook has an extensible metadata field per cell, which we've already
> >> exploited for another tool, so adding something that allows "right click
> >> and select tag" should be straightforward. It would be nice if this info
> >> was available as a class on the HTML element containing the material, so
> >> that it could be styled via CSS, but that's a bonus.
> >>
> >> * (large) Literal ________'s in our code will make it invalid, so our
> >> notebooks either won't have any output (because we didn't execute the
> >> code) or will be littered with error messages (because we did).  The
> >> former's not bad, but if we do that, we need two version of each
> >> notebook: one with the blanks, and one with them filled in.  As always,
> >> keeping variants of a single document in sync will be a pain.
> >>
> >> If the code in cells was stored as HTML, we could put something like
> >> <span class="fillin">char == "A"</span> in the code, then use Javascript
> >> to display ________, erase it when the learner started typing, or fill
> >> it in with the right answer when the learner just wanted to catch up or
> >> see the answer. Problem is, the code *isn't* stored as HTML; it's stored
> >> as JSON that looks like this:
> >>
> >>       {
> >>        "cell_type": "code",
> >>        "collapsed": false,
> >>        "input": [
> >>         "import math\n",
> >>         "print \"pi is\", math.pi\n",
> >>         "print \"square root of 5 is\", math.sqrt(5)"
> >>        ],
> >>        "language": "python",
> >>        "metadata": {},
> >>        "outputs": [
> >>         {
> >>          "output_type": "stream",
> >>          "stream": "stdout",
> >>          "text": [
> >>           "pi is 3.14159265359\n",
> >>           "square root of 5 is 2.2360679775\n"
> >>          ]
> >>         }
> >>        ],
> >>        "prompt_number": 1
> >>       }
> >>
> >> We could use character offsets to identify subsections of the code, then
> >> intercept rendering to insert span elements, but that way lies
> madness...
> >>
> >> So as I said at the outset, I feel like I'm thrashing here, and would
> >> welcome suggestions on how to proceed.
> >>
> >> Thanks,
> >> Greg
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130729/e1e09d44/attachment.html>

From bussonniermatthias at gmail.com  Mon Jul 29 15:36:37 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 29 Jul 2013 21:36:37 +0200
Subject: [IPython-dev] prose, notebooks,
	and thrashing (and a re-direct to another list)
In-Reply-To: <51F6BBB3.4070509@third-bit.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
	<51F6BBB3.4070509@third-bit.com>
Message-ID: <44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>


Le 29 juil. 2013 ? 21:00, Greg Wilson a ?crit :

> Thanks for your feedback, Matthias; replies inline.
>>> Problem #1: how do we manage material that ought to appear in multiple
>>> documents?
>> 2.0 and probably will need dexy. (http://www.dexy.it/)
> Brian Granger has suggested going the other way: build one notebook with 
> everything in it, tag each cell with the sub-documents it ought to 
> appear in, and write a transformer to create those sub-documents.  Seems 
> like that'll work right now (i.e., once the tagging UI plugin is built, 
> the transformer is written, etc.) --- does that seem like a good way 
> forward?

I was thinking of material other than notebook that ought to be in many IPynb.
Skimmed too fast.
I concurred with brian for the "One ipynb to rule them all" and produce multiple documents.


>> ... Have you seen slideshow mode ?
> Yup; it solves the "progressive reveal" problem, but doesn't force 
> instructors and learners to do a bit of typing as they go along (which 
> is very beneficial pedagogically).


>> Yes, that's on my todo list. This would require SC to ship notebook with custom profiles.
>> There is just some edge cases at loading time.
> Is this the same as the tagging Brian Granger mentioned?

Yes.

>>> * (large) Literal ________'s in our code will make it invalid...
>> You could store the "answers" into the metadata of each cell that maps the Nth ____
>> to it's value. We can come up with something.


> Rather than messing with character offsets and the like, can we use a 
> plugin to prevent execution of any cell that contains a marker like 
> ________ ?  In most cases, cells will only have two or three of these, 
> so it's not unreasonable to ask people to delete 'em and replace 'em 
> manually, rather than trying to do something clever with the UI.

Yes it is do-able (not sure if it is supported), I was more concerned of storing the "answer"
into the metadata, ability to get back to *initial* value of a cell.

Something along

CodeCell.prototype.old_execute = CodeCell.prototype.execute

CodeCell.prototype.execute = function () {
        if ( ! this.get_text().match('_{4,}') ){
            this.old_execute(arguments); // not sure of the unpacking of arguments here, might be [arguments]
        } else {
           // hire a GSoC student to write the code that will highlight the ____ in red.
        }
};


wrap in a IIFE, listen for the correct even in custom.js and load with require()
-- 
Matthias



From michaelmalak at yahoo.com  Mon Jul 29 19:37:57 2013
From: michaelmalak at yahoo.com (Michael Malak)
Date: Mon, 29 Jul 2013 16:37:57 -0700 (PDT)
Subject: [IPython-dev] Turning off output scroll window
In-Reply-To: <44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>	<51F6868D.1050903@third-bit.com>	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>	<51F6BBB3.4070509@third-bit.com>
	<44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
Message-ID: <1375141077.37702.YahooMailNeo@web160801.mail.bf1.yahoo.com>

When output is very long, iPython Notebook conveniently creates an inner scroll window to contain the output. In a few cases, though, it would be desirable to see all the output, e.g. when printing to PDF. Is there a way to turn off that behavior and see all the output inline?



From benjaminrk at gmail.com  Mon Jul 29 19:45:28 2013
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 29 Jul 2013 16:45:28 -0700
Subject: [IPython-dev] Turning off output scroll window
In-Reply-To: <1375141077.37702.YahooMailNeo@web160801.mail.bf1.yahoo.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
	<51F6BBB3.4070509@third-bit.com>
	<44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
	<1375141077.37702.YahooMailNeo@web160801.mail.bf1.yahoo.com>
Message-ID: <CAHNn8BW-p1vxT9OvuXN3VxMcVWjqJeVqROukLqT2WYm2VAGGDw@mail.gmail.com>

If you output to PDF via nbconvert, the interactive scroll behavior is not
involved.


On Mon, Jul 29, 2013 at 4:37 PM, Michael Malak <michaelmalak at yahoo.com>wrote:

> When output is very long, iPython Notebook conveniently creates an inner
> scroll window to contain the output. In a few cases, though, it would be
> desirable to see all the output, e.g. when printing to PDF. Is there a way
> to turn off that behavior and see all the output inline?
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130729/29c16558/attachment.html>

From anfedorov at gmail.com  Mon Jul 29 20:20:14 2013
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Mon, 29 Jul 2013 17:20:14 -0700
Subject: [IPython-dev] Displaying custom widgets in iPython notebook
Message-ID: <CAESh9jU5e2vTA-by_pwn4rMno_J2PHCrBbetQM2PopswNtnT=w@mail.gmail.com>

I'm dealing with a bit of an interesting use-case, where non-programmers
are using the iPython notebook to run queries on our production database
and interpreting the results.

In this use case, but also in others, I think it would make sense to create
a method similar to __str__, but returning a widget made of HTML/CSS/JS
meant to be displayed in a browser, not just text-only output.

Does it seems appropriate for Python objects to be able to display
themselves natively in a browser, perhaps self-contained in an iframe?

Cheers,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130729/4c13b403/attachment.html>

From scopatz at gmail.com  Mon Jul 29 20:21:05 2013
From: scopatz at gmail.com (Anthony Scopatz)
Date: Mon, 29 Jul 2013 19:21:05 -0500
Subject: [IPython-dev] Turning off output scroll window
In-Reply-To: <CAHNn8BW-p1vxT9OvuXN3VxMcVWjqJeVqROukLqT2WYm2VAGGDw@mail.gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
	<51F6BBB3.4070509@third-bit.com>
	<44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
	<1375141077.37702.YahooMailNeo@web160801.mail.bf1.yahoo.com>
	<CAHNn8BW-p1vxT9OvuXN3VxMcVWjqJeVqROukLqT2WYm2VAGGDw@mail.gmail.com>
Message-ID: <CAPk-6T7DOd1rtHtYtSeRJP3CvAtnkXxm2nuTw=NyOxc0n9MRvw@mail.gmail.com>

For the interactive notebook though, is there a way to do this?

Be Well
Anthony


On Mon, Jul 29, 2013 at 6:45 PM, MinRK <benjaminrk at gmail.com> wrote:

> If you output to PDF via nbconvert, the interactive scroll behavior is not
> involved.
>
>
> On Mon, Jul 29, 2013 at 4:37 PM, Michael Malak <michaelmalak at yahoo.com>wrote:
>
>> When output is very long, iPython Notebook conveniently creates an inner
>> scroll window to contain the output. In a few cases, though, it would be
>> desirable to see all the output, e.g. when printing to PDF. Is there a way
>> to turn off that behavior and see all the output inline?
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130729/7ec29a5a/attachment.html>

From bussonniermatthias at gmail.com  Mon Jul 29 20:27:14 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 30 Jul 2013 02:27:14 +0200
Subject: [IPython-dev] Turning off output scroll window
In-Reply-To: <CAPk-6T7DOd1rtHtYtSeRJP3CvAtnkXxm2nuTw=NyOxc0n9MRvw@mail.gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
	<51F6BBB3.4070509@third-bit.com>
	<44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
	<1375141077.37702.YahooMailNeo@web160801.mail.bf1.yahoo.com>
	<CAHNn8BW-p1vxT9OvuXN3VxMcVWjqJeVqROukLqT2WYm2VAGGDw@mail.gmail.com>
	<CAPk-6T7DOd1rtHtYtSeRJP3CvAtnkXxm2nuTw=NyOxc0n9MRvw@mail.gmail.com>
Message-ID: <DBB4181D-B957-4620-9627-E208F89BAD3B@gmail.com>


Le 30 juil. 2013 ? 02:21, Anthony Scopatz a ?crit :

> For the interactive notebook though, is there a way to do this?

Click on side of cell, or 
Cell > all outputs > expand.
-- 
M



From bussonniermatthias at gmail.com  Mon Jul 29 20:29:46 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 30 Jul 2013 02:29:46 +0200
Subject: [IPython-dev] Displaying custom widgets in iPython notebook
In-Reply-To: <CAESh9jU5e2vTA-by_pwn4rMno_J2PHCrBbetQM2PopswNtnT=w@mail.gmail.com>
References: <CAESh9jU5e2vTA-by_pwn4rMno_J2PHCrBbetQM2PopswNtnT=w@mail.gmail.com>
Message-ID: <766E3E0C-F970-43B4-83BC-C6A7A09F8F1B@gmail.com>


Le 30 juil. 2013 ? 02:20, Andrey Fedorov a ?crit :

> I'm dealing with a bit of an interesting use-case, where non-programmers are using the iPython notebook to run queries on our production database and interpreting the results.
> 
> In this use case, but also in others, I think it would make sense to create a method similar to __str__, but returning a widget made of HTML/CSS/JS meant to be displayed in a browser, not just text-only output.
> 
> Does it seems appropriate for Python objects to be able to display themselves natively in a browser, perhaps self-contained in an iframe?


This is the point of IPython 2.0 draft to do this in December.
-- 
M

From bussonniermatthias at gmail.com  Tue Jul 30 02:12:35 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 30 Jul 2013 08:12:35 +0200
Subject: [IPython-dev] prose, notebooks,
	and thrashing (and a re-direct to another list)
In-Reply-To: <44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
References: <CAESh9jUp-e8Jk5LB6vibUvBUPisMnZhP+4LfLg1MhPwT1gMX0Q@mail.gmail.com>
	<51F6868D.1050903@third-bit.com>
	<D39DDF8C-2427-4D2E-BD1E-17E9F769931C@gmail.com>
	<51F6BBB3.4070509@third-bit.com>
	<44DCED6E-8049-415A-A720-3BD288E5D3B6@gmail.com>
Message-ID: <D237D21C-336E-4EB8-B3DC-C04978E1BD0E@gmail.com>


>> Rather than messing with character offsets and the like, can we use a 
>> plugin to prevent execution of any cell that contains a marker like 
>> ________ ?  In most cases, cells will only have two or three of these, 
>> so it's not unreasonable to ask people to delete 'em and replace 'em 
>> manually, rather than trying to do something clever with the UI.
> 
> Yes it is do-able (not sure if it is supported), I was more concerned of storing the "answer"
> into the metadata, ability to get back to *initial* value of a cell.
> 
> Something along
> 
> <something almost correct>

Please have a look at 

https://github.com/ipython-contrib/IPython-notebook-extensions/pull/11

it implements what you asked in no_exec_dunder.js

Put it in `.ipython/profile_default/static/custom/`

then add the following to your custom.js

$([IPython.events]).on('app_initialized.NotebookApp', function(){
      require(['custom/no_exec_dunder']);
});

-- 
Matthias

From wagnerfl at student.ethz.ch  Tue Jul 30 06:07:55 2013
From: wagnerfl at student.ethz.ch (Florian M. Wagner)
Date: Tue, 30 Jul 2013 12:07:55 +0200
Subject: [IPython-dev] Problems with ipcluster and LSF protocol (v.0.13.2)
Message-ID: <51F7907B.7060607@student.ethz.ch>

Dear IPython Users,

I am trying to start engines on a big cluster, which uses the Load 
Sharing Facility protocol. The engines seem to start correctly:

    *[wagnerfl at brutus1 ~]$ ipcluster start --profile=cluster
    --engines=LSF --n=4 --IPClusterApp.log_level=DEBUG*
    2013-07-30 11:55:31,893.893 [IPClusterStart] Using existing profile
    dir: u'/cluster/home02/erdw/wagnerfl/.ipython/profile_cluster'
    2013-07-30 11:55:31.914 [IPClusterStart] Starting ipcluster with
    [daemon=False]
    2013-07-30 11:55:31.915 [IPClusterStart] Creating pid file:
    /cluster/home02/erdw/wagnerfl/.ipython/profile_cluster/pid/ipcluster.pid
    2013-07-30 11:55:31.916 [IPClusterStart] Starting Controller with
    LocalControllerLauncher
    2013-07-30 11:55:32.915 [IPClusterStart] Starting 4 Engines with
    LSFJob array.
    2013-07-30 11:55:33.199 [IPClusterStart] Job submitted with job id:
    '27633338'
    2013-07-30 11:56:03.200 [IPClusterStart] Engines appear to have
    started successfully


However, I cannot access them from the Client.

    In [1]: from IPython.parallel import Client
    In [2]: c = Client(profile='cluster')
    In [3]: c.ids
    Out[3]: []

Could you give me a hint how to debug this, or is it not worth trying 
and should I better wait for 1.0?

Thanks a lot in advance
Florian





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130730/2014260b/attachment.html>

From wagnerfl at student.ethz.ch  Tue Jul 30 09:40:23 2013
From: wagnerfl at student.ethz.ch (Florian M. Wagner)
Date: Tue, 30 Jul 2013 15:40:23 +0200
Subject: [IPython-dev] Problems with ipcluster and LSF protocol
 (v.0.13.2) - SOLVED
In-Reply-To: <51F7907B.7060607@student.ethz.ch>
References: <51F7907B.7060607@student.ethz.ch>
Message-ID: <51F7C247.9090403@student.ethz.ch>

Nevermind,

Engines are found by the Client now. If everyone is experiencing a 
similar problem with LSF, I would recommend the excellent How-To by 
Plamen G. Krastev for a proper configuration:

http://rc.fas.harvard.edu/kb/high-performance-computing/configuring-an-ipython-cluster-2/

Best regards
Florian

Am 30.07.2013 12:07, schrieb Florian M. Wagner:
> Dear IPython Users,
>
> I am trying to start engines on a big cluster, which uses the Load 
> Sharing Facility protocol. The engines seem to start correctly:
>
>     *[wagnerfl at brutus1 ~]$ ipcluster start --profile=cluster
>     --engines=LSF --n=4 --IPClusterApp.log_level=DEBUG*
>     2013-07-30 11:55:31,893.893 [IPClusterStart] Using existing
>     profile dir: u'/cluster/home02/erdw/wagnerfl/.ipython/profile_cluster'
>     2013-07-30 11:55:31.914 [IPClusterStart] Starting ipcluster with
>     [daemon=False]
>     2013-07-30 11:55:31.915 [IPClusterStart] Creating pid file:
>     /cluster/home02/erdw/wagnerfl/.ipython/profile_cluster/pid/ipcluster.pid
>     2013-07-30 11:55:31.916 [IPClusterStart] Starting Controller with
>     LocalControllerLauncher
>     2013-07-30 11:55:32.915 [IPClusterStart] Starting 4 Engines with
>     LSFJob array.
>     2013-07-30 11:55:33.199 [IPClusterStart] Job submitted with job
>     id: '27633338'
>     2013-07-30 11:56:03.200 [IPClusterStart] Engines appear to have
>     started successfully
>
>
> However, I cannot access them from the Client.
>
>     In [1]: from IPython.parallel import Client
>     In [2]: c = Client(profile='cluster')
>     In [3]: c.ids
>     Out[3]: []
>
> Could you give me a hint how to debug this, or is it not worth trying 
> and should I better wait for 1.0?
>
> Thanks a lot in advance
> Florian
>
>
>
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130730/504b681a/attachment.html>

From jason-sage at creativetrax.com  Tue Jul 30 12:11:59 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 30 Jul 2013 09:11:59 -0700
Subject: [IPython-dev] interactive widgets
Message-ID: <51F7E5CF.7010202@creativetrax.com>

Could someone summarize some of the discussion that happened last week 
on interactive widgets (like manipulate/interact controls, etc.)?  I'm 
working more on extending Sage's implementation this week, and I'm 
curious what happened last week in the IPython camp.

Thanks,

Jason


From bussonniermatthias at gmail.com  Tue Jul 30 13:36:53 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 30 Jul 2013 19:36:53 +0200
Subject: [IPython-dev] interactive widgets
In-Reply-To: <51F7E5CF.7010202@creativetrax.com>
References: <51F7E5CF.7010202@creativetrax.com>
Message-ID: <D3730B0C-01D6-4453-9E56-DBAC2E30CF41@gmail.com>

Hi Jason.

You can have allow here 
https://hackpad.com/IPython-Interactive-JavaScript-Widget-Design-jRD6dHFt5I8

We have only define the rough edges of what we will do. 

	- create widget step (from kernel to javascript IIRC)
        - bidirectionnal update_state   js<-> kernel
        - possibly a destroy widget message

Data in json form. No binary websocket for now.
-- 
Matthias



Le 30 juil. 2013 ? 18:11, Jason Grout a ?crit :

> Could someone summarize some of the discussion that happened last week 
> on interactive widgets (like manipulate/interact controls, etc.)?  I'm 
> working more on extending Sage's implementation this week, and I'm 
> curious what happened last week in the IPython camp.
> 
> Thanks,
> 
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From benjaminrk at gmail.com  Tue Jul 30 14:05:45 2013
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 30 Jul 2013 11:05:45 -0700
Subject: [IPython-dev] Problems with ipcluster and LSF protocol
	(v.0.13.2)
In-Reply-To: <51F7907B.7060607@student.ethz.ch>
References: <51F7907B.7060607@student.ethz.ch>
Message-ID: <CAHNn8BUe=QMdMyaGKHd5Fw552p+Qn79doD17UNHPcUaqhyasqw@mail.gmail.com>

Do you see any jobs that are running or queued? Does the batch system log
output of jobs?


On Tue, Jul 30, 2013 at 3:07 AM, Florian M. Wagner <wagnerfl at student.ethz.ch
> wrote:

>  Dear IPython Users,
>
> I am trying to start engines on a big cluster, which uses the Load Sharing
> Facility protocol. The engines seem to start correctly:
>
>  *[wagnerfl at brutus1 ~]$ ipcluster start --profile=cluster --engines=LSF
> --n=4 --IPClusterApp.log_level=DEBUG*
> 2013-07-30 11:55:31,893.893 [IPClusterStart] Using existing profile dir:
> u'/cluster/home02/erdw/wagnerfl/.ipython/profile_cluster'
> 2013-07-30 11:55:31.914 [IPClusterStart] Starting ipcluster with
> [daemon=False]
> 2013-07-30 11:55:31.915 [IPClusterStart] Creating pid file:
> /cluster/home02/erdw/wagnerfl/.ipython/profile_cluster/pid/ipcluster.pid
> 2013-07-30 11:55:31.916 [IPClusterStart] Starting Controller with
> LocalControllerLauncher
> 2013-07-30 11:55:32.915 [IPClusterStart] Starting 4 Engines with LSF Job
> array.
> 2013-07-30 11:55:33.199 [IPClusterStart] Job submitted with job id:
> '27633338'
> 2013-07-30 11:56:03.200 [IPClusterStart] Engines appear to have started
> successfully
>
>
> However, I cannot access them from the Client.
>
> In [1]: from IPython.parallel import Client
> In [2]: c = Client(profile='cluster')
> In [3]: c.ids
> Out[3]: []
>
>  Could you give me a hint how to debug this, or is it not worth trying and
> should I better wait for 1.0?
>
> Thanks a lot in advance
> Florian
>
>
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130730/3a7ac8f9/attachment.html>

From mdroe at stsci.edu  Tue Jul 30 17:28:42 2013
From: mdroe at stsci.edu (Michael Droettboom)
Date: Tue, 30 Jul 2013 17:28:42 -0400
Subject: [IPython-dev] matplotlib webagg benchmarks
Message-ID: <51F8300A.5080303@stsci.edu>

As promised in last week's Google Hangout to the IPython developers 
meeting -- I have some concrete timings and numbers on the matplotlib 
WebAgg backend in a couple of different scenarios.

First, let me apologize -- the way I was timing binary websockets vs. 
text websockets previously was wrong.  The actual impact of it is much 
smaller than I had originally estimated -- so the discussion about 
whether to include binary websockets in IPython may have been all for 
naught.

For benchmarking, I used two different plots.  One is the classic 
"simple_plot.py" sine wave, which tests sort of the "easy case" where 
very little of the image is updated in each frame, and the other was 
"animation/dynamic_image.py" in which most of the plot is updated in 
each frame.

I tested both scenarios with client and server on my local machine, and 
through an ssh tunnel that goes over wifi, the public university 
network, to my home's 15/5 MBps cable connection 28 miles away and back.

For (A), the average frame weighs in at around 20kb.  For (B), it's 
around 90kb.  For base64, multiply by those numbers by 4 / 3.

On my local machine, I can push through about 18 fps, so a bandwidth of 
2.8MBps (were it sustained, which it rarely is).  On the tunnel, I 
fluctuate between 7 and 10 fps, which is quite usable, and quite near 
the practical upper limit on the bandwidth of that connection.

However, the problematic thing for the remote connection is the 
latency.  Locally, I average a fairly steady 250ms to roundtrip from a 
mouse event to an updated frame.  Remotely, it fluctuates randomly 
between 400ms (still usable) and 3000ms.  Some more careful dynamic 
scaling of events can probably make that easier to use, perhaps.  I know 
games often use UDP and handle robustness to packet loss in a different 
way as a way to remove some of the latency of TCP.  I have no idea if 
such a thing would be possible over a web socket, of course.

I could not measure any statistically significant change in framerate or 
latency between a binary websocket and a non-binary one.  However, there 
is a 10% increase in CPU time on both the python side and the browser.  
It so happens that I wasn't saturating my CPU, so it had no net impact.  
Likewise, I am not saturating my bandwidth, so the additional size 
doesn't matter in this case.  But I suspect if either one of those 
resources is starved, the additional 10% cpu time and 25% bandwidth 
increase may matter.

Mike


From wagnerfl at student.ethz.ch  Wed Jul 31 09:32:35 2013
From: wagnerfl at student.ethz.ch (Florian M. Wagner)
Date: Wed, 31 Jul 2013 15:32:35 +0200
Subject: [IPython-dev] Problems with ipcluster and LSF protocol
	(v.0.13.2)
In-Reply-To: <CAHNn8BUe=QMdMyaGKHd5Fw552p+Qn79doD17UNHPcUaqhyasqw@mail.gmail.com>
References: <51F7907B.7060607@student.ethz.ch>
	<CAHNn8BUe=QMdMyaGKHd5Fw552p+Qn79doD17UNHPcUaqhyasqw@mail.gmail.com>
Message-ID: <51F911F3.60309@student.ethz.ch>

Hey Min,

thanks for the reply. I think c.HubFactory.ip = '*' did it for me. The 
engines are found by the Client now. Although, I am experiencing memory 
issues. I think, since ipcluster starts the engines with bsub only, I 
just have the standard resources available. Is it possible to pass 
arguments like the requested walltime etc. to ipcluster or would I have 
to do that over the config files prior to job execution?

Great work with IPython.parallel. I love it!

Am 30.07.2013 20:05, schrieb MinRK:
> Do you see any jobs that are running or queued? Does the batch system 
> log output of jobs?
>
>
> On Tue, Jul 30, 2013 at 3:07 AM, Florian M. Wagner 
> <wagnerfl at student.ethz.ch <mailto:wagnerfl at student.ethz.ch>> wrote:
>
>     Dear IPython Users,
>
>     I am trying to start engines on a big cluster, which uses the Load
>     Sharing Facility protocol. The engines seem to start correctly:
>
>         *[wagnerfl at brutus1 ~]$ ipcluster start --profile=cluster
>         --engines=LSF --n=4 --IPClusterApp.log_level=DEBUG*
>         2013-07-30 11:55:31,893.893 [IPClusterStart] Using existing
>         profile dir:
>         u'/cluster/home02/erdw/wagnerfl/.ipython/profile_cluster'
>         2013-07-30 11:55:31.914 [IPClusterStart] Starting ipcluster
>         with [daemon=False]
>         2013-07-30 11:55:31.915 [IPClusterStart] Creating pid file:
>         /cluster/home02/erdw/wagnerfl/.ipython/profile_cluster/pid/ipcluster.pid
>         2013-07-30 11:55:31.916 [IPClusterStart] Starting Controller
>         with LocalControllerLauncher
>         2013-07-30 11:55:32.915 [IPClusterStart] Starting 4 Engines
>         with LSFJob array.
>         2013-07-30 11:55:33.199 [IPClusterStart] Job submitted with
>         job id: '27633338'
>         2013-07-30 11:56:03.200 [IPClusterStart] Engines appear to
>         have started successfully
>
>
>     However, I cannot access them from the Client.
>
>         In [1]: from IPython.parallel import Client
>         In [2]: c = Client(profile='cluster')
>         In [3]: c.ids
>         Out[3]: []
>
>     Could you give me a hint how to debug this, or is it not worth
>     trying and should I better wait for 1.0?
>
>     Thanks a lot in advance
>     Florian
>
>
>
>
>
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130731/e5ca81f0/attachment.html>