From takowl at gmail.com  Sat Dec  1 06:26:22 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 1 Dec 2012 11:26:22 +0000
Subject: [IPython-dev] [nbconvert] Reining in the number of constructor
	arguments
In-Reply-To: <CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
References: <CALc6Xo7fDnke3SwHaoYVJSA4Qd6RZYYtGSeiHsj819Gw_x+B1g@mail.gmail.com>
	<CAKCGbV6a-u-rA2xbY3PtYHfryaQ69zHz7L8QdBQO95D3AUm0Sg@mail.gmail.com>
	<CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
Message-ID: <CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>

On 1 December 2012 00:34, David Warde-Farley <d.warde.farley at gmail.com>wrote:

> It seems like the right place for this code would be in a class method
> attached to each
> Converter class. Then nbconvert.py could iterate over the classes it
> knows about and
> call cls.cli_argument_parser(), and add that to the global one.
>

Have a look at the Configurable/Application framework we use in the main
IPython codebase - I think this should be flexible enough to use in
nbconvert as well.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121201/00b878b1/attachment.html>

From ellisonbg at gmail.com  Sat Dec  1 15:12:06 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 1 Dec 2012 12:12:06 -0800
Subject: [IPython-dev] [nbconvert] Reining in the number of constructor
	arguments
In-Reply-To: <CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>
References: <CALc6Xo7fDnke3SwHaoYVJSA4Qd6RZYYtGSeiHsj819Gw_x+B1g@mail.gmail.com>
	<CAKCGbV6a-u-rA2xbY3PtYHfryaQ69zHz7L8QdBQO95D3AUm0Sg@mail.gmail.com>
	<CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
	<CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>
Message-ID: <CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>

Thomas makes a very good point:

If the nbcovert code is getting complex enough that these issues are
coming up, then there is no question about what we should do.  We
should handle it exactly like we do everywhere else in the IPython
code base.  The top level converter classes should inherit from
Configurable and the command line programs should be implemented as
Applications.  All of the config file stuff and command line options
will then be handled automatically.  This approach allows the
converters to be configured in numerous ways:

* Command ilne
* Passing kwargs to the classes.
* Setting attributes
* Config files

Is there any reason we wouldn't do this?

On Sat, Dec 1, 2012 at 3:26 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 1 December 2012 00:34, David Warde-Farley <d.warde.farley at gmail.com>
> wrote:
>>
>> It seems like the right place for this code would be in a class method
>> attached to each
>> Converter class. Then nbconvert.py could iterate over the classes it
>> knows about and
>> call cls.cli_argument_parser(), and add that to the global one.
>
>
> Have a look at the Configurable/Application framework we use in the main
> IPython codebase - I think this should be flexible enough to use in
> nbconvert as well.
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sat Dec  1 15:18:35 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 1 Dec 2012 12:18:35 -0800
Subject: [IPython-dev] [nbconvert] Reining in the number of constructor
	arguments
In-Reply-To: <CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>
References: <CALc6Xo7fDnke3SwHaoYVJSA4Qd6RZYYtGSeiHsj819Gw_x+B1g@mail.gmail.com>
	<CAKCGbV6a-u-rA2xbY3PtYHfryaQ69zHz7L8QdBQO95D3AUm0Sg@mail.gmail.com>
	<CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
	<CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>
	<CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>
Message-ID: <CAH4pYpR5cfxAKXb8zg2i_svF37xHwAWE+aTjPSF6XqNvamW=+Q@mail.gmail.com>

I should mention one other reason we need to use the
Configurable/Application framework.  nbcovert will eventually be
integrated into the notebook web application, which itself uses all of
that technology for configuration.

On Sat, Dec 1, 2012 at 12:12 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Thomas makes a very good point:
>
> If the nbcovert code is getting complex enough that these issues are
> coming up, then there is no question about what we should do.  We
> should handle it exactly like we do everywhere else in the IPython
> code base.  The top level converter classes should inherit from
> Configurable and the command line programs should be implemented as
> Applications.  All of the config file stuff and command line options
> will then be handled automatically.  This approach allows the
> converters to be configured in numerous ways:
>
> * Command ilne
> * Passing kwargs to the classes.
> * Setting attributes
> * Config files
>
> Is there any reason we wouldn't do this?
>
> On Sat, Dec 1, 2012 at 3:26 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> On 1 December 2012 00:34, David Warde-Farley <d.warde.farley at gmail.com>
>> wrote:
>>>
>>> It seems like the right place for this code would be in a class method
>>> attached to each
>>> Converter class. Then nbconvert.py could iterate over the classes it
>>> knows about and
>>> call cls.cli_argument_parser(), and add that to the global one.
>>
>>
>> Have a look at the Configurable/Application framework we use in the main
>> IPython codebase - I think this should be flexible enough to use in
>> nbconvert as well.
>>
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From rmcgibbo at gmail.com  Sat Dec  1 21:24:11 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Sat, 1 Dec 2012 18:24:11 -0800
Subject: [IPython-dev] Function specific hooks into the ipython tab
	completion system
In-Reply-To: <DAD945D1-3987-4665-B520-187B90A3AC62@gmail.com>
References: <96A177C3-9E59-44BD-8268-52F802E12C37@gmail.com>
	<CAKgW=6LL=L6=z-zKgFnnS1MYCRWddhVaW89ZC3wCiADTXe6BgQ@mail.gmail.com>
	<357B7D49-C7B4-413A-A530-AA8F63EA79BF@gmail.com>
	<669615BA-66DA-4CA6-8848-ED64BC4E3710@gmail.com>
	<50B77355.8050404@physics.harvard.edu>
	<0B0DB9F0-4400-43CA-8BCD-EA2DEA7CE1FE@gmail.com>
	<3384313335217771310@unknownmsgid>
	<D9065EBB-62FD-4D4C-AF6E-A777C3294DB5@gmail.com>
	<CAOvn4qgTbiN_p_j2do3fufgw2+juT0Lf0QUPhTh97YifCPFm=w@mail.gmail.com>
	<DAD945D1-3987-4665-B520-187B90A3AC62@gmail.com>
Message-ID: <759763F6-F0C4-48A7-8698-9732CF4B90E2@gmail.com>

My PR is up for this feature now, here.

One thing I discovered when working on the code for this feature is that I think the IPython completer code could use some refactoring.

With this PR, there are/will be three independent hooks into the completer system. One of them is is what's used for `import` and `cd` and also provides a somewhat user facing hook API (`get_ipython().set_hook('custom_completer', ...` and `IPython.core.completerlib.quick_completer`). 

Another way of interacting is via interactiveshell.set_custom_completer, which, confusingly, is NOT the same mechanism as `get_ipython().set_hook('custom_completer', ...` (!!).

And now there's this new feature. It's a bit of crazyness.

Because none of the completer methods are talking to each other effectively, there's some duplication of effort. For instance, before any of the completers get called, the input line of code is split on delimiters, but only the last element is saved and passed around. So completers that need more info need to reparse the line. The obvious parsing to do is tokenizing the line, but if two completers need to do that, they don't currently have any way to share the tokens, so the tokenizer will just get run twice.

From working on this new feature, I have some idea of what could be done to improve the situation.

Is it important to maintain full backwards compatibility with the `get_ipython().set_hook('custom_completer', ...` / `IPython.core.completerlib.quick_completer` API? (Is anyone using that in the wild?)

-Robert

On Nov 30, 2012, at 3:55 PM, Robert McGibbon wrote:

> Awesome. Keep me in the loop! I'm happy to play a support role.
> 
> -Robert
> 
> On Nov 30, 2012, at 3:51 PM, Thomas Kluyver wrote:
> 
>> On 30 November 2012 23:47, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>> If someone else wants to try to get this discussion going in the wider python community, I'm all ears. But I don't
>> think that person is going to be me.
>> 
>> I might do that. Thanks for your thoughts - I look forward to seeing the results.
>> 
>> Thomas
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121201/f971cf25/attachment.html>

From d.warde.farley at gmail.com  Sat Dec  1 22:55:42 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Sat, 1 Dec 2012 22:55:42 -0500
Subject: [IPython-dev] [nbconvert] Reining in the number of constructor
	arguments
In-Reply-To: <CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>
References: <CALc6Xo7fDnke3SwHaoYVJSA4Qd6RZYYtGSeiHsj819Gw_x+B1g@mail.gmail.com>
	<CAKCGbV6a-u-rA2xbY3PtYHfryaQ69zHz7L8QdBQO95D3AUm0Sg@mail.gmail.com>
	<CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
	<CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>
	<CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>
Message-ID: <CALc6Xo4EDqwf6uDRvsJ=RNpi-cDSB8PVWzEgfad0etqEx1xSkw@mail.gmail.com>

On Sat, Dec 1, 2012 at 3:12 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Thomas makes a very good point:
>
> If the nbcovert code is getting complex enough that these issues are
> coming up, then there is no question about what we should do.  We
> should handle it exactly like we do everywhere else in the IPython
> code base.  The top level converter classes should inherit from
> Configurable and the command line programs should be implemented as
> Applications.  All of the config file stuff and command line options
> will then be handled automatically.  This approach allows the
> converters to be configured in numerous ways:
>
> * Command ilne
> * Passing kwargs to the classes.
> * Setting attributes
> * Config files
>
> Is there any reason we wouldn't do this?

Not that I can see. Does this imply that attributes declared at the
class level should be declared with Traitlets? (Sorry, a bit new to
this part of the codebase.)

Since rendering a notebook is a complex process that it might be nice
to have some feedback on throughout, should we maybe inherit from
LoggingConfigurable?

David


From d.warde.farley at gmail.com  Sat Dec  1 23:45:03 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Sat, 1 Dec 2012 23:45:03 -0500
Subject: [IPython-dev] [nbconvert] Reining in the number of constructor
	arguments
In-Reply-To: <CALc6Xo4EDqwf6uDRvsJ=RNpi-cDSB8PVWzEgfad0etqEx1xSkw@mail.gmail.com>
References: <CALc6Xo7fDnke3SwHaoYVJSA4Qd6RZYYtGSeiHsj819Gw_x+B1g@mail.gmail.com>
	<CAKCGbV6a-u-rA2xbY3PtYHfryaQ69zHz7L8QdBQO95D3AUm0Sg@mail.gmail.com>
	<CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
	<CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>
	<CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>
	<CALc6Xo4EDqwf6uDRvsJ=RNpi-cDSB8PVWzEgfad0etqEx1xSkw@mail.gmail.com>
Message-ID: <CALc6Xo5raVxV2oFr0AXtw=3L9JUqyZF=jehToOHh_+fT-s7iEQ@mail.gmail.com>

Another question: am I right that it does not seem appropriate to have
the input/output files/streams as traitlet/configurable parameters?
>From my birds-eye view, it seems that this facility is more for
default behaviour modification than specifying "fill-ins" for the core
function of a class. Am I wrong?


From ellisonbg at gmail.com  Sun Dec  2 00:00:31 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 1 Dec 2012 21:00:31 -0800
Subject: [IPython-dev] [nbconvert] Reining in the number of constructor
	arguments
In-Reply-To: <CALc6Xo5raVxV2oFr0AXtw=3L9JUqyZF=jehToOHh_+fT-s7iEQ@mail.gmail.com>
References: <CALc6Xo7fDnke3SwHaoYVJSA4Qd6RZYYtGSeiHsj819Gw_x+B1g@mail.gmail.com>
	<CAKCGbV6a-u-rA2xbY3PtYHfryaQ69zHz7L8QdBQO95D3AUm0Sg@mail.gmail.com>
	<CALc6Xo4oS-tCuzn+qoMMKeDE8dMLJn9by1FKfZXZpYdSQYvS=Q@mail.gmail.com>
	<CAOvn4qj+b+RJp6aPbDHms7Fr3TYahr=NZriofCd-jLwRS-=kMg@mail.gmail.com>
	<CAH4pYpRpxiJ_H_C=Pq4VWZ8_jHv=nteGS9iq51PsPy9uUchm4Q@mail.gmail.com>
	<CALc6Xo4EDqwf6uDRvsJ=RNpi-cDSB8PVWzEgfad0etqEx1xSkw@mail.gmail.com>
	<CALc6Xo5raVxV2oFr0AXtw=3L9JUqyZF=jehToOHh_+fT-s7iEQ@mail.gmail.com>
Message-ID: <CAH4pYpSWipY9N3GkfgF1ASR6W0x=_Vej_8rik8hhcrQwU=gq2Q@mail.gmail.com>

Minimally they should be used for any thing that we ever might want to
*change*.  But sometimes, even in the core functionality of a class
you might want the type checking that traitlets provides.

On Sat, Dec 1, 2012 at 8:45 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> Another question: am I right that it does not seem appropriate to have
> the input/output files/streams as traitlet/configurable parameters?
> >From my birds-eye view, it seems that this facility is more for
> default behaviour modification than specifying "fill-ins" for the core
> function of a class. Am I wrong?
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From satra at mit.edu  Sun Dec  2 09:32:50 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Sun, 2 Dec 2012 09:32:50 -0500
Subject: [IPython-dev] rmagic in notebook
Message-ID: <CA+A4wOmMQGah6-4m7-tkXvtYoff6fs+yVm2SbBvfByS2heARqA@mail.gmail.com>

hi,

two potential issues:

1.  dev docs: magic help not showing up

2. how should one escape $ when doing something like this:

%R
plot(t, dmap$X[,1])

shows up as a syntax error in the notebook.

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121202/a738db88/attachment.html>

From satra at mit.edu  Sun Dec  2 09:47:20 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Sun, 2 Dec 2012 09:47:20 -0500
Subject: [IPython-dev] rmagic in notebook
In-Reply-To: <CA+A4wOmMQGah6-4m7-tkXvtYoff6fs+yVm2SbBvfByS2heARqA@mail.gmail.com>
References: <CA+A4wOmMQGah6-4m7-tkXvtYoff6fs+yVm2SbBvfByS2heARqA@mail.gmail.com>
Message-ID: <CA+A4wO=xpFq224kueK7o0Ky9NNUkypjQ0PgLcagkHeZadPpz+A@mail.gmail.com>

>
> one potential issue:
>
> 1.  dev docs: magic help not showing up
>

the following problem is now solved.

2. how should one escape $ when doing something like this:
>
> %R
> plot(t, dmap$X[,1])
>

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121202/3a9667cc/attachment.html>

From takowl at gmail.com  Sun Dec  2 10:36:05 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 2 Dec 2012 15:36:05 +0000
Subject: [IPython-dev] rmagic in notebook
In-Reply-To: <CA+A4wO=xpFq224kueK7o0Ky9NNUkypjQ0PgLcagkHeZadPpz+A@mail.gmail.com>
References: <CA+A4wOmMQGah6-4m7-tkXvtYoff6fs+yVm2SbBvfByS2heARqA@mail.gmail.com>
	<CA+A4wO=xpFq224kueK7o0Ky9NNUkypjQ0PgLcagkHeZadPpz+A@mail.gmail.com>
Message-ID: <CAOvn4qgET=MEkJvxp1qdEvQWTc2Kk4ZwRF=h8Umb1atVD7EjVg@mail.gmail.com>

On 2 December 2012 14:47, Satrajit Ghosh <satra at mit.edu> wrote:

> the following problem is now solved.


If you solve a problem, please relate how you did so, for others who might
run into it in the future. Or was it simply solved by a newer version of
the IPython code?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121202/1de8dafa/attachment.html>

From satra at mit.edu  Sun Dec  2 13:50:23 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Sun, 2 Dec 2012 13:50:23 -0500
Subject: [IPython-dev] rmagic in notebook
In-Reply-To: <CAOvn4qgET=MEkJvxp1qdEvQWTc2Kk4ZwRF=h8Umb1atVD7EjVg@mail.gmail.com>
References: <CA+A4wOmMQGah6-4m7-tkXvtYoff6fs+yVm2SbBvfByS2heARqA@mail.gmail.com>
	<CA+A4wO=xpFq224kueK7o0Ky9NNUkypjQ0PgLcagkHeZadPpz+A@mail.gmail.com>
	<CAOvn4qgET=MEkJvxp1qdEvQWTc2Kk4ZwRF=h8Umb1atVD7EjVg@mail.gmail.com>
Message-ID: <CA+A4wO=qxNi5tTKsX6aboHtb9yNVSFhN_+Xh-zxVY6T9aKsXvQ@mail.gmail.com>

hi thomas,

oops. sorry - i wrote that email too quickly. there were a couple of things.

the invalid syntax came from using %R instead of %%R with multiline code -
we could perhaps improve the error message.

and then when i had tried %%R i had a 'print dmap' statement in the R code
and R returned a very unhelpful


R error message: u'Error while parsing the string.'


finally i fixed the %R and the print and a library import that i had
forgotten and all was good.


cheers,

satra


On Sun, Dec 2, 2012 at 10:36 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 2 December 2012 14:47, Satrajit Ghosh <satra at mit.edu> wrote:
>
>> the following problem is now solved.
>
>
> If you solve a problem, please relate how you did so, for others who might
> run into it in the future. Or was it simply solved by a newer version of
> the IPython code?
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121202/97fb59f1/attachment.html>

From takowl at gmail.com  Sun Dec  2 14:14:23 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 2 Dec 2012 19:14:23 +0000
Subject: [IPython-dev] rmagic in notebook
In-Reply-To: <CA+A4wO=qxNi5tTKsX6aboHtb9yNVSFhN_+Xh-zxVY6T9aKsXvQ@mail.gmail.com>
References: <CA+A4wOmMQGah6-4m7-tkXvtYoff6fs+yVm2SbBvfByS2heARqA@mail.gmail.com>
	<CA+A4wO=xpFq224kueK7o0Ky9NNUkypjQ0PgLcagkHeZadPpz+A@mail.gmail.com>
	<CAOvn4qgET=MEkJvxp1qdEvQWTc2Kk4ZwRF=h8Umb1atVD7EjVg@mail.gmail.com>
	<CA+A4wO=qxNi5tTKsX6aboHtb9yNVSFhN_+Xh-zxVY6T9aKsXvQ@mail.gmail.com>
Message-ID: <CAOvn4qhHHrM63y-N6QW8nKLZBtKT3WHfq09RnP+Q4g6SPdnDXg@mail.gmail.com>

Thanks Satra,

On 2 December 2012 18:50, Satrajit Ghosh <satra at mit.edu> wrote:

> the invalid syntax came from using %R instead of %%R with multiline code -
> we could perhaps improve the error message.
>

As with lots of error messages, the trick is working out what the user
meant - %R is perfectly valid as a line magic, after all. %R could raise an
error if it gets no arguments, but a SyntaxError like the one you saw will
be raised when we try to parse the cell, which is before line magics get
called.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121202/20205a52/attachment.html>

From damianavila at gmail.com  Sun Dec  2 23:38:19 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Mon, 03 Dec 2012 01:38:19 -0300
Subject: [IPython-dev] slider converter in nbconvert
Message-ID: <50BC2CBB.9060200@gmail.com>

People,

I wrote some basic and dirty code to implement a "slider" option in 
nbconvert to output html-slideshow based in reveal.js.

You can see a demo video here: 
http://www.youtube.com/watch?v=mNEAJvFDneE&feature=g-upl

And you can get more info to test it in this post: 
http://www.damian.oquanta.info/posts/slider-for-nbconvert.html

Any feedback will be appreciated.

Enjoy!

Dami?n.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/b5f0622b/attachment.html>

From carl.input at gmail.com  Mon Dec  3 07:14:28 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 3 Dec 2012 12:14:28 +0000
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <50BC2CBB.9060200@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
Message-ID: <CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>

I've not had chance to run the code, but the demo was good. You should
maybe start trying to get the styles closer to the Notebook though. Yours
looked pretty, but too different. I found myself looking more at what CSS
can do than at the slide functionality itself. Good work though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/3c52da37/attachment.html>

From damianavila at gmail.com  Mon Dec  3 08:06:35 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Mon, 03 Dec 2012 10:06:35 -0300
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
Message-ID: <50BCA3DB.6060507@gmail.com>

El 03/12/12 09:14, Carl Smith escribi?:
>
> I've not had chance to run the code, but the demo was good. You should 
> maybe start trying to get the styles closer to the Notebook though. 
> Yours looked pretty, but too different. I found myself looking more at 
> what CSS can do than at the slide functionality itself. Good work though.
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Hi Carl,

The  first step was to get a functional option in nbconvert to get the 
slides.

For sure, it would be great to have a style closer to the notebook.
But, to get to this point, I have to do a lot of css hack and I don't 
have time for that now.

Also, I like the idea of having other styles to get more "shocking" 
slideshows.
I mean that, maybe, reveal.js/deck.js/Iimpress.js styles are better 
"suited" to show some contents to "some" public.
To "other" public, ie. an academic one, a notebook style would be better.

In the end, it is a matter of taste, but I agree with you that I need to 
add a theme with a notebook style.

Thanks for your feedback, Carl!

Dami?n.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/65b256a8/attachment.html>

From maximilian.albert at gmail.com  Mon Dec  3 08:43:10 2012
From: maximilian.albert at gmail.com (Maximilian Albert)
Date: Mon, 3 Dec 2012 13:43:10 +0000
Subject: [IPython-dev] [nbconvert] Specifying output filename/target
	directory: feedback needed
Message-ID: <CAGA_dmhejbeWDdW43mZUCLOGhaLE1X3Okn=bYpKNnaq0xC+aKg@mail.gmail.com>

Hi all,

a while ago I submitted a pull request (see [1]) to add a "--output" switch
to nbconvert which would allow the user to specify the output directory
and/or output filename of the converted file. At the time I received a
number of really helpful comments, but it turned out that my original
implementation was too complicated and attempted to achieve too much, so it
needed reworking.

Since then I have spent quite a bit of time thinking about it and reworking
it, but I haven't uploaded a new PR because I continuously stumbled upon
certain design questions that have kept me from coming up with a solution
that I'd be happy with (and also I was flooded with work over the past
couple of weeks). In the meantime, there has also been another PR [2]
implementing very similar functionality (namely, specifying an output
filename). However, even though in itself it looks good, I presume it will
suffer from related problems when we try to extend it to support target
directories.

So I figured it would be a good idea to gather some thoughts from the
community before we rush into implementing a quick solution which is hard
to improve later on (e.g. due to concerns about backward compatibility
etc.).

There appear to be two main use cases for an --output flag:

(i) Specifying a target *filename* for the converted file.
(ii) Specifying a destination *directory* for the converted file.

It seems natural to handle these with the same flag. However, this raises a
few questions:

(a) How do we decide whether the --output argument is a filename or a
directory?

We could say that if the argument has a valid extension (e.g. '.rst' or
'.html') then it should be interpreted as a filename, otherwise as a
directory. A minor drawback might be that the user is then forced to use
filenames whose suffix matches the output format, which is is probably ok
in most cases but perhaps not always. Also, what if the --output argument
has a valid extension but the "--format" argument specifies a different
output format? For example:

   nbconvert --format rst --output foo.html foo.ipynb

I guess in this case the "--format" argument should have precedence, but
then the logical thing to do interpret foo.html not a filename but as a
directory, so the resulting output file would be foo.html/foo.rst. This is
not very intuitive. Thus the next question is:

(b) Should we deduce the output format automatically from the filename
extension or always rely on the --format switch? As mentioned, the former
is quite intuitive in most cases but could sometimes lead to subtle
annoyances. But either solution would force us to do a bit of "intelligent
guesswork" as to whether the --output argument is supposed to be a filename
or a directory.

So it seems that allowing the --output argument to be interpreted as either
a filename or a directory would work well for 98% of the cases, but there
are corner cases where it would behave in unexpected and/or annoying ways.

The alternative is to always and unconditionally interpret the --output
argument as a directory and have another switch such as "--output-filename"
to specify the filename. This is a slightly cleaner solution in that it
doesn't require us to guess what the user intended to do. However, it seems
unnecessarily awkward for most use cases. I for one would certainly quickly
get annoyed by it.

In all this, we shouldn't forget that any solution should be easily
extendable to handle the conversion of multiple files, ? la:

   nbconvert --output foo/ *.ipynb

Any thoughts? It seems that it can't be *that* difficult to design a good
CLI for such a simple task. Am I totally overthinking this? And how do
other programs handle it?

Sorry for this long email and thanks in advance for any comments!

Best wishes,
Max


[1] https://github.com/ipython/nbconvert/pull/44
[2] https://github.com/ipython/nbconvert/pull/62
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/af1114e6/attachment.html>

From wen.g.gong at gmail.com  Mon Dec  3 10:21:39 2012
From: wen.g.gong at gmail.com (W Gong)
Date: Mon, 3 Dec 2012 10:21:39 -0500
Subject: [IPython-dev] [nbconvert] Specifying output filename/target
 directory: feedback needed
In-Reply-To: <CAGA_dmhejbeWDdW43mZUCLOGhaLE1X3Okn=bYpKNnaq0xC+aKg@mail.gmail.com>
References: <CAGA_dmhejbeWDdW43mZUCLOGhaLE1X3Okn=bYpKNnaq0xC+aKg@mail.gmail.com>
Message-ID: <CAJzFRJG2gP1LdJi-1KsQ49kJuuyYLWUoHDzRA01d8sOHGvaHQg@mail.gmail.com>

Hi,

here is my wishful spec:

   nbconvert --output <folder or file> --format <html (default) | rst |
...> foo.ipynb

default format is html, so --format is optional
default output is ./foo.html so --output is also optional
internally you determine if the value of output is a folder or file:

     case 1: if input is single file, output is folder, then output is
folder/foo.<fmt>
     case 2: if input is single file, output is file, then output is the
given file
     case 3: if input is multiple (folder or glob), output is a file, then
error out,
     case 4: if input is folder or glob, output is folder, then generate
multiple files as folder/*.<fmt>

Thanks,

Wen



On Mon, Dec 3, 2012 at 8:43 AM, Maximilian Albert <
maximilian.albert at gmail.com> wrote:

> Hi all,
>
> a while ago I submitted a pull request (see [1]) to add a "--output"
> switch to nbconvert which would allow the user to specify the output
> directory and/or output filename of the converted file. At the time I
> received a number of really helpful comments, but it turned out that my
> original implementation was too complicated and attempted to achieve too
> much, so it needed reworking.
>
> Since then I have spent quite a bit of time thinking about it and
> reworking it, but I haven't uploaded a new PR because I continuously
> stumbled upon certain design questions that have kept me from coming up
> with a solution that I'd be happy with (and also I was flooded with work
> over the past couple of weeks). In the meantime, there has also been
> another PR [2] implementing very similar functionality (namely, specifying
> an output filename). However, even though in itself it looks good, I
> presume it will suffer from related problems when we try to extend it to
> support target directories.
>
> So I figured it would be a good idea to gather some thoughts from the
> community before we rush into implementing a quick solution which is hard
> to improve later on (e.g. due to concerns about backward compatibility
> etc.).
>
> There appear to be two main use cases for an --output flag:
>
> (i) Specifying a target *filename* for the converted file.
> (ii) Specifying a destination *directory* for the converted file.
>
> It seems natural to handle these with the same flag. However, this raises
> a few questions:
>
> (a) How do we decide whether the --output argument is a filename or a
> directory?
>
> We could say that if the argument has a valid extension (e.g. '.rst' or
> '.html') then it should be interpreted as a filename, otherwise as a
> directory. A minor drawback might be that the user is then forced to use
> filenames whose suffix matches the output format, which is is probably ok
> in most cases but perhaps not always. Also, what if the --output argument
> has a valid extension but the "--format" argument specifies a different
> output format? For example:
>
>    nbconvert --format rst --output foo.html foo.ipynb
>
> I guess in this case the "--format" argument should have precedence, but
> then the logical thing to do interpret foo.html not a filename but as a
> directory, so the resulting output file would be foo.html/foo.rst. This is
> not very intuitive. Thus the next question is:
>
> (b) Should we deduce the output format automatically from the filename
> extension or always rely on the --format switch? As mentioned, the former
> is quite intuitive in most cases but could sometimes lead to subtle
> annoyances. But either solution would force us to do a bit of "intelligent
> guesswork" as to whether the --output argument is supposed to be a filename
> or a directory.
>
> So it seems that allowing the --output argument to be interpreted as
> either a filename or a directory would work well for 98% of the cases, but
> there are corner cases where it would behave in unexpected and/or annoying
> ways.
>
> The alternative is to always and unconditionally interpret the --output
> argument as a directory and have another switch such as "--output-filename"
> to specify the filename. This is a slightly cleaner solution in that it
> doesn't require us to guess what the user intended to do. However, it seems
> unnecessarily awkward for most use cases. I for one would certainly quickly
> get annoyed by it.
>
> In all this, we shouldn't forget that any solution should be easily
> extendable to handle the conversion of multiple files, ? la:
>
>    nbconvert --output foo/ *.ipynb
>
> Any thoughts? It seems that it can't be *that* difficult to design a good
> CLI for such a simple task. Am I totally overthinking this? And how do
> other programs handle it?
>
> Sorry for this long email and thanks in advance for any comments!
>
> Best wishes,
> Max
>
>
> [1] https://github.com/ipython/nbconvert/pull/44
> [2] https://github.com/ipython/nbconvert/pull/62
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 

Thanks,

- Wen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/dc75f13d/attachment.html>

From bussonniermatthias at gmail.com  Mon Dec  3 11:19:04 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 3 Dec 2012 17:19:04 +0100
Subject: [IPython-dev] [nbconvert] Specifying output filename/target
	directory: feedback needed
In-Reply-To: <CAJzFRJG2gP1LdJi-1KsQ49kJuuyYLWUoHDzRA01d8sOHGvaHQg@mail.gmail.com>
References: <CAGA_dmhejbeWDdW43mZUCLOGhaLE1X3Okn=bYpKNnaq0xC+aKg@mail.gmail.com>
	<CAJzFRJG2gP1LdJi-1KsQ49kJuuyYLWUoHDzRA01d8sOHGvaHQg@mail.gmail.com>
Message-ID: <08A2CE0B-ED17-4F79-9BC6-6BB761D9B1C8@gmail.com>


Le 3 d?c. 2012 ? 16:21, W Gong a ?crit :

> Hi, 
> 
> here is my wishful spec:
> 
>    nbconvert --output <folder or file> --format <html (default) | rst | ...> foo.ipynb
> 
> default format is html, so --format is optional
> default output is ./foo.html so --output is also optional
> internally you determine if the value of output is a folder or file:
> 
>      case 1: if input is single file, output is folder, then output is folder/foo.<fmt>
>      case 2: if input is single file, output is file, then output is the given file
>      case 3: if input is multiple (folder or glob), output is a file, then error out, 
>      case 4: if input is folder or glob, output is folder, then generate multiple files as folder/*.<fmt>
> 
> Thanks,
> 
> Wen

There is a discussion on using IPython Configurable class to handle that. 
As with IPython, you would be able to pass --MyClass.myoption=myconfiguration 
It will allow to avoid to pass all the flags around and would come for free with anyone subclassing.

IMHO and as I already said somewhere,  nbconvert is a nice tool, but the command line part is only secondary.
Also unix philosophy is "do one thing, do it well".
And we won't be able to fit everything in nbconvert, not as command line parameters.

Everyone is speaking of files, but the biggest use of nbconvert will be integrated in notebook/ nbviewer/other tools.
There you will have the option to "download as pdf/html ?" I hope those will not be done with subprocess calling nbconvert.

If the choice where only mine, I would totally hide the file part in nbconvert classes.
Not even the constructor would take a filename but a Json object.
I've always learned to hide IO as much as possible, and for me, converting from A to B does not even involve files.

Then you just expose some attributes of converter object. 
After you just have to create a thin wrapper around theses object that act the way you want in 10 lines and take folder/filesname as parameter.

For what it is worth.
-- 
Matthias



> 
> 
> 
> On Mon, Dec 3, 2012 at 8:43 AM, Maximilian Albert <maximilian.albert at gmail.com> wrote:
> Hi all,
> 
> a while ago I submitted a pull request (see [1]) to add a "--output" switch to nbconvert which would allow the user to specify the output directory and/or output filename of the converted file. At the time I received a number of really helpful comments, but it turned out that my original implementation was too complicated and attempted to achieve too much, so it needed reworking.
> 
> Since then I have spent quite a bit of time thinking about it and reworking it, but I haven't uploaded a new PR because I continuously stumbled upon certain design questions that have kept me from coming up with a solution that I'd be happy with (and also I was flooded with work over the past couple of weeks). In the meantime, there has also been another PR [2] implementing very similar functionality (namely, specifying an output filename). However, even though in itself it looks good, I presume it will suffer from related problems when we try to extend it to support target directories.
> 
> So I figured it would be a good idea to gather some thoughts from the community before we rush into implementing a quick solution which is hard to improve later on (e.g. due to concerns about backward compatibility etc.).
> 
> There appear to be two main use cases for an --output flag:
> 
> (i) Specifying a target *filename* for the converted file.
> (ii) Specifying a destination *directory* for the converted file.
> 
> It seems natural to handle these with the same flag. However, this raises a few questions:
> 
> (a) How do we decide whether the --output argument is a filename or a directory?
> 
> We could say that if the argument has a valid extension (e.g. '.rst' or '.html') then it should be interpreted as a filename, otherwise as a directory. A minor drawback might be that the user is then forced to use filenames whose suffix matches the output format, which is is probably ok in most cases but perhaps not always. Also, what if the --output argument has a valid extension but the "--format" argument specifies a different output format? For example:
> 
>    nbconvert --format rst --output foo.html foo.ipynb
> 
> I guess in this case the "--format" argument should have precedence, but then the logical thing to do interpret foo.html not a filename but as a directory, so the resulting output file would be foo.html/foo.rst. This is not very intuitive. Thus the next question is:
> 
> (b) Should we deduce the output format automatically from the filename extension or always rely on the --format switch? As mentioned, the former is quite intuitive in most cases but could sometimes lead to subtle annoyances. But either solution would force us to do a bit of "intelligent guesswork" as to whether the --output argument is supposed to be a filename or a directory.
> 
> So it seems that allowing the --output argument to be interpreted as either a filename or a directory would work well for 98% of the cases, but there are corner cases where it would behave in unexpected and/or annoying ways.
> 
> The alternative is to always and unconditionally interpret the --output argument as a directory and have another switch such as "--output-filename" to specify the filename. This is a slightly cleaner solution in that it doesn't require us to guess what the user intended to do. However, it seems unnecessarily awkward for most use cases. I for one would certainly quickly get annoyed by it.
> 
> In all this, we shouldn't forget that any solution should be easily extendable to handle the conversion of multiple files, ? la:
> 
>    nbconvert --output foo/ *.ipynb
> 
> Any thoughts? It seems that it can't be *that* difficult to design a good CLI for such a simple task. Am I totally overthinking this? And how do other programs handle it?
> 
> Sorry for this long email and thanks in advance for any comments!
> 
> Best wishes,
> Max
> 
> 
> [1] https://github.com/ipython/nbconvert/pull/44
> [2] https://github.com/ipython/nbconvert/pull/62
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> 
> -- 
> 
> Thanks,
> 
> - Wen
> 
> 
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/c22e4d41/attachment.html>

From ellisonbg at gmail.com  Mon Dec  3 12:40:44 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 3 Dec 2012 09:40:44 -0800
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <50BC2CBB.9060200@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
Message-ID: <CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>

Damian,

This looks great, I really like that you can tune the theme and also
the transitions.  I am also impressed at how nice the PDF generation
is.  I think this will be really useful for people!  One comment,
because it is likely that we will eventuallu support multiple slide
show Js libraries, I think we should name this converter "reveal"
rather than "slider".  But we can cover all that when you have a PR
ready.

Cheers,

Brian

On Sun, Dec 2, 2012 at 8:38 PM, Dami?n Avila <damianavila at gmail.com> wrote:
> People,
>
> I wrote some basic and dirty code to implement a "slider" option in
> nbconvert to output html-slideshow based in reveal.js.
>
> You can see a demo video here:
> http://www.youtube.com/watch?v=mNEAJvFDneE&feature=g-upl
>
> And you can get more info to test it in this post:
> http://www.damian.oquanta.info/posts/slider-for-nbconvert.html
>
> Any feedback will be appreciated.
>
> Enjoy!
>
> Dami?n.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From damianavila at gmail.com  Mon Dec  3 14:17:26 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Mon, 03 Dec 2012 16:17:26 -0300
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
Message-ID: <50BCFAC6.80807@gmail.com>

El 03/12/12 14:40, Brian Granger escribi?:
> Damian,
>
> This looks great, I really like that you can tune the theme and also
> the transitions.  I am also impressed at how nice the PDF generation
> is.  I think this will be really useful for people!  One comment,
> because it is likely that we will eventuallu support multiple slide
> show Js libraries, I think we should name this converter "reveal"
> rather than "slider".  But we can cover all that when you have a PR
> ready.
>
> Cheers,
>
> Brian
>
>

Brian, the tuning of the theme and transitions is provided from 
reveal.js. I agree that is nice feature to have.
And the pdf creation is great (you have to use Chrome to get a correct pdf).
I agree with you that we should name this converter "reveal" (it was my 
first name and then switch to "slider").
Finally, I have never PRed to the ipython project. Do you (or a wiki) 
have some guidelines to prepare the PR?

Regards.

Dami?n.





From bussonniermatthias at gmail.com  Mon Dec  3 15:50:24 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 3 Dec 2012 21:50:24 +0100
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <50BCFAC6.80807@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
	<50BCFAC6.80807@gmail.com>
Message-ID: <9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>


Le 3 d?c. 2012 ? 20:17, Dami?n Avila a ?crit :

> El 03/12/12 14:40, Brian Granger escribi?:
>> Damian,
>> 
>> This looks great, I really like that you can tune the theme and also
>> the transitions.  I am also impressed at how nice the PDF generation
>> is.  I think this will be really useful for people!  One comment,
>> because it is likely that we will eventuallu support multiple slide
>> show Js libraries, I think we should name this converter "reveal"
>> rather than "slider".  But we can cover all that when you have a PR
>> ready.
>> 
>> Cheers,
>> 
>> Brian
>> 
>> 
> 
> Brian, the tuning of the theme and transitions is provided from 
> reveal.js. I agree that is nice feature to have.
> And the pdf creation is great (you have to use Chrome to get a correct pdf).
> I agree with you that we should name this converter "reveal" (it was my 
> first name and then switch to "slider").
> Finally, I have never PRed to the ipython project. Do you (or a wiki) 
> have some guidelines to prepare the PR?

Hi ! 
I'm pretty overwhelmed, I'd **love** to have a deeper look at what you did, 
and even maybe integrate it to nbviewer.

You can't break anything by doing a PR, feel free to try.
https://help.github.com/categories/63/articles  <- how to do a PR.

I think you wrote somewhere that you used --- as separator, I would use metadata to bee clean.
(we could come up with something common between notebook slideshow ext and reveal js mode for metadata)
IIRC reveal.js allow slide and subslide, it would be nice to be able to do this to.

Thanks! 
-- 
Matthias

> Regards.
> 
> Dami?n.
> 
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From ellisonbg at gmail.com  Mon Dec  3 16:02:24 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 3 Dec 2012 13:02:24 -0800
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
	<50BCFAC6.80807@gmail.com>
	<9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>
Message-ID: <CAH4pYpR+=2Gg030ioBgn_D-LN784fkzV1=H-HGEnGjt35JYR-w@mail.gmail.com>

Yes, eventually both the notebook slide show mode and the reveal
nbcoverter should both use cell metadata to make the slide decisions.

On Mon, Dec 3, 2012 at 12:50 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 3 d?c. 2012 ? 20:17, Dami?n Avila a ?crit :
>
>> El 03/12/12 14:40, Brian Granger escribi?:
>>> Damian,
>>>
>>> This looks great, I really like that you can tune the theme and also
>>> the transitions.  I am also impressed at how nice the PDF generation
>>> is.  I think this will be really useful for people!  One comment,
>>> because it is likely that we will eventuallu support multiple slide
>>> show Js libraries, I think we should name this converter "reveal"
>>> rather than "slider".  But we can cover all that when you have a PR
>>> ready.
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>>
>>
>> Brian, the tuning of the theme and transitions is provided from
>> reveal.js. I agree that is nice feature to have.
>> And the pdf creation is great (you have to use Chrome to get a correct pdf).
>> I agree with you that we should name this converter "reveal" (it was my
>> first name and then switch to "slider").
>> Finally, I have never PRed to the ipython project. Do you (or a wiki)
>> have some guidelines to prepare the PR?
>
> Hi !
> I'm pretty overwhelmed, I'd **love** to have a deeper look at what you did,
> and even maybe integrate it to nbviewer.
>
> You can't break anything by doing a PR, feel free to try.
> https://help.github.com/categories/63/articles  <- how to do a PR.
>
> I think you wrote somewhere that you used --- as separator, I would use metadata to bee clean.
> (we could come up with something common between notebook slideshow ext and reveal js mode for metadata)
> IIRC reveal.js allow slide and subslide, it would be nice to be able to do this to.
>
> Thanks!
> --
> Matthias
>
>> Regards.
>>
>> Dami?n.
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Mon Dec  3 16:05:35 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 3 Dec 2012 21:05:35 +0000
Subject: [IPython-dev] [nbconvert] Specifying output filename/target
 directory: feedback needed
In-Reply-To: <08A2CE0B-ED17-4F79-9BC6-6BB761D9B1C8@gmail.com>
References: <CAGA_dmhejbeWDdW43mZUCLOGhaLE1X3Okn=bYpKNnaq0xC+aKg@mail.gmail.com>
	<CAJzFRJG2gP1LdJi-1KsQ49kJuuyYLWUoHDzRA01d8sOHGvaHQg@mail.gmail.com>
	<08A2CE0B-ED17-4F79-9BC6-6BB761D9B1C8@gmail.com>
Message-ID: <CAP-uhDd+=5p38Uco3WGn+f_YwYpYQFa5dLiD_fEWtD6oA+ST2A@mail.gmail.com>

Just two pennies: Can't you just say if the path is to an existing dir,
it's a dir, else it's a file or an error.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121203/0001a046/attachment.html>

From d.warde.farley at gmail.com  Mon Dec  3 18:21:09 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Mon, 3 Dec 2012 18:21:09 -0500
Subject: [IPython-dev] [nbconvert] Specifying output filename/target
 directory: feedback needed
In-Reply-To: <CAP-uhDd+=5p38Uco3WGn+f_YwYpYQFa5dLiD_fEWtD6oA+ST2A@mail.gmail.com>
References: <CAGA_dmhejbeWDdW43mZUCLOGhaLE1X3Okn=bYpKNnaq0xC+aKg@mail.gmail.com>
	<CAJzFRJG2gP1LdJi-1KsQ49kJuuyYLWUoHDzRA01d8sOHGvaHQg@mail.gmail.com>
	<08A2CE0B-ED17-4F79-9BC6-6BB761D9B1C8@gmail.com>
	<CAP-uhDd+=5p38Uco3WGn+f_YwYpYQFa5dLiD_fEWtD6oA+ST2A@mail.gmail.com>
Message-ID: <CALc6Xo4vZKAOT1x8LMeqDrdVnbGMVzp6se66oRFEJAdc+j+HRw@mail.gmail.com>

Fernando was pretty adamant that the output path should always be a
directory, IIRC he also suggested it should be created if
non-existent.

On Mon, Dec 3, 2012 at 4:05 PM, Carl Smith <carl.input at gmail.com> wrote:
> Just two pennies: Can't you just say if the path is to an existing dir, it's
> a dir, else it's a file or an error.
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From damianavila at gmail.com  Mon Dec  3 18:44:48 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Mon, 03 Dec 2012 20:44:48 -0300
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
	<50BCFAC6.80807@gmail.com>
	<9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>
Message-ID: <50BD3970.2020706@gmail.com>

El 03/12/12 17:50, Matthias BUSSONNIER escribi?:

> Hi !
> I'm pretty overwhelmed, I'd **love** to have a deeper look at what you did,
> and even maybe integrate it to nbviewer.
This would be great!
> You can't break anything by doing a PR, feel free to try.
> https://help.github.com/categories/63/articles  <- how to do a PR.
Thanks, I asked for an specific documentation inside ipython project.
So, following that link, I will try to PR and, eventually, you will have 
to guide me :-)
> I think you wrote somewhere that you used --- as separator, I would use metadata to bee clean.
> (we could come up with something common between notebook slideshow ext and reveal js mode for metadata)
I have to think how can I do that, let me think a bit... but I agree 
with you that we have to come up with a common separator.
> IIRC reveal.js allow slide and subslide, it would be nice to be able to do this to.
In the current implementation is difficult to get subslides... but let 
me think a bit... again
> Thanks!
Thanks to you!



From damianavila at gmail.com  Mon Dec  3 23:41:45 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Tue, 04 Dec 2012 01:41:45 -0300
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <CAH4pYpR+=2Gg030ioBgn_D-LN784fkzV1=H-HGEnGjt35JYR-w@mail.gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
	<50BCFAC6.80807@gmail.com>
	<9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>
	<CAH4pYpR+=2Gg030ioBgn_D-LN784fkzV1=H-HGEnGjt35JYR-w@mail.gmail.com>
Message-ID: <50BD7F09.1000000@gmail.com>

El 03/12/12 18:02, Brian Granger escribi?:
> Yes, eventually both the notebook slide show mode and the reveal
> nbcoverter should both use cell metadata to make the slide decisions.
>
> On Mon, Dec 3, 2012 at 12:50 PM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> Le 3 d?c. 2012 ? 20:17, Dami?n Avila a ?crit :
>>
>>> El 03/12/12 14:40, Brian Granger escribi?:
>>>> Damian,
>>>>
>>>> This looks great, I really like that you can tune the theme and also
>>>> the transitions.  I am also impressed at how nice the PDF generation
>>>> is.  I think this will be really useful for people!  One comment,
>>>> because it is likely that we will eventuallu support multiple slide
>>>> show Js libraries, I think we should name this converter "reveal"
>>>> rather than "slider".  But we can cover all that when you have a PR
>>>> ready.
>>>>
>>>> Cheers,
>>>>
>>>> Brian
>>>>
>>>>
>>> Brian, the tuning of the theme and transitions is provided from
>>> reveal.js. I agree that is nice feature to have.
>>> And the pdf creation is great (you have to use Chrome to get a correct pdf).
>>> I agree with you that we should name this converter "reveal" (it was my
>>> first name and then switch to "slider").
>>> Finally, I have never PRed to the ipython project. Do you (or a wiki)
>>> have some guidelines to prepare the PR?
>> Hi !
>> I'm pretty overwhelmed, I'd **love** to have a deeper look at what you did,
>> and even maybe integrate it to nbviewer.
>>
>> You can't break anything by doing a PR, feel free to try.
>> https://help.github.com/categories/63/articles  <- how to do a PR.
>>
>> I think you wrote somewhere that you used --- as separator, I would use metadata to bee clean.
>> (we could come up with something common between notebook slideshow ext and reveal js mode for metadata)
>> IIRC reveal.js allow slide and subslide, it would be nice to be able to do this to.
>>
>> Thanks!
>> --
>> Matthias
>>
>>> Regards.
>>>
>>> Dami?n.
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

OK, here I go with the PR: https://github.com/ipython/nbconvert/pull/63
We can discuss there for new ideas and improvements.

Regards.

Dami?n.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121204/365a8a8a/attachment.html>

From ellisonbg at gmail.com  Mon Dec  3 23:55:19 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 3 Dec 2012 20:55:19 -0800
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <50BD7F09.1000000@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAH4pYpQyYK6yheawoj1KZXMkmsd+dLYxjF5gVctP=TQVYfwjzQ@mail.gmail.com>
	<50BCFAC6.80807@gmail.com>
	<9DD09762-0DDB-45B4-A709-F354B8A382FF@gmail.com>
	<CAH4pYpR+=2Gg030ioBgn_D-LN784fkzV1=H-HGEnGjt35JYR-w@mail.gmail.com>
	<50BD7F09.1000000@gmail.com>
Message-ID: <CAH4pYpSEC_eoX5UXRj-hMu-sjFfirGU2CZUE=8Z1GoyEBW1YBA@mail.gmail.com>

Great!

On Mon, Dec 3, 2012 at 8:41 PM, Dami?n Avila <damianavila at gmail.com> wrote:
> El 03/12/12 18:02, Brian Granger escribi?:
>
> Yes, eventually both the notebook slide show mode and the reveal
> nbcoverter should both use cell metadata to make the slide decisions.
>
> On Mon, Dec 3, 2012 at 12:50 PM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>
> Le 3 d?c. 2012 ? 20:17, Dami?n Avila a ?crit :
>
> El 03/12/12 14:40, Brian Granger escribi?:
>
> Damian,
>
> This looks great, I really like that you can tune the theme and also
> the transitions.  I am also impressed at how nice the PDF generation
> is.  I think this will be really useful for people!  One comment,
> because it is likely that we will eventuallu support multiple slide
> show Js libraries, I think we should name this converter "reveal"
> rather than "slider".  But we can cover all that when you have a PR
> ready.
>
> Cheers,
>
> Brian
>
>
> Brian, the tuning of the theme and transitions is provided from
> reveal.js. I agree that is nice feature to have.
> And the pdf creation is great (you have to use Chrome to get a correct pdf).
> I agree with you that we should name this converter "reveal" (it was my
> first name and then switch to "slider").
> Finally, I have never PRed to the ipython project. Do you (or a wiki)
> have some guidelines to prepare the PR?
>
> Hi !
> I'm pretty overwhelmed, I'd **love** to have a deeper look at what you did,
> and even maybe integrate it to nbviewer.
>
> You can't break anything by doing a PR, feel free to try.
> https://help.github.com/categories/63/articles  <- how to do a PR.
>
> I think you wrote somewhere that you used --- as separator, I would use
> metadata to bee clean.
> (we could come up with something common between notebook slideshow ext and
> reveal js mode for metadata)
> IIRC reveal.js allow slide and subslide, it would be nice to be able to do
> this to.
>
> Thanks!
> --
> Matthias
>
> Regards.
>
> Dami?n.
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> OK, here I go with the PR: https://github.com/ipython/nbconvert/pull/63
> We can discuss there for new ideas and improvements.
>
> Regards.
>
> Dami?n.
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From massimodisasha at gmail.com  Wed Dec  5 05:42:24 2012
From: massimodisasha at gmail.com (epi)
Date: Wed, 5 Dec 2012 05:42:24 -0500
Subject: [IPython-dev] custom JS and IPython.notebook.kernel usage (svn
	example)
Message-ID: <315615DE-AF96-45C5-9ABE-221217B1F43A@gmail.com>

Hi,


i'm used to have my "c.NotebookManager.notebook_dir" under version control, for now SVN ? i should switch to GIT

i was playing with js in a custom.js file,  in order to add some "buttons"  to "COMMITT"  change to a notebook into my repository
(the plan was to add a custom toolbar to execute simple svn commands)

an example to store the results of svn info in a dict can be like :



		IPython.toolbar.add_buttons_group([
		    {
		         'label'   : 'SVN Info',
		         'icon'    : 'ui-icon-info', 
		         'callback': function(){IPython.notebook.kernel.execute('svninfolist = !svn info'),
				                        IPython.notebook.kernel.execute('svninfo = {}'),
				                        IPython.notebook.kernel.execute('for i in svninfolist[:-1]: j=i.split(":") ; svninfo[j[0]]=j[1].strip()')
										}
		    }
		    ]);

this give me a dict like :

2]:

svninfo
Out[2]:
{'Last Changed Author': 'epifanio',
 'Last Changed Date': '2012-11-20 15:30:34 -0500 (Tue, 20 Nov 2012)',
 'Last Changed Rev': '7847',
 'Node Kind': 'directory',
 'Path': '.',
 'Repository Root': 'https://myuser at myserver.org/svn',
 'Repository UUID': 'e8c77636-c7aa-4457-b6c1-051e789c14f2',
 'Revision': '7877',
 'Schedule': 'normal',
 'URL': 'https://myuser at myserver.org/svn/path/to/dir'
'}

while in order to "commit" changes, i was looking for something like :

- detect the name  for the "notebook" (filename.ipnb)

- clean all the output of the "notebook"

- save the "notebook"

- commit the saved file (here should prompt a js text input box where to add the commit notes, but i can live without it as a start)


		IPython.toolbar.add_buttons_group([
		    {
		         'label'   : 'Save & Commit Notebook',
		         'icon'    : 'ui-icon-circle-arrow-n', 
		         'callback': function(){IPython.notebook.kernel.execute('detect the notebook name'),
				                        IPython.notebook.kernel.execute('clean notebook'),
							IPython.notebook.kernel.execute('save notebook'),
							IPython.notebook.kernel.execute('svn commit notebook')}
		    }
		    ]);


i need an help to learn how to use "IPython.notebook.kernel.execute" from inside a JS code
i was looking in `docs/examples/widgets/directview` but i got lost :(

have you any hints on how can i use the ipython.notebook.kernel  in order to :

- print out some text in a new notebook cell 
- detect the name for the active notebook
- clear all the output for the active notebook


Thanks a lot for you help,

Massimo


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121205/72763a20/attachment.html>

From bussonniermatthias at gmail.com  Wed Dec  5 10:23:25 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 5 Dec 2012 16:23:25 +0100
Subject: [IPython-dev] custom JS and IPython.notebook.kernel usage (svn
	example)
In-Reply-To: <315615DE-AF96-45C5-9ABE-221217B1F43A@gmail.com>
References: <315615DE-AF96-45C5-9ABE-221217B1F43A@gmail.com>
Message-ID: <4784D0DA-F8A4-45DC-9B0B-06A7652CC2DF@gmail.com>

Hi there !

You did already a pretty decent job !

we are starting to document javascript you can see what it looks like here :
http://elacave.lmdb.eu/~carreau/yui/
And if you have more do cot add we'll be happy to take your pull request.



Le 5 d?c. 2012 ? 11:42, epi a ?crit :

> Hi,
> 
> 
> i'm used to have my "c.NotebookManager.notebook_dir" under version control, for now SVN ? i should switch to GIT
> 
> i was playing with js in a custom.js file,  in order to add some "buttons"  to "COMMITT"  change to a notebook into my repository
> (the plan was to add a custom toolbar to execute simple svn commands)
> 
> an example to store the results of svn info in a dict can be like :
> 
> 
> 
> 		IPython.toolbar.add_buttons_group([
> 		    {
> 		         'label'   : 'SVN Info',
> 		         'icon'    : 'ui-icon-info', 
> 		         'callback': function(){IPython.notebook.kernel.execute('svninfolist = !svn info'),
> 				                        IPython.notebook.kernel.execute('svninfo = {}'),
> 				                        IPython.notebook.kernel.execute('for i in svninfolist[:-1]: j=i.split(":") ; svninfo[j[0]]=j[1].strip()')
> 										}
> 		    }
> 		    ]);
> 
> this give me a dict like :
> 
> 2]:
> 
> svninfo
> Out[2]:
> {'Last Changed Author': 'epifanio',
>  'Last Changed Date': '2012-11-20 15:30:34 -0500 (Tue, 20 Nov 2012)',
>  'Last Changed Rev': '7847',
>  'Node Kind': 'directory',
>  'Path': '.',
>  'Repository Root': 'https://myuser at myserver.org/svn',
>  'Repository UUID': 'e8c77636-c7aa-4457-b6c1-051e789c14f2',
>  'Revision': '7877',
>  'Schedule': 'normal',
>  'URL': 'https://myuser at myserver.org/svn/path/to/dir'
> '}
> 
> while in order to "commit" changes, i was looking for something like :
> 
> - detect the name  for the "notebook" (filename.ipnb)

I'm not sure this can be easily accessible, but it could be a request for enhancement,
what you can do is : 
IPython.notebook.notebook_name
To lowercasse, and space to _ then add ipynb.

> - clean all the output of the "notebook"

this can be done progrmatically on server side with a custom script on the ipynb file.

> 
> - save the "notebook"

IPython.notebook.save_notebook()

> 
> - commit the saved file (here should prompt a js text input box where to add the commit notes, but i can live without it as a start)

it is doable, just a question of time to code it.

> 
> 
> 		IPython.toolbar.add_buttons_group([
> 		    {
> 		         'label'   : 'Save & Commit Notebook',
> 		         'icon'    : 'ui-icon-circle-arrow-n', 
> 		         'callback': function(){IPython.notebook.kernel.execute('detect the notebook name'),
> 				                        IPython.notebook.kernel.execute('clean notebook'),
> 							IPython.notebook.kernel.execute('save notebook'),
> 							IPython.notebook.kernel.execute('svn commit notebook')}
> 		    }
> 		    ]);
> 
> 
> i need an help to learn how to use "IPython.notebook.kernel.execute" from inside a JS code
> i was looking in `docs/examples/widgets/directview` but i got lost :(


So kernel.execute take callbacks, which get the data back, it's a little tricky.

let's do it for git : 

IPython.notebook.kernel.execute(
	'a=!git status', 
		{'execute_reply': function(data){console.log('data:',data)}},
	 	{'user_variables':['a']}
    )

( Note for later, to do this properly, it should be done with user_expression, but there is an issue, i don't now why)

function(data){console.log('data:',data)}

will be called with the response as parameter, 
here login the data to the js console.

Data will be an object with a few parameters ( i'll let you explore), the one that interests us is user_variable ( we requested 'a' above) 

which itself as a 'a' key.

data.user_variables.a is the following strig :
"['# On branch master', '# Changes not staged for commit:', '#   (use "git add <file>..." to update what will be committed)', '#   (use "git checkout -- <file>..." to discard?.."
(using user_expression would prevnt polluting the kernel with the 'a' variable but it raises.
then you can do whatever you wish with the variable in the callback.

see 
http://wiki.ipython.org/Notebook_feedback
to strip notebook programatically ( in the gist at the end of the page) 


> 
> have you any hints on how can i use the ipython.notebook.kernel  in order to :
> 
> - print out some text in a new notebook cell 

IPython.notebook.get_cell(i).set_text(?)

IPython.notebook.select(n)
IPython.notebook.insert_cell_above()
?

should be of help.

> - detect the name for the active notebook
> - clear all the output for the active notebook
> 
> 
> Thanks a lot for you help,

click on button -> callback a

callback a fetch info on the kernel trigger callback b on the response

callback b -> show a dialog to ask for commit message

click on ok trigger the correct information to be send on the kernel to commit.


Does it help ? 
We probably want to do something like that in a more generic way embedded into IPython notebook. 

We'll be happy to get your feedback on that !

-- 
Matthias

> 
> Massimo
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121205/9e7fedfb/attachment.html>

From emmanuelle.gouillart at nsup.org  Wed Dec  5 17:24:17 2012
From: emmanuelle.gouillart at nsup.org (Emmanuelle Gouillart)
Date: Wed, 5 Dec 2012 23:24:17 +0100
Subject: [IPython-dev] using the notebook for teaching?
Message-ID: <20121205222417.GB5894@phare.normalesup.org>


	Hello,

	from next year on in France, Python will be taught as the first
programming language for scientific computing in an important fraction of
French undergraduate schools (the scientific "preparatory classes" in the
French educational system). I'm trying to set up a course on scientific
Python for the future French professors that will teach Scientific
Python, and I'm wondering whether I should promote the use of the Ipython
notebook as the main educational tool for classes, or whether I should
stick to more traditional tools.

	My main concern is about stability. As I see it, a professor
could have a set of different notebook files corresponding to the
different sessions, and encourage the students to do the same. However,
when I have tried to open existing .ipynb files, I've run across an
incompatibility between ipython 0.12 and notebooks in version 3 format
(error message "Unreadable JSON notebook"). I have not tried whether
backward compatibility works, ie whether ipython 0.13 can read version 2
format. Is it planned that the notebook format will still change a lot or
not? Indeed, I do not expect the undergraduate schools to update ipython
very frequently, and it would be quite a problem if a professor cannot
grade a student's notebook because it was written on a more recent
Ipython... On the other hand, the notebook is really cool and has a lot
of advantages for teaching, so I really have mixed feelings... Stability
is paramount for earning the trust of professors, since a fraction of
them is used to Matlab/Scilab and a bit wary of Python (they will have
the choice between Scilab and Python, although Python is recommended by
the French "official program"): I cannot afford to take too many risks.
What do you think? Any insights on using the notebook for teaching are
very welcome.

	Cheers,
	Emmanuelle


From takowl at gmail.com  Wed Dec  5 18:11:35 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 5 Dec 2012 23:11:35 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121205222417.GB5894@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
Message-ID: <CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>

On 5 December 2012 22:24, Emmanuelle Gouillart <
emmanuelle.gouillart at nsup.org> wrote:

>         My main concern is about stability. As I see it, a professor
> could have a set of different notebook files corresponding to the
> different sessions, and encourage the students to do the same. However,
> when I have tried to open existing .ipynb files, I've run across an
> incompatibility between ipython 0.12 and notebooks in version 3 format
> (error message "Unreadable JSON notebook"). I have not tried whether
> backward compatibility works, ie whether ipython 0.13 can read version 2
> format. Is it planned that the notebook format will still change a lot or
> not? Indeed, I do not expect the undergraduate schools to update ipython
> very frequently, and it would be quite a problem if a professor cannot
> grade a student's notebook because it was written on a more recent
> Ipython... On the other hand, the notebook is really cool and has a lot
> of advantages for teaching, so I really have mixed feelings... Stability
> is paramount for earning the trust of professors, since a fraction of
> them is used to Matlab/Scilab and a bit wary of Python (they will have
> the choice between Scilab and Python, although Python is recommended by
> the French "official program"): I cannot afford to take too many risks.
> What do you think? Any insights on using the notebook for teaching are
> very welcome.
>

0.13 can indeed open v2 notebooks, although it only saves in v3, so it's
not practical to use 0.12 and 0.13 together. But we intend to keep
compatibility for much longer now - the v3 format was designed to allow
much more extension without breaking the ability of 0.13 to read it. This
came up in the discussion around standardising the Scipy stack as well.
There we said that if we have to break compatibility again, we will need to
have an overlap period where IPython can read the new format, but still
saves in the old format by default.

Obviously the notebook is still quite young and rapidly developing, but we
see the format break from 0.12 to 0.13 as the exception, not the rule. If
you decide to use it, make sure everyone starts with at least 0.13.

That's great news about the scientific computing curriculum. Good luck with
training more people in Python!

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121205/047c7098/attachment.html>

From ellisonbg at gmail.com  Wed Dec  5 19:32:14 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 5 Dec 2012 16:32:14 -0800
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
Message-ID: <CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>

Hi,

Glad you hear you are thinking about using the notebook in this
capacity.  We think it is a great tool for teaching.  You ask good
questions about the stability of the notebook format, but I will reply
to Thomas as well...

> 0.13 can indeed open v2 notebooks, although it only saves in v3, so it's not
> practical to use 0.12 and 0.13 together. But we intend to keep compatibility
> for much longer now - the v3 format was designed to allow much more
> extension without breaking the ability of 0.13 to read it. This came up in
> the discussion around standardising the Scipy stack as well. There we said
> that if we have to break compatibility again, we will need to have an
> overlap period where IPython can read the new format, but still saves in the
> old format by default.

I don't recall this discussion about the notebook format and am
unaware that we made any such promise.  Do you have a link to the
discussion?  I neither oppose nor approve of the decision (I haven't
thought enough about it) at this point, I am just unaware of it and
want to learn more.

> Obviously the notebook is still quite young and rapidly developing, but we
> see the format break from 0.12 to 0.13 as the exception, not the rule. If
> you decide to use it, make sure everyone starts with at least 0.13.

Yes.  Right now, the notebook format is quite stable.  But the reason
we created notebook format versions in the first place is to enable
the version to change.  While we want to avoid incrementing the
version, it will happen.  But I want to emphasize that at this point,
we have no plans of incrementing the version any time soon.  In fact,
I am not aware of any feature being discussed that would require such
changing the format.

Cheers,

Brian

> That's great news about the scientific computing curriculum. Good luck with
> training more people in Python!
>
> Best wishes,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From takowl at gmail.com  Wed Dec  5 19:56:17 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 00:56:17 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
Message-ID: <CAOvn4qhMZn9iUHK3Z2zD0rQ29dL46AaAiCNE4-+Gzzmmvs32PA@mail.gmail.com>

On 6 December 2012 00:32, Brian Granger <ellisonbg at gmail.com> wrote:

> I don't recall this discussion about the notebook format and am
> unaware that we made any such promise.  Do you have a link to the
> discussion?  I neither oppose nor approve of the decision (I haven't
> thought enough about it) at this point, I am just unaware of it and
> want to learn more.
>

I'm not sure that it was exactly a promise, but when we were deciding
whether or not the notebook should be specified as part of the core Scipy
Stack, the question of stability came up. The compatibility break that we
had from 0.12 to 0.13 was putting some people off including it, but we
clearly didn't want to commit to infinite forwards compatibility. I
suggested the compromise of a transition period, which all sides seemed
happy with.

Put another way, any two consecutive major versions of IPython should allow
interchanging notebook files without any special precautions. I think
that's a realistic target to aim for.

As for a link, sorry, I don't have a direct one. I think it was somewhere
in this monster thread on scipy-user (
http://article.gmane.org/gmane.comp.python.scientific.user/32620 ), but it
could also have been on the numfocus list. Fernando was there arguing
IPython's corner.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/c847b186/attachment.html>

From jiffyclub at gmail.com  Wed Dec  5 20:05:15 2012
From: jiffyclub at gmail.com (Matt)
Date: Wed, 5 Dec 2012 20:05:15 -0500
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121205222417.GB5894@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
Message-ID: <81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>

Hi Emmanuelle,

You may find it useful to get in touch with Software Carpentry (http://software-carpentry.org). We've started using the IPython Notebook extensively in our scientific Python instruction and consider it more or less the best tool for the job:
http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/.

You can send inquiries to info at software-carpentry.org, we've even got some members in France.

Best,
Matt

On Dec 5, 2012, at 5:24 PM, Emmanuelle Gouillart <emmanuelle.gouillart at nsup.org> wrote:

> 
> 	Hello,
> 
> 	from next year on in France, Python will be taught as the first
> programming language for scientific computing in an important fraction of
> French undergraduate schools (the scientific "preparatory classes" in the
> French educational system). I'm trying to set up a course on scientific
> Python for the future French professors that will teach Scientific
> Python, and I'm wondering whether I should promote the use of the Ipython
> notebook as the main educational tool for classes, or whether I should
> stick to more traditional tools.
> 
> 	My main concern is about stability. As I see it, a professor
> could have a set of different notebook files corresponding to the
> different sessions, and encourage the students to do the same. However,
> when I have tried to open existing .ipynb files, I've run across an
> incompatibility between ipython 0.12 and notebooks in version 3 format
> (error message "Unreadable JSON notebook"). I have not tried whether
> backward compatibility works, ie whether ipython 0.13 can read version 2
> format. Is it planned that the notebook format will still change a lot or
> not? Indeed, I do not expect the undergraduate schools to update ipython
> very frequently, and it would be quite a problem if a professor cannot
> grade a student's notebook because it was written on a more recent
> Ipython... On the other hand, the notebook is really cool and has a lot
> of advantages for teaching, so I really have mixed feelings... Stability
> is paramount for earning the trust of professors, since a fraction of
> them is used to Matlab/Scilab and a bit wary of Python (they will have
> the choice between Scilab and Python, although Python is recommended by
> the French "official program"): I cannot afford to take too many risks.
> What do you think? Any insights on using the notebook for teaching are
> very welcome.
> 
> 	Cheers,
> 	Emmanuelle
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121205/33e28cb9/attachment.html>

From emmanuelle.gouillart at nsup.org  Thu Dec  6 03:45:10 2012
From: emmanuelle.gouillart at nsup.org (Emmanuelle Gouillart)
Date: Thu, 6 Dec 2012 09:45:10 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
Message-ID: <20121206084510.GA29924@phare.normalesup.org>

Hi Thomas and Brian,

thank you very much for your answers. I'm glad to know that no format
change is planned for now, and that developers also consider
forward-compatibility to be important.

It will be hard to have everybody use 0.13 during the training: 0.12 is
the version packaged by the latest Ubuntu LTS (12.04) and by the latest
EPD. I do not expect people to be more bleeding-edge than the latest
Ubuntu LTS. Fortunately, the training will be in Spring, hopefully
there will exist a new version of EPD with Ipython 13.

Thanks again,
Emmanuelle

On Wed, Dec 05, 2012 at 04:32:14PM -0800, Brian Granger wrote:
> Hi,

> Glad you hear you are thinking about using the notebook in this
> capacity.  We think it is a great tool for teaching.  You ask good
> questions about the stability of the notebook format, but I will reply
> to Thomas as well...

> > 0.13 can indeed open v2 notebooks, although it only saves in v3, so it's not
> > practical to use 0.12 and 0.13 together. But we intend to keep compatibility
> > for much longer now - the v3 format was designed to allow much more
> > extension without breaking the ability of 0.13 to read it. This came up in
> > the discussion around standardising the Scipy stack as well. There we said
> > that if we have to break compatibility again, we will need to have an
> > overlap period where IPython can read the new format, but still saves in the
> > old format by default.

> I don't recall this discussion about the notebook format and am
> unaware that we made any such promise.  Do you have a link to the
> discussion?  I neither oppose nor approve of the decision (I haven't
> thought enough about it) at this point, I am just unaware of it and
> want to learn more.

> > Obviously the notebook is still quite young and rapidly developing, but we
> > see the format break from 0.12 to 0.13 as the exception, not the rule. If
> > you decide to use it, make sure everyone starts with at least 0.13.

> Yes.  Right now, the notebook format is quite stable.  But the reason
> we created notebook format versions in the first place is to enable
> the version to change.  While we want to avoid incrementing the
> version, it will happen.  But I want to emphasize that at this point,
> we have no plans of incrementing the version any time soon.  In fact,
> I am not aware of any feature being discussed that would require such
> changing the format.

> Cheers,

> Brian

> > That's great news about the scientific computing curriculum. Good luck with
> > training more people in Python!

> > Best wishes,
> > Thomas

> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev


From emmanuelle.gouillart at nsup.org  Thu Dec  6 04:05:12 2012
From: emmanuelle.gouillart at nsup.org (Emmanuelle Gouillart)
Date: Thu, 6 Dec 2012 10:05:12 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
Message-ID: <20121206090512.GB29924@phare.normalesup.org>


Hi Matt,

thank you for your answer. The article on software carpentry is very
interesting and I should definitely ask for more details on the Software
Carpentry mailing-list.

In fact, I have no doubt that I (as the instructor) can use the Ipython
Notebook in a pleasant and efficient way. The question is whether this is
the right tool for hundreds of maths and physics teachers, most of whom
are not very geeky (in France, teachers in the "preparatory classes"
undergraduate schools are not researchers, they do 100% of teaching). I
guess I have to take a bet on the evolution of the notebook ;-)... 

Cheers,
Emmanuelle

On Wed, Dec 05, 2012 at 08:05:15PM -0500, Matt wrote:
>    Hi Emmanuelle,
>    You may find it useful to get in touch with Software Carpentry
>    ([1]http://software-carpentry.org). We've started using the IPython
>    Notebook extensively in our scientific Python instruction and consider it
>    more or less the best tool for the job:
>    [2]http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/.
>    You can send inquiries to [3]info at software-carpentry.org, we've even got
>    some members in France.
>    Best,
>    Matt
>    On Dec 5, 2012, at 5:24 PM, Emmanuelle Gouillart
>    <[4]emmanuelle.gouillart at nsup.org> wrote:

>              Hello,

>              from next year on in France, Python will be taught as the first
>      programming language for scientific computing in an important fraction
>      of
>      French undergraduate schools (the scientific "preparatory classes" in
>      the
>      French educational system). I'm trying to set up a course on scientific
>      Python for the future French professors that will teach Scientific
>      Python, and I'm wondering whether I should promote the use of the
>      Ipython
>      notebook as the main educational tool for classes, or whether I should
>      stick to more traditional tools.

>              My main concern is about stability. As I see it, a professor
>      could have a set of different notebook files corresponding to the
>      different sessions, and encourage the students to do the same. However,
>      when I have tried to open existing .ipynb files, I've run across an
>      incompatibility between ipython 0.12 and notebooks in version 3 format
>      (error message "Unreadable JSON notebook"). I have not tried whether
>      backward compatibility works, ie whether ipython 0.13 can read version 2
>      format. Is it planned that the notebook format will still change a lot
>      or
>      not? Indeed, I do not expect the undergraduate schools to update ipython
>      very frequently, and it would be quite a problem if a professor cannot
>      grade a student's notebook because it was written on a more recent
>      Ipython... On the other hand, the notebook is really cool and has a lot
>      of advantages for teaching, so I really have mixed feelings... Stability
>      is paramount for earning the trust of professors, since a fraction of
>      them is used to Matlab/Scilab and a bit wary of Python (they will have
>      the choice between Scilab and Python, although Python is recommended by
>      the French "official program"): I cannot afford to take too many risks.
>      What do you think? Any insights on using the notebook for teaching are
>      very welcome.

>              Cheers,
>              Emmanuelle
>      _______________________________________________
>      IPython-dev mailing list
>      [5]IPython-dev at scipy.org
>      http://mail.scipy.org/mailman/listinfo/ipython-dev

> References

>    Visible links
>    1. http://software-carpentry.org/
>    2. http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/
>    3. mailto:info at software-carpentry.org
>    4. mailto:emmanuelle.gouillart at nsup.org
>    5. mailto:IPython-dev at scipy.org

> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Thu Dec  6 06:22:19 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 11:22:19 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121206084510.GA29924@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
Message-ID: <CAOvn4qgHprgBP1QT8cQ7ZyX1+mDcVL58h9OLKiLodisqTO6xAg@mail.gmail.com>

On 6 December 2012 08:45, Emmanuelle Gouillart <
emmanuelle.gouillart at nsup.org> wrote:

> It will be hard to have everybody use 0.13 during the training: 0.12 is
> the version packaged by the latest Ubuntu LTS (12.04) and by the latest
> EPD. I do not expect people to be more bleeding-edge than the latest
> Ubuntu LTS. Fortunately, the training will be in Spring, hopefully
> there will exist a new version of EPD with Ipython 13.
>

If the Ubuntu users are comfortable using a PPA, Julian Taylor has
helpfully packaged 0.13.1 here:
https://launchpad.net/~jtaylor/+archive/ipython

Adding the PPA requires just one command:
sudo apt-add-repository ppa:jtaylor/ipython

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/5e515217/attachment.html>

From jiffyclub at gmail.com  Thu Dec  6 08:19:03 2012
From: jiffyclub at gmail.com (Matt)
Date: Thu, 6 Dec 2012 08:19:03 -0500
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121206084510.GA29924@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
Message-ID: <D699507C-A16D-42B1-A1B2-18A36071493D@gmail.com>

Hi Emmanuelle,

There is some progress on EPD: https://twitter.com/enthought/status/268457051007623169.
It's very frustrating for us too that IPython 0.13 is not in EPD Free. We've been experimenting with Anaconda CE.

Best,
Matt

On Dec 6, 2012, at 3:45 AM, Emmanuelle Gouillart <emmanuelle.gouillart at nsup.org> wrote:

> Hi Thomas and Brian,
> 
> thank you very much for your answers. I'm glad to know that no format
> change is planned for now, and that developers also consider
> forward-compatibility to be important.
> 
> It will be hard to have everybody use 0.13 during the training: 0.12 is
> the version packaged by the latest Ubuntu LTS (12.04) and by the latest
> EPD. I do not expect people to be more bleeding-edge than the latest
> Ubuntu LTS. Fortunately, the training will be in Spring, hopefully
> there will exist a new version of EPD with Ipython 13.
> 
> Thanks again,
> Emmanuelle
> 
> On Wed, Dec 05, 2012 at 04:32:14PM -0800, Brian Granger wrote:
>> Hi,
> 
>> Glad you hear you are thinking about using the notebook in this
>> capacity.  We think it is a great tool for teaching.  You ask good
>> questions about the stability of the notebook format, but I will reply
>> to Thomas as well...
> 
>>> 0.13 can indeed open v2 notebooks, although it only saves in v3, so it's not
>>> practical to use 0.12 and 0.13 together. But we intend to keep compatibility
>>> for much longer now - the v3 format was designed to allow much more
>>> extension without breaking the ability of 0.13 to read it. This came up in
>>> the discussion around standardising the Scipy stack as well. There we said
>>> that if we have to break compatibility again, we will need to have an
>>> overlap period where IPython can read the new format, but still saves in the
>>> old format by default.
> 
>> I don't recall this discussion about the notebook format and am
>> unaware that we made any such promise.  Do you have a link to the
>> discussion?  I neither oppose nor approve of the decision (I haven't
>> thought enough about it) at this point, I am just unaware of it and
>> want to learn more.
> 
>>> Obviously the notebook is still quite young and rapidly developing, but we
>>> see the format break from 0.12 to 0.13 as the exception, not the rule. If
>>> you decide to use it, make sure everyone starts with at least 0.13.
> 
>> Yes.  Right now, the notebook format is quite stable.  But the reason
>> we created notebook format versions in the first place is to enable
>> the version to change.  While we want to avoid incrementing the
>> version, it will happen.  But I want to emphasize that at this point,
>> we have no plans of incrementing the version any time soon.  In fact,
>> I am not aware of any feature being discussed that would require such
>> changing the format.
> 
>> Cheers,
> 
>> Brian
> 
>>> That's great news about the scientific computing curriculum. Good luck with
>>> training more people in Python!
> 
>>> Best wishes,
>>> Thomas
> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/9f4c5505/attachment.html>

From aron.ahmadia at gmail.com  Thu Dec  6 08:23:43 2012
From: aron.ahmadia at gmail.com (Aron Ahmadia)
Date: Thu, 6 Dec 2012 13:23:43 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <D699507C-A16D-42B1-A1B2-18A36071493D@gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
	<D699507C-A16D-42B1-A1B2-18A36071493D@gmail.com>
Message-ID: <90706085-E95B-4539-AEC6-79BB95F661B0@gmail.com>

I have had very good luck with Anaconda so far and it supports 0.13.1 and quite a few other packages out of the box. 

A

Sent from my iPhone

On Dec 6, 2012, at 1:19 PM, Matt <jiffyclub at gmail.com> wrote:

> Hi Emmanuelle,
> 
> There is some progress on EPD: https://twitter.com/enthought/status/268457051007623169.
> It's very frustrating for us too that IPython 0.13 is not in EPD Free. We've been experimenting with Anaconda CE.
> 
> Best,
> Matt
> 
> On Dec 6, 2012, at 3:45 AM, Emmanuelle Gouillart <emmanuelle.gouillart at nsup.org> wrote:
> 
>> Hi Thomas and Brian,
>> 
>> thank you very much for your answers. I'm glad to know that no format
>> change is planned for now, and that developers also consider
>> forward-compatibility to be important.
>> 
>> It will be hard to have everybody use 0.13 during the training: 0.12 is
>> the version packaged by the latest Ubuntu LTS (12.04) and by the latest
>> EPD. I do not expect people to be more bleeding-edge than the latest
>> Ubuntu LTS. Fortunately, the training will be in Spring, hopefully
>> there will exist a new version of EPD with Ipython 13.
>> 
>> Thanks again,
>> Emmanuelle
>> 
>> On Wed, Dec 05, 2012 at 04:32:14PM -0800, Brian Granger wrote:
>>> Hi,
>> 
>>> Glad you hear you are thinking about using the notebook in this
>>> capacity.  We think it is a great tool for teaching.  You ask good
>>> questions about the stability of the notebook format, but I will reply
>>> to Thomas as well...
>> 
>>>> 0.13 can indeed open v2 notebooks, although it only saves in v3, so it's not
>>>> practical to use 0.12 and 0.13 together. But we intend to keep compatibility
>>>> for much longer now - the v3 format was designed to allow much more
>>>> extension without breaking the ability of 0.13 to read it. This came up in
>>>> the discussion around standardising the Scipy stack as well. There we said
>>>> that if we have to break compatibility again, we will need to have an
>>>> overlap period where IPython can read the new format, but still saves in the
>>>> old format by default.
>> 
>>> I don't recall this discussion about the notebook format and am
>>> unaware that we made any such promise.  Do you have a link to the
>>> discussion?  I neither oppose nor approve of the decision (I haven't
>>> thought enough about it) at this point, I am just unaware of it and
>>> want to learn more.
>> 
>>>> Obviously the notebook is still quite young and rapidly developing, but we
>>>> see the format break from 0.12 to 0.13 as the exception, not the rule. If
>>>> you decide to use it, make sure everyone starts with at least 0.13.
>> 
>>> Yes.  Right now, the notebook format is quite stable.  But the reason
>>> we created notebook format versions in the first place is to enable
>>> the version to change.  While we want to avoid incrementing the
>>> version, it will happen.  But I want to emphasize that at this point,
>>> we have no plans of incrementing the version any time soon.  In fact,
>>> I am not aware of any feature being discussed that would require such
>>> changing the format.
>> 
>>> Cheers,
>> 
>>> Brian
>> 
>>>> That's great news about the scientific computing curriculum. Good luck with
>>>> training more people in Python!
>> 
>>>> Best wishes,
>>>> Thomas
>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/35988dd6/attachment.html>

From bussonniermatthias at gmail.com  Thu Dec  6 08:36:16 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 6 Dec 2012 14:36:16 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121206084510.GA29924@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
Message-ID: <2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>

Hi Emanuelle, 

If I remember correctly, EPD did make an exception for IPython and the 0.13 version is already available with free EPD, 
They did this for SciPy 12" I think.

Just just have to explicitly ask for upgrade. 

(got this from ML archive http://mail.scipy.org/pipermail/ipython-user/2012-August/010882.html 
and http://mail.scipy.org/pipermail/ipython-user/2012-August/010883.html ) 

This update restriction in EPD Free was true in the past, but for SciPy
2012 (and still continuing), we made ipython updates available to all EPD
users, free or not, via the commands I mentioned earlier today. This update
capability in EPD Free only applies to ipython, not for pandas or the other
100 or so packages in EPD Basic.

For the record, the current EPD Free release is 7.3-1. The current EPD
Basic release is 7.3-2.

hth,
Jonathan
Also I am working in Institut Curie in Paris, so if you want we can meet.

There are a few stuff that could be done with IPython notebook which are not documented online as we
made only proof of concept at scipy 12". For example Josh Boom ask for the ability to "password protect"
some cell to just give the ipynb file to his student with the embedded solution, and just "publish the password" after a deadline.

Also, just to be clear, 0.12 (v2) had an annoying misconception, so we push people forward to use 0.13+ (v3)
It is still possible to convert between v2 <-> v3 but it just have to be done explicitly from a command line.

Thanks. 
-- 
Matthias

Le 6 d?c. 2012 ? 09:45, Emmanuelle Gouillart a ?crit :

> Hi Thomas and Brian,
> 
> thank you very much for your answers. I'm glad to know that no format
> change is planned for now, and that developers also consider
> forward-compatibility to be important.
> 
> It will be hard to have everybody use 0.13 during the training: 0.12 is
> the version packaged by the latest Ubuntu LTS (12.04) and by the latest
> EPD. I do not expect people to be more bleeding-edge than the latest
> Ubuntu LTS. Fortunately, the training will be in Spring, hopefully
> there will exist a new version of EPD with Ipython 13.
> 
> Thanks again,
> Emmanuelle
> 
> On Wed, Dec 05, 2012 at 04:32:14PM -0800, Brian Granger wrote:
>> Hi,
> 
>> Glad you hear you are thinking about using the notebook in this
>> capacity.  We think it is a great tool for teaching.  You ask good
>> questions about the stability of the notebook format, but I will reply
>> to Thomas as well...
> 
>>> 0.13 can indeed open v2 notebooks, although it only saves in v3, so it's not
>>> practical to use 0.12 and 0.13 together. But we intend to keep compatibility
>>> for much longer now - the v3 format was designed to allow much more
>>> extension without breaking the ability of 0.13 to read it. This came up in
>>> the discussion around standardising the Scipy stack as well. There we said
>>> that if we have to break compatibility again, we will need to have an
>>> overlap period where IPython can read the new format, but still saves in the
>>> old format by default.
> 
>> I don't recall this discussion about the notebook format and am
>> unaware that we made any such promise.  Do you have a link to the
>> discussion?  I neither oppose nor approve of the decision (I haven't
>> thought enough about it) at this point, I am just unaware of it and
>> want to learn more.
> 
>>> Obviously the notebook is still quite young and rapidly developing, but we
>>> see the format break from 0.12 to 0.13 as the exception, not the rule. If
>>> you decide to use it, make sure everyone starts with at least 0.13.
> 
>> Yes.  Right now, the notebook format is quite stable.  But the reason
>> we created notebook format versions in the first place is to enable
>> the version to change.  While we want to avoid incrementing the
>> version, it will happen.  But I want to emphasize that at this point,
>> we have no plans of incrementing the version any time soon.  In fact,
>> I am not aware of any feature being discussed that would require such
>> changing the format.
> 
>> Cheers,
> 
>> Brian
> 
>>> That's great news about the scientific computing curriculum. Good luck with
>>> training more people in Python!
> 
>>> Best wishes,
>>> Thomas
> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/cf8b02c4/attachment.html>

From gael.varoquaux at normalesup.org  Thu Dec  6 08:39:10 2012
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 6 Dec 2012 14:39:10 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
	<2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
Message-ID: <20121206133910.GA2940@phare.normalesup.org>

On Thu, Dec 06, 2012 at 02:36:16PM +0100, Matthias BUSSONNIER wrote:
> Also I am working in Institut Curie in Paris, so if you want we can meet.

Nice! I am in Paris too, and my sister (very active in Python and scipy
communities) is in Paris too, and at Curie too. We should definitely
organize a get-together.

Anybody else want to join?

G


From aron.ahmadia at gmail.com  Thu Dec  6 08:54:32 2012
From: aron.ahmadia at gmail.com (Aron Ahmadia)
Date: Thu, 6 Dec 2012 13:54:32 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121206133910.GA2940@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
	<2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
	<20121206133910.GA2940@phare.normalesup.org>
Message-ID: <6594E5EF-C5A4-40B5-9001-100AF5B1DE79@gmail.com>

I will be in Paris in early January, not sure yet if I would have time to meet. 

A

Sent from my iPhone

On Dec 6, 2012, at 1:39 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> On Thu, Dec 06, 2012 at 02:36:16PM +0100, Matthias BUSSONNIER wrote:
>> Also I am working in Institut Curie in Paris, so if you want we can meet.
> 
> Nice! I am in Paris too, and my sister (very active in Python and scipy
> communities) is in Paris too, and at Curie too. We should definitely
> organize a get-together.
> 
> Anybody else want to join?
> 
> G
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From gael.varoquaux at normalesup.org  Thu Dec  6 08:53:46 2012
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 6 Dec 2012 14:53:46 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <6594E5EF-C5A4-40B5-9001-100AF5B1DE79@gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
	<2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
	<20121206133910.GA2940@phare.normalesup.org>
	<6594E5EF-C5A4-40B5-9001-100AF5B1DE79@gmail.com>
Message-ID: <20121206135346.GB2940@phare.normalesup.org>

On Thu, Dec 06, 2012 at 01:54:32PM +0000, Aron Ahmadia wrote:
> I will be in Paris in early January, not sure yet if I would have time to meet. 

If you have time, let's schedule something.

G


From emmanuelle.gouillart at nsup.org  Thu Dec  6 09:57:15 2012
From: emmanuelle.gouillart at nsup.org (Emmanuelle Gouillart)
Date: Thu, 6 Dec 2012 15:57:15 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
	<2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
Message-ID: <20121206145715.GA2702@phare.normalesup.org>

Hi Matthias,

>    If I remember correctly, EPD did make an exception for IPython and the
>    0.13 version is already available with free EPD,?

OK, I got confused by http://www.enthought.com/products/epdlibraries.php
that mentions Ipython 0.12 as the version shipped by EPD 7.3

>    Also I am working in Institut Curie in Paris, so if you want we can meet.
>    There are a few stuff that could be done with IPython notebook which are
>    not documented online as we
>    made only proof of concept at scipy 12". For example Josh Boom ask for the
>    ability to "password protect"
>    some cell to just give the ipynb file to his student with the embedded
>    solution, and just "publish the password" after a deadline.
>    Also, just to be clear, 0.12 (v2) had an annoying misconception, so we
>    push people forward to use 0.13+ (v3)
>    It is still possible to convert between v2 <-> v3 but it just have to be
>    done explicitly from a command line.
>    --?

It'd be great to meet for a drink soon with Ga?l and Nelle! And maybe
with Aron in January!

Cheers,
Emmanuelle


From wen.g.gong at gmail.com  Thu Dec  6 10:28:40 2012
From: wen.g.gong at gmail.com (W Gong)
Date: Thu, 6 Dec 2012 10:28:40 -0500
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121206090512.GB29924@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
Message-ID: <CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>

ipynb could be made more user-friendly, if new buttons are added to the
tool bar,
e.g. to add an youtube link, the button prompts for url or the code, then
auto insert a cell:

from IPython.display import YouTubeVideo
YouTubeVideo('MTx6ha6fRwo')

ipynb has huge potential, but it is still at early stage

Thanks,
W

On Thu, Dec 6, 2012 at 4:05 AM, Emmanuelle Gouillart <
emmanuelle.gouillart at nsup.org> wrote:

>
> Hi Matt,
>
> thank you for your answer. The article on software carpentry is very
> interesting and I should definitely ask for more details on the Software
> Carpentry mailing-list.
>
> In fact, I have no doubt that I (as the instructor) can use the Ipython
> Notebook in a pleasant and efficient way. The question is whether this is
> the right tool for hundreds of maths and physics teachers, most of whom
> are not very geeky (in France, teachers in the "preparatory classes"
> undergraduate schools are not researchers, they do 100% of teaching). I
> guess I have to take a bet on the evolution of the notebook ;-)...
>
> Cheers,
> Emmanuelle
>
> On Wed, Dec 05, 2012 at 08:05:15PM -0500, Matt wrote:
> >    Hi Emmanuelle,
> >    You may find it useful to get in touch with Software Carpentry
> >    ([1]http://software-carpentry.org). We've started using the IPython
> >    Notebook extensively in our scientific Python instruction and
> consider it
> >    more or less the best tool for the job:
> >    [2]
> http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/
> .
> >    You can send inquiries to [3]info at software-carpentry.org, we've even
> got
> >    some members in France.
> >    Best,
> >    Matt
> >    On Dec 5, 2012, at 5:24 PM, Emmanuelle Gouillart
> >    <[4]emmanuelle.gouillart at nsup.org> wrote:
>
> >              Hello,
>
> >              from next year on in France, Python will be taught as the
> first
> >      programming language for scientific computing in an important
> fraction
> >      of
> >      French undergraduate schools (the scientific "preparatory classes"
> in
> >      the
> >      French educational system). I'm trying to set up a course on
> scientific
> >      Python for the future French professors that will teach Scientific
> >      Python, and I'm wondering whether I should promote the use of the
> >      Ipython
> >      notebook as the main educational tool for classes, or whether I
> should
> >      stick to more traditional tools.
>
> >              My main concern is about stability. As I see it, a professor
> >      could have a set of different notebook files corresponding to the
> >      different sessions, and encourage the students to do the same.
> However,
> >      when I have tried to open existing .ipynb files, I've run across an
> >      incompatibility between ipython 0.12 and notebooks in version 3
> format
> >      (error message "Unreadable JSON notebook"). I have not tried whether
> >      backward compatibility works, ie whether ipython 0.13 can read
> version 2
> >      format. Is it planned that the notebook format will still change a
> lot
> >      or
> >      not? Indeed, I do not expect the undergraduate schools to update
> ipython
> >      very frequently, and it would be quite a problem if a professor
> cannot
> >      grade a student's notebook because it was written on a more recent
> >      Ipython... On the other hand, the notebook is really cool and has a
> lot
> >      of advantages for teaching, so I really have mixed feelings...
> Stability
> >      is paramount for earning the trust of professors, since a fraction
> of
> >      them is used to Matlab/Scilab and a bit wary of Python (they will
> have
> >      the choice between Scilab and Python, although Python is
> recommended by
> >      the French "official program"): I cannot afford to take too many
> risks.
> >      What do you think? Any insights on using the notebook for teaching
> are
> >      very welcome.
>
> >              Cheers,
> >              Emmanuelle
> >      _______________________________________________
> >      IPython-dev mailing list
> >      [5]IPython-dev at scipy.org
> >      http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> > References
>
> >    Visible links
> >    1. http://software-carpentry.org/
> >    2.
> http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/
> >    3. mailto:info at software-carpentry.org
> >    4. mailto:emmanuelle.gouillart at nsup.org
> >    5. mailto:IPython-dev at scipy.org
>
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 

Thanks,

- Wen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/8caa22df/attachment.html>

From bussonniermatthias at gmail.com  Thu Dec  6 10:53:35 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 6 Dec 2012 16:53:35 +0100
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <20121206145715.GA2702@phare.normalesup.org>
References: <20121205222417.GB5894@phare.normalesup.org>
	<CAOvn4qjjc4L+W4Judw4v7WDX3HiPvnUX6AQ1U6UYQ1T479o=2w@mail.gmail.com>
	<CAH4pYpRURfn-4u+YKuwVu4Zw6jR2_hqcpCeWBYTYDLP4LZNCrg@mail.gmail.com>
	<20121206084510.GA29924@phare.normalesup.org>
	<2947A5C2-6613-456C-9835-D8080638A0F0@gmail.com>
	<20121206145715.GA2702@phare.normalesup.org>
Message-ID: <C3558B0C-C639-41C9-8E5F-A267E090906F@gmail.com>


Le 6 d?c. 2012 ? 15:57, Emmanuelle Gouillart a ?crit :

> Hi Matthias,
> 
>>   If I remember correctly, EPD did make an exception for IPython and the
>>   0.13 version is already available with free EPD, 
> 
> OK, I got confused by http://www.enthought.com/products/epdlibraries.php
> that mentions Ipython 0.12 as the version shipped by EPD 7.3

Yes, this is because 0.12 is actually shipped when you install EPD 7.3 and IPython 
is the only package that can be upgraded to 0.13. I don't think there a mentions anywhere 
on EPD website that you can do that.

> 
> It'd be great to meet for a drink soon with Ga?l and Nelle! And maybe
> with Aron in January!
> 

It'll be great !


Le 6 d?c. 2012 ? 16:28, W Gong a ?crit :

> ipynb could be made more user-friendly, if new buttons are added to the tool bar,
> e.g. to add an youtube link, the button prompts for url or the code, then auto insert a cell:
> 
> from IPython.display import YouTubeVideo
> YouTubeVideo('MTx6ha6fRwo')

This is doable in last master for quite some time.
http://elacave.lmdb.eu/~carreau/yui/classes/IPython.customjs.html

This is just not something that will be integrated in the core (IMHO).

-- 
Matthias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/e86630c1/attachment.html>

From erik.tollerud at gmail.com  Thu Dec  6 10:58:00 2012
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Thu, 6 Dec 2012 10:58:00 -0500
Subject: [IPython-dev] qtconsole give heartbeat warning when waking from
	sleep on OS X 10.8
Message-ID: <CAARXUwVrr2Kp7JFN89SfZxvZrRCX4=JpQrzG+Uud94uY+T=+tg@mail.gmail.com>

I've been using the qtconsole for a while now, and I've noticed an
annoying behavior that seems to consistently occur now on the latest
Mac OS X (10.8): When I wake my computer from sleep (usually after a
*long* sleep like overnight), any qtconsole windows pop up the dialog
indicating that "The kernel heartbeat has been inactive for 3.00
seconds. Do you want to restart the kernel?" If I choose "no",
everything works just fine, so the kernel is actually running fine.
Any idea why this is happening?  (Note that it did *not* happen on OS
X 10.7)


From takowl at gmail.com  Thu Dec  6 11:05:46 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 16:05:46 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
Message-ID: <CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>

On 6 December 2012 15:28, W Gong <wen.g.gong at gmail.com> wrote:

> ipynb could be made more user-friendly, if new buttons are added to the
> tool bar,
> e.g. to add an youtube link, the button prompts for url or the code, then
> auto insert a cell:
>
> from IPython.display import YouTubeVideo
> YouTubeVideo('MTx6ha6fRwo')
>

I think the challenge is that there is an enormous wealth of things you can
do in the notebook - far more than we can put into the toolbar, and it's
pretty heavy even for a hierarchical menu (see the magics menu in the Qt
console). Is embedding Youtube videos so important to so many users that it
merits a valuable toolbar button?

What's interesting is how we integrate the notebook as a way to edit and
run code, with the notebook as a rich text presentation system. Embedding a
video needn't go through a Python cell, but at present that's simpler than
putting the relevant HTML into a Markdown cell. From the other direction,
we're now talking about ways to embed variable references into Markdown
cells, and definining a syntax to do that. I get the feeling we're missing
a coherent approach here.

To spark discussion, maybe what we need is something like a 'template
cell', that's in between a client-side Markdown cell and a code cell:
- Edit syntax something like Jinja2 templates, with fields that you can
fill in, loops, if clauses, and so on. We're thinking of using Jinja2
anyway, so let's not invent our own templating language.
- When you execute it (shift-enter), it is sent to the kernel to render the
template.
- Python objects used in fields are rendered according to our display
protocol, i.e. _repr_html_() will be preferred.
- In the interface, the rendered output replaces the editor (like a
Markdown cell) until you double-click to edit it again.

Thoughts?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/b8b11923/attachment.html>

From jason-sage at creativetrax.com  Thu Dec  6 11:27:52 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 06 Dec 2012 10:27:52 -0600
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
Message-ID: <50C0C788.4070406@creativetrax.com>

On 12/6/12 10:05 AM, Thomas Kluyver wrote:
> On 6 December 2012 15:28, W Gong <wen.g.gong at gmail.com
> <mailto:wen.g.gong at gmail.com>> wrote:
>
>     ipynb could be made more user-friendly, if new buttons are added to
>     the tool bar,
>     e.g. to add an youtube link, the button prompts for url or the code,
>     then auto insert a cell:
>
>     from IPython.display import YouTubeVideo
>     YouTubeVideo('MTx6ha6fRwo')
>
>
> I think the challenge is that there is an enormous wealth of things you
> can do in the notebook - far more than we can put into the toolbar, and
> it's pretty heavy even for a hierarchical menu (see the magics menu in
> the Qt console). Is embedding Youtube videos so important to so many
> users that it merits a valuable toolbar button?
>
> What's interesting is how we integrate the notebook as a way to edit and
> run code, with the notebook as a rich text presentation system.
> Embedding a video needn't go through a Python cell, but at present
> that's simpler than putting the relevant HTML into a Markdown cell. From
> the other direction, we're now talking about ways to embed variable
> references into Markdown cells, and definining a syntax to do that. I
> get the feeling we're missing a coherent approach here.
>
> To spark discussion, maybe what we need is something like a 'template
> cell', that's in between a client-side Markdown cell and a code cell:
> - Edit syntax something like Jinja2 templates, with fields that you can
> fill in, loops, if clauses, and so on. We're thinking of using Jinja2
> anyway, so let's not invent our own templating language.
> - When you execute it (shift-enter), it is sent to the kernel to render
> the template.
> - Python objects used in fields are rendered according to our display
> protocol, i.e. _repr_html_() will be preferred.
> - In the interface, the rendered output replaces the editor (like a
> Markdown cell) until you double-click to edit it again.
>
> Thoughts?

Just as a point of reference, Sage has %html cells.  Anything in 
<sage></sage> tags is evaluated and converted to latex and rendered with 
mathjax.  The whole thing is shipped back to the browser and rendered as 
html there.  So:

%html
The <b>value</b> is <sage>integrate(x^2,(x,0,1))</sage>.

would render as html and the integral calculation is replaced with the 
result.

It's not as flexible as what you are imagining, but like I said, it's an 
interesting point of reference.

Thanks,

Jason




From takowl at gmail.com  Thu Dec  6 12:26:45 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 17:26:45 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <50C0C788.4070406@creativetrax.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
	<50C0C788.4070406@creativetrax.com>
Message-ID: <CAOvn4qjfa7sRGtbimH2Pew+BGib0MhLiQt+aJg2-JTBsdibV5Q@mail.gmail.com>

On 6 December 2012 16:27, Jason Grout <jason-sage at creativetrax.com> wrote:

> Just as a point of reference, Sage has %html cells.  Anything in
> <sage></sage> tags is evaluated and converted to latex and rendered with
> mathjax.  The whole thing is shipped back to the browser and rendered as
> html there.  So:
>
> %html
> The <b>value</b> is <sage>integrate(x^2,(x,0,1))</sage>.


Thanks, that's a useful reference. It would be quite easy to write a cell
magic that did something similar in IPython. I envisage something a bit
more integrated, so that when the output is shown, the template code is
hidden. It could be used to create more report-like notebooks.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/cb088bb7/attachment.html>

From asmeurer at gmail.com  Thu Dec  6 14:10:27 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Thu, 6 Dec 2012 12:10:27 -0700
Subject: [IPython-dev] qtconsole give heartbeat warning when waking from
 sleep on OS X 10.8
In-Reply-To: <CAARXUwVrr2Kp7JFN89SfZxvZrRCX4=JpQrzG+Uud94uY+T=+tg@mail.gmail.com>
References: <CAARXUwVrr2Kp7JFN89SfZxvZrRCX4=JpQrzG+Uud94uY+T=+tg@mail.gmail.com>
Message-ID: <8098907704772587165@unknownmsgid>

What kind of computer do you have? In particular, do you have one of
the new MacBook Airs or retina MacBook Pros that support power nap?

Aaron Meurer

On Dec 6, 2012, at 8:58 AM, Erik Tollerud <erik.tollerud at gmail.com> wrote:

> I've been using the qtconsole for a while now, and I've noticed an
> annoying behavior that seems to consistently occur now on the latest
> Mac OS X (10.8): When I wake my computer from sleep (usually after a
> *long* sleep like overnight), any qtconsole windows pop up the dialog
> indicating that "The kernel heartbeat has been inactive for 3.00
> seconds. Do you want to restart the kernel?" If I choose "no",
> everything works just fine, so the kernel is actually running fine.
> Any idea why this is happening?  (Note that it did *not* happen on OS
> X 10.7)
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From jrocher at enthought.com  Thu Dec  6 14:29:54 2012
From: jrocher at enthought.com (Jonathan Rocher)
Date: Thu, 6 Dec 2012 13:29:54 -0600
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOvn4qjfa7sRGtbimH2Pew+BGib0MhLiQt+aJg2-JTBsdibV5Q@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
	<50C0C788.4070406@creativetrax.com>
	<CAOvn4qjfa7sRGtbimH2Pew+BGib0MhLiQt+aJg2-JTBsdibV5Q@mail.gmail.com>
Message-ID: <CAOzk5QcE_44w7+KhNG-pfPZzFmtBNPHCicyUXAz9h1FA2YbX8Q@mail.gmail.com>

(Re-forwarding because previous emails from Jonathan March didn't go
through. Apologies if you received it multiple times)

This is a wonderful opportunity for scientific Python and the IPython
Notebook.

There have been some big changes this year to EPD Free and Academic. Sorry
for not making these more widely known.

The bottom line for this thread is that any EPD Free user can register a
free login and use it to dynamically update any EPD Free package, including
IPython (currently at 0.13.1).

More detail:

1. EPD's user account system has, for some months, provided Free accounts
(not just Free downloads). These accounts allow registered Free users to
update any EPD Free package to the latest repository version, by using
either the enpkg command-line utility or the GUI package manager (now in
beta).

2. Following discussions and polls in SciPy and numfocus lists about a
"base scientific package stack", EPD Free users may now install all
consensus packages, specifically including pandas and sympy, by using enpkg
or the beta GUI package manager.

3. This system now also provides free Academic subscriptions (not just
downloads as in the past), which allow updating any EPD package (not just
Free) to any version in the repository, by using enpkg or the beta GUI
package manager.

For EPD Free accounts: http://www.enthought.com/products/epd_free.php
For EPD Academic accounts: http://www.enthought.com/products/edudownload.php

hth,
Jonathan March



On Thu, Dec 6, 2012 at 11:26 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 6 December 2012 16:27, Jason Grout <jason-sage at creativetrax.com> wrote:
>
>> Just as a point of reference, Sage has %html cells.  Anything in
>> <sage></sage> tags is evaluated and converted to latex and rendered with
>> mathjax.  The whole thing is shipped back to the browser and rendered as
>> html there.  So:
>>
>> %html
>> The <b>value</b> is <sage>integrate(x^2,(x,0,1))</sage>.
>
>
> Thanks, that's a useful reference. It would be quite easy to write a cell
> magic that did something similar in IPython. I envisage something a bit
> more integrated, so that when the output is shown, the template code is
> hidden. It could be used to create more report-like notebooks.
>
> Best wishes,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Jonathan Rocher, PhD
Scientific software developer
Enthought, Inc.
jrocher at enthought.com
1-512-536-1057
http://www.enthought.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/8a9fe621/attachment.html>

From wen.g.gong at gmail.com  Thu Dec  6 14:41:40 2012
From: wen.g.gong at gmail.com (W Gong)
Date: Thu, 6 Dec 2012 14:41:40 -0500
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
Message-ID: <CAJzFRJHNA0ULRcQUx5EAZ-9bG6j8AGiFvWgm8mmQ1CZapBxuLQ@mail.gmail.com>

I am not suggesting that the core dev team to do everything user asks for,
but to illustrate how to balance needs of advanced vs novice users.
To me, ipynb is a killer app for learning/teaching/researching users.
If one can lower the entry-barrier, it will encourage user's adoption.

On Thu, Dec 6, 2012 at 11:05 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 6 December 2012 15:28, W Gong <wen.g.gong at gmail.com> wrote:
>
>> ipynb could be made more user-friendly, if new buttons are added to the
>> tool bar,
>> e.g. to add an youtube link, the button prompts for url or the code, then
>> auto insert a cell:
>>
>> from IPython.display import YouTubeVideo
>> YouTubeVideo('MTx6ha6fRwo')
>>
>
> I think the challenge is that there is an enormous wealth of things you
> can do in the notebook - far more than we can put into the toolbar, and
> it's pretty heavy even for a hierarchical menu (see the magics menu in the
> Qt console). Is embedding Youtube videos so important to so many users that
> it merits a valuable toolbar button?
>
> What's interesting is how we integrate the notebook as a way to edit and
> run code, with the notebook as a rich text presentation system. Embedding a
> video needn't go through a Python cell, but at present that's simpler than
> putting the relevant HTML into a Markdown cell. From the other direction,
> we're now talking about ways to embed variable references into Markdown
> cells, and definining a syntax to do that. I get the feeling we're missing
> a coherent approach here.
>
> To spark discussion, maybe what we need is something like a 'template
> cell', that's in between a client-side Markdown cell and a code cell:
> - Edit syntax something like Jinja2 templates, with fields that you can
> fill in, loops, if clauses, and so on. We're thinking of using Jinja2
> anyway, so let's not invent our own templating language.
> - When you execute it (shift-enter), it is sent to the kernel to render
> the template.
> - Python objects used in fields are rendered according to our display
> protocol, i.e. _repr_html_() will be preferred.
> - In the interface, the rendered output replaces the editor (like a
> Markdown cell) until you double-click to edit it again.
>
> Thoughts?
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 

Thanks,

- Wen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/a3141b48/attachment.html>

From takowl at gmail.com  Thu Dec  6 16:12:40 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 21:12:40 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAJzFRJHNA0ULRcQUx5EAZ-9bG6j8AGiFvWgm8mmQ1CZapBxuLQ@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
	<CAJzFRJHNA0ULRcQUx5EAZ-9bG6j8AGiFvWgm8mmQ1CZapBxuLQ@mail.gmail.com>
Message-ID: <CAOvn4qjM3+h_z=ThsZPAbtsj9L15CiX96qF3XsqnSGwByn+wag@mail.gmail.com>

On 6 December 2012 19:41, W Gong <wen.g.gong at gmail.com> wrote:

> I am not suggesting that the core dev team to do everything user asks for,
> but to illustrate how to balance needs of advanced vs novice users.
> To me, ipynb is a killer app for learning/teaching/researching users.
> If one can lower the entry-barrier, it will encourage user's adoption.
>

Hi Wen,

It's not just a question of what the core dev team can do - I assume
toolbar buttons can be added quite easily. I'm thinking about the
interface: is adding a Youtube video an important enough task to take up
precious UI space? I doubt that it is.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/cdd5e68e/attachment.html>

From takowl at gmail.com  Thu Dec  6 16:13:48 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 21:13:48 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOzk5QcE_44w7+KhNG-pfPZzFmtBNPHCicyUXAz9h1FA2YbX8Q@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
	<50C0C788.4070406@creativetrax.com>
	<CAOvn4qjfa7sRGtbimH2Pew+BGib0MhLiQt+aJg2-JTBsdibV5Q@mail.gmail.com>
	<CAOzk5QcE_44w7+KhNG-pfPZzFmtBNPHCicyUXAz9h1FA2YbX8Q@mail.gmail.com>
Message-ID: <CAOvn4qhpWfTV47+sF51feHMvvOCScB8meT4i3JbZXK=Jix2oWA@mail.gmail.com>

On 6 December 2012 19:29, Jonathan Rocher <jrocher at enthought.com> wrote:

> 2. Following discussions and polls in SciPy and numfocus lists about a
> "base scientific package stack", EPD Free users may now install all
> consensus packages, specifically including pandas and sympy, by using enpkg
> or the beta GUI package manager.
>

Thanks Jonathan (both Jonathans). Will pandas & sympy be included in the
main installer in a future version of EPD Free?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/e1b1a868/attachment.html>

From jrocher at enthought.com  Thu Dec  6 16:20:40 2012
From: jrocher at enthought.com (Jonathan Rocher)
Date: Thu, 6 Dec 2012 15:20:40 -0600
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOvn4qhpWfTV47+sF51feHMvvOCScB8meT4i3JbZXK=Jix2oWA@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
	<50C0C788.4070406@creativetrax.com>
	<CAOvn4qjfa7sRGtbimH2Pew+BGib0MhLiQt+aJg2-JTBsdibV5Q@mail.gmail.com>
	<CAOzk5QcE_44w7+KhNG-pfPZzFmtBNPHCicyUXAz9h1FA2YbX8Q@mail.gmail.com>
	<CAOvn4qhpWfTV47+sF51feHMvvOCScB8meT4i3JbZXK=Jix2oWA@mail.gmail.com>
Message-ID: <CAOzk5QeKyhWewWDce13LRWoq5-ScJPV4AUH10EH9R9H1ppaDgw@mail.gmail.com>

Yes they will both be included in the next version of the EPDFree installer
(and will be updatable with registration like any other package).

Jonathans


On Thu, Dec 6, 2012 at 3:13 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 6 December 2012 19:29, Jonathan Rocher <jrocher at enthought.com> wrote:
>
>> 2. Following discussions and polls in SciPy and numfocus lists about a
>> "base scientific package stack", EPD Free users may now install all
>> consensus packages, specifically including pandas and sympy, by using enpkg
>> or the beta GUI package manager.
>>
>
> Thanks Jonathan (both Jonathans). Will pandas & sympy be included in the
> main installer in a future version of EPD Free?
>
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Jonathan Rocher, PhD
Scientific software developer
Enthought, Inc.
jrocher at enthought.com
1-512-536-1057
http://www.enthought.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/b092edd0/attachment.html>

From takowl at gmail.com  Thu Dec  6 16:37:43 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Dec 2012 21:37:43 +0000
Subject: [IPython-dev] using the notebook for teaching?
In-Reply-To: <CAOzk5QeKyhWewWDce13LRWoq5-ScJPV4AUH10EH9R9H1ppaDgw@mail.gmail.com>
References: <20121205222417.GB5894@phare.normalesup.org>
	<81BB3ACE-DE52-46E2-B2FE-F65E347531B2@gmail.com>
	<20121206090512.GB29924@phare.normalesup.org>
	<CAJzFRJHjpnXpM=MQsRyadqQdWJCF=pkcGV3jTcSiw9Peus9utw@mail.gmail.com>
	<CAOvn4qjUuHvo-tCj+PB11xMavFqDARBipwNivQRuENt_+j8MZA@mail.gmail.com>
	<50C0C788.4070406@creativetrax.com>
	<CAOvn4qjfa7sRGtbimH2Pew+BGib0MhLiQt+aJg2-JTBsdibV5Q@mail.gmail.com>
	<CAOzk5QcE_44w7+KhNG-pfPZzFmtBNPHCicyUXAz9h1FA2YbX8Q@mail.gmail.com>
	<CAOvn4qhpWfTV47+sF51feHMvvOCScB8meT4i3JbZXK=Jix2oWA@mail.gmail.com>
	<CAOzk5QeKyhWewWDce13LRWoq5-ScJPV4AUH10EH9R9H1ppaDgw@mail.gmail.com>
Message-ID: <CAOvn4qgg+w_Qc1QXt9W20QZvnm-Z5iD0sFnhhYpd=2Q+oTbh8g@mail.gmail.com>

On 6 December 2012 21:20, Jonathan Rocher <jrocher at enthought.com> wrote:

> Yes they will both be included in the next version of the EPDFree
> installer (and will be updatable with registration like any other package).


Excellent! Thanks again.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121206/b5015b4a/attachment.html>

From ellisonbg at gmail.com  Fri Dec  7 01:27:47 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 6 Dec 2012 22:27:47 -0800
Subject: [IPython-dev] IPython nosetests extension
In-Reply-To: <CAKrk_d8wL93isdsZQg1ENfatESncmZina2=wEn1b=xbdVXgyYw@mail.gmail.com>
References: <CAKrk_d_081PjEcCqiFTUQPM=uLQ+f3aOF=GWQhPdczScnmhTGw@mail.gmail.com>
	<CAH4pYpQRBXCNuyndEcZosmrG-4jgXUK4bU+MFK0apNEV_2uhug@mail.gmail.com>
	<CAKrk_d8wL93isdsZQg1ENfatESncmZina2=wEn1b=xbdVXgyYw@mail.gmail.com>
Message-ID: <CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>

>> Hi, this is really exciting and will be a huge improvement for using
>> the notebook for real work.  I will try to look a bit more at this
>> over the next week (in the middle of finals right now).  But a few
>> comments:
>>
>> * Why exactly do you need cell ids?
>
>
> I've enhanced the test failure stack traces such that the function's
> "filename" (like <ipython-input-DD-HHHHHHHHHHHH>) links to the cell that
> defines that code. That makes it easy to click close to the definition of a
> failing test or function. Even better would be linking directly to the line
> of code in question? :)  Unfortunately the current implementation has a
> visual bug when linking to an anchor: the top IPython bar shifts up about
> 0.3em (it shifts back if you hit the empty fragment, "#").

Ahh, this is a nice feature.  Can you describe a bit more about how
you track this information as it flows though the code.  Do the cell
ids get sent to the kernel?  What does the kernet do with them?

>>
>> * We are soon going to completely remove the ability to publish
>> javascript from Python.  There are two many horrific security
>> problems.  The replacement abstraction is "javascript plugins" that
>> are under review here:
>>
>> https://github.com/ipython/ipython/pull/2518
>>
>> There is more work to do on this, but this will give you an idea of
>> where we are headed.  Please ask questions about the plugins on that
>> PR.
>
>
> That makes a lot of sense. From a quick reading of it, an extension would be
> able to register JS to be loaded onto the page, and the publish_json method
> would allow the payload to be handed to one of those custom functions.
> There's no reason I can't port this extension to that style once that lands.

Yep it shouldn't be too difficult.

> I do have a few questions up-front:
> 1. Is there a better way than peeking at sys.displayhook to determine what
> kind of output to produce?

I am not quite sure what you mean and how you "peek" and
sys.displayhook for this.

> 2. I'd really like to have cell anchor generation happen always. Seems like
> it would be useful to provide intra-notebook links to cells. I'm not sure
> how one would manage those links as the execution order evolves, though. For
> this extension, that migration is actually a good thing, as the anchor
> points to the code that was actually run even if it's no longer visible.

I am trying to understand what the usage cases are.  Would these links
be typed in by hand by a user?  If so, we would want the ids to have a
human readable form.  If they are always generated by code they could
be ugly.

The other thing we have to keep in mind is that we can't ever couple
the kernel to the notebook frontend.  That is, the kernel can't ever
know about these cell ids in any formal way.

> 3. I'd love to have a consistent interface to stream something like stdout
> character-wise instead of line-wise. Right now it's hacked to hook stdout
> directly on the console, and with even more horrible jQuery machinations on
> the notebook side.

Can't you call flush to do this?  Can you describe this a bit more.
Again, not following how all of this is used.  It sounds messy...

Cheers,

Brian

> Thanks!
>
>
>> Cheers,
>>
>> Brian
>>
>> On Tue, Nov 27, 2012 at 11:49 AM, Taavi Burns <taavi.burns at gmail.com>
>> wrote:
>> > https://github.com/taavi/ipython_nose
>> >
>> > I've gotten the %nose magic Greg Ward and I worked on at the PyCon
>> > Canada
>> > sprints to a reasonable point. I'm not yet convinced that it's ready for
>> > integration into IPython (I think I'd like to get IPython to add id
>> > attributes on the cells so I don't have to add them myself with JS), but
>> > it
>> > should be good enough for some real-world testing.
>> >
>> > It's pip-installable in development mode:
>> > . YOUR/VIRTUALENV/activate
>> > git clone https://github.com/taavi/ipython_nose.git
>> > cd ipython_nose
>> > pip install -e .
>> >
>> > I might not bother putting it on PyPI if it's as likely to get included
>> > in
>> > the base IPython as Fernando was suggesting. ;)
>> >
>> > I'd appreciate any feedback on features and bugs. Feel free to lodge
>> > them in
>> > the github tracker. Pull requests are also welcome! Please keep in mind
>> > that
>> > the end goal is to integrate it into IPython proper, so some work might
>> > be
>> > deferred until then.
>> >
>> > Thanks!
>> >
>> > --
>> > taa
>> > /*eof*/
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> taa
> /*eof*/
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From aron at ahmadia.net  Fri Dec  7 05:40:37 2012
From: aron at ahmadia.net (Aron Ahmadia)
Date: Fri, 7 Dec 2012 10:40:37 +0000
Subject: [IPython-dev] IPython nosetests extension
In-Reply-To: <CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>
References: <CAKrk_d_081PjEcCqiFTUQPM=uLQ+f3aOF=GWQhPdczScnmhTGw@mail.gmail.com>
	<CAH4pYpQRBXCNuyndEcZosmrG-4jgXUK4bU+MFK0apNEV_2uhug@mail.gmail.com>
	<CAKrk_d8wL93isdsZQg1ENfatESncmZina2=wEn1b=xbdVXgyYw@mail.gmail.com>
	<CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>
Message-ID: <CAPhiW4iiSO_yr-BCvunb0oxkPowp9qrk-2bXTOd_NJ5omD+xEQ@mail.gmail.com>

Just wanted to chip in and say this idea is very cool.  I don't have any
good ideas/thoughts on the implementation, but I am happy to alpha test
when you have a sample notebook ready to try out.


On Fri, Dec 7, 2012 at 6:27 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> >> Hi, this is really exciting and will be a huge improvement for using
> >> the notebook for real work.  I will try to look a bit more at this
> >> over the next week (in the middle of finals right now).  But a few
> >> comments:
> >>
> >> * Why exactly do you need cell ids?
> >
> >
> > I've enhanced the test failure stack traces such that the function's
> > "filename" (like <ipython-input-DD-HHHHHHHHHHHH>) links to the cell that
> > defines that code. That makes it easy to click close to the definition
> of a
> > failing test or function. Even better would be linking directly to the
> line
> > of code in question? :)  Unfortunately the current implementation has a
> > visual bug when linking to an anchor: the top IPython bar shifts up about
> > 0.3em (it shifts back if you hit the empty fragment, "#").
>
> Ahh, this is a nice feature.  Can you describe a bit more about how
> you track this information as it flows though the code.  Do the cell
> ids get sent to the kernel?  What does the kernet do with them?
>
> >>
> >> * We are soon going to completely remove the ability to publish
> >> javascript from Python.  There are two many horrific security
> >> problems.  The replacement abstraction is "javascript plugins" that
> >> are under review here:
> >>
> >> https://github.com/ipython/ipython/pull/2518
> >>
> >> There is more work to do on this, but this will give you an idea of
> >> where we are headed.  Please ask questions about the plugins on that
> >> PR.
> >
> >
> > That makes a lot of sense. From a quick reading of it, an extension
> would be
> > able to register JS to be loaded onto the page, and the publish_json
> method
> > would allow the payload to be handed to one of those custom functions.
> > There's no reason I can't port this extension to that style once that
> lands.
>
> Yep it shouldn't be too difficult.
>
> > I do have a few questions up-front:
> > 1. Is there a better way than peeking at sys.displayhook to determine
> what
> > kind of output to produce?
>
> I am not quite sure what you mean and how you "peek" and
> sys.displayhook for this.
>
> > 2. I'd really like to have cell anchor generation happen always. Seems
> like
> > it would be useful to provide intra-notebook links to cells. I'm not sure
> > how one would manage those links as the execution order evolves, though.
> For
> > this extension, that migration is actually a good thing, as the anchor
> > points to the code that was actually run even if it's no longer visible.
>
> I am trying to understand what the usage cases are.  Would these links
> be typed in by hand by a user?  If so, we would want the ids to have a
> human readable form.  If they are always generated by code they could
> be ugly.
>
> The other thing we have to keep in mind is that we can't ever couple
> the kernel to the notebook frontend.  That is, the kernel can't ever
> know about these cell ids in any formal way.
>
> > 3. I'd love to have a consistent interface to stream something like
> stdout
> > character-wise instead of line-wise. Right now it's hacked to hook stdout
> > directly on the console, and with even more horrible jQuery machinations
> on
> > the notebook side.
>
> Can't you call flush to do this?  Can you describe this a bit more.
> Again, not following how all of this is used.  It sounds messy...
>
> Cheers,
>
> Brian
>
> > Thanks!
> >
> >
> >> Cheers,
> >>
> >> Brian
> >>
> >> On Tue, Nov 27, 2012 at 11:49 AM, Taavi Burns <taavi.burns at gmail.com>
> >> wrote:
> >> > https://github.com/taavi/ipython_nose
> >> >
> >> > I've gotten the %nose magic Greg Ward and I worked on at the PyCon
> >> > Canada
> >> > sprints to a reasonable point. I'm not yet convinced that it's ready
> for
> >> > integration into IPython (I think I'd like to get IPython to add id
> >> > attributes on the cells so I don't have to add them myself with JS),
> but
> >> > it
> >> > should be good enough for some real-world testing.
> >> >
> >> > It's pip-installable in development mode:
> >> > . YOUR/VIRTUALENV/activate
> >> > git clone https://github.com/taavi/ipython_nose.git
> >> > cd ipython_nose
> >> > pip install -e .
> >> >
> >> > I might not bother putting it on PyPI if it's as likely to get
> included
> >> > in
> >> > the base IPython as Fernando was suggesting. ;)
> >> >
> >> > I'd appreciate any feedback on features and bugs. Feel free to lodge
> >> > them in
> >> > the github tracker. Pull requests are also welcome! Please keep in
> mind
> >> > that
> >> > the end goal is to integrate it into IPython proper, so some work
> might
> >> > be
> >> > deferred until then.
> >> >
> >> > Thanks!
> >> >
> >> > --
> >> > taa
> >> > /*eof*/
> >> >
> >> > _______________________________________________
> >> > IPython-dev mailing list
> >> > IPython-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >
> >>
> >>
> >>
> >> --
> >> Brian E. Granger
> >> Cal Poly State University, San Luis Obispo
> >> bgranger at calpoly.edu and ellisonbg at gmail.com
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> >
> > --
> > taa
> > /*eof*/
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121207/d2f14a1a/attachment.html>

From aka.tkf at gmail.com  Fri Dec  7 06:00:25 2012
From: aka.tkf at gmail.com (Takafumi Arakaki)
Date: Fri, 7 Dec 2012 12:00:25 +0100
Subject: [IPython-dev] Enhancement proposal for kernel messaging protocol
In-Reply-To: <CAH4pYpTcRQtqJrS-wh4=1W-SzwmPcvOwBkzOXQcLpK8LW7VWJg@mail.gmail.com>
References: <CAPM+gBTCkTQeuWbptUED0H1tqS6O+WNX=9dX-qC61EcfhVsHGA@mail.gmail.com>
	<CAH4pYpTcRQtqJrS-wh4=1W-SzwmPcvOwBkzOXQcLpK8LW7VWJg@mail.gmail.com>
Message-ID: <CAPM+gBRMV45C9Hr+pbgJkzjznyYD-vjRB5dDzOud5HkV3Ma82A@mail.gmail.com>

Thanks for the comment.  I opened a pull request for protocol
versioning in github sometime ago:
https://github.com/ipython/ipython/pull/2649

For history_request enhancement, I already made the patches but
I will wait for the relevant PR to be merged:
https://github.com/ipython/ipython/pull/2609


On Wed, Nov 28, 2012 at 4:35 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>> I wrote a PR for adding new option to messaging protocol [1]_.
>> Bussonnier Matthias and Thomas Kluyver suggested to discuss about
>> possible extensions for current messaging protocol, and then make an
>> IPEP before working on these extensions.  So, let me start the
>> discussion here.
>
> Great, thanks for getting this started.
>
>>
>> 1. Protocol introspection.
>>
>> Currently, messaging protocol has no introspection mechanism.  It is
>> not a problem when the messaging protocol is used only for internal
>> communication.  However, there are various software using IPython via
>> messaging protocol.  Also, IPython version of the kernel and client
>> can be different, for example when used over ssh connection.
>> Therefore, we need some introspection mechanism to inform messaging
>> protocol supported by connecting kernel.
>
> I agree with this assessment.
>
>> One way to inform supported messaging protocol is to add version
>> number to the messaging protocol.  There are several options for
>> version numbering:
>>
>> a. Protocol version = IPython version.
>>
>>    This may be the simplest way to do it.  But in development version,
>>    supported protocol will be unclear.
>
> The problem with this is that the message protocol is quite stable
> over the ipython versions.  This is why the notebook format has its
> own version numbers.
>
>> b. Semantic Versioning [2]_.
>>
>>    This is some what close to how notebook version works (right?).
>>    Simple explanation: It takes the form of `x.y.z`.  Patch version
>>    `z` is incremented for bug fix.  Minor version `y` is incremented
>>    for backward compatible changes.  Major version `x` is incremented
>>    for backwards incompatible changes.
>>
>>    For the messaging protocol, the next version number will be 1.1.0
>>    if we think the current protocol is public.  It will be 0.2.0 if we
>>    think the current protocol is still in development.
>>
>> There are several ways to let client know the version number.
>
> I would use the exact same convention that we do in the notebook format.  X.Y:
>
> X = major versions, incompatible
> Y = minor versions, compatible
>
>> c. Add `version_request` / `version_reply` messaging protocol.
>
> Yes, this is what I would prefer.
>
>> d. Add `version` slot to the message format.  It can be in
>>    the top level, `header` or `metadata`.
>> e. Send version number during "hand shake" (but there is no
>>    such stage for the current messaging protocol, I suppose?).
>>
>>
>> 2. New optional `unique` key to `history_request`.
>>
>> I propose [1]_ to add this new boolean key `unique` to
>> `history_request` [3]_.  When this key is specified and true,
>> `history_reply` contains only entries of unique input.  This is useful
>> for searching history using `hist_access_type: 'search'`.  This
>> operation can be done in client side.  However, sending large number
>> of entries over messaging protocol is time consuming.  One case this
>> `unique` key can be critical is when using `history_request` in an
>> interactive UI to search IPython history in as-you-type manner
>> ("QuickSilver style"; for example, my Emacs client EIN [4]_ has such
>> UI).  For such kind of UI speed is critical and it is preferred to be
>> done in the kernel side.
>>
>> (As you can see, this is the reason why I made the PR in the first
>> place!)
>
> Yes, I can see that this would be a good thing to have.
>
>>
>> 3. Messaging protocol to pass structured data to kernel.
>>
>> Currently it is not possible to pass structured data such as dict from
>> client to kernel.  You can generate Python code and use
>> `execute_request`, but it is difficult to do this properly in
>> non-Python client such as Javascript client.  Note that it is
>> already possible to pass structured data to client from kernel
>> by using JSON repr.
>>
>> (I think I saw some argument related to this issue tracker, but
>> I can't find it now.  Please post the link if you know.)
>>
>> There are several ways to solve this problem:
>>
>> a. Add RPC-like message `call_request` / `call_reply`.  Client can
>>    execute a short code to register "methods" in kernel to use this
>>    RPC.  For example `__import__('mylib.ipyutils').register()`.
>>
>> b. Add `override_variables` slot to `execute_request`.  This slot
>>    takes a dict and the value in the dict will override the value
>>    correspond to its key.  For example, client can execute `func(x)`
>>    with `override_variables: {'x': 1}`.
>>
>>    The problem is that it contaminates user's namespace.  To solve this
>>    problem, we can add another `namespace` slot to `execute_request`.
>>    This way, client can import its supporting library without
>>    contaminating user's namespace and execute its functions using
>>    `override_variables`.  For example, client can issue `execute_request`
>>    like this::
>>
>>      {'code' "import mylib; mylib.ipyutils.func(x)",
>>       'namespace': 'mylib',
>>       'override_variables': {'x': 1}}
>>
>>    Merit of this approach comparing to the other one (`call_request`)
>>    is that `namespace` can be used for another purpose.  For example,
>>    notebook client can have separated namespace for each worksheet,
>>    to avoid namespace collision.  Sharing data between namespaces can
>>    be easily done by using singleton object.  Other possible application
>>    is editor integration.  For example, you have associate one namespace
>>    to each file (= python module) opened in the editor.
>
> Even though we will eventually need this, I don't think we are ready
> to tackle it yet.  This will happen as part of the work on interactive
> widgets that will hopefully get started in Jan.
>
> But, for the first two of these enhancements, I think they sound great
> and don't need an IPEP.  A separate PR for each would be great :)
>
> Cheers,
>
> Brian
>
>> .. [1] https://github.com/ipython/ipython/pull/2609
>> .. [2] http://semver.org/
>> .. [3] http://ipython.org/ipython-doc/dev/development/messaging.html#history
>> .. [4] http://github.com/tkf/emacs-ipython-notebook
>>
>> ---
>> Takafumi Arakaki
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From massimodisasha at gmail.com  Sun Dec  9 01:26:23 2012
From: massimodisasha at gmail.com (epi)
Date: Sun, 9 Dec 2012 01:26:23 -0500
Subject: [IPython-dev] custom JS and IPython.notebook.kernel usage (svn
	example)
In-Reply-To: <4784D0DA-F8A4-45DC-9B0B-06A7652CC2DF@gmail.com>
References: <315615DE-AF96-45C5-9ABE-221217B1F43A@gmail.com>
	<4784D0DA-F8A4-45DC-9B0B-06A7652CC2DF@gmail.com>
Message-ID: <FF011CE5-C95E-4B73-B5A1-4D695D79D522@gmail.com>

> Hi there !
> 
> You did already a pretty decent job !
> 
> we are starting to document javascript you can see what it looks like here :
> http://elacave.lmdb.eu/~carreau/yui/
> And if you have more do cot add we'll be happy to take your pull request.
> 

Thanks a lot Matthias,


i'm trying to go ahead,  but with some problems

i should go ahead by little steps .. when i'll be able to do something that make sense i'll be more than happy to start a PR

so for now i'll try to work on a "svn info" button as test case (no input required and no needs to check the notebook name)

what it should do :

1)  run a system call that print some text as log
2)  parse the log in a dict
3) have the dict printed in a new cell 


i'm aware that's my lack of JS knowledge, below some questions to better understand what you suggest me :

Il giorno 05/dic/2012, alle ore 10:23, Matthias BUSSONNIER <bussonniermatthias at gmail.com> ha scritto:

>> 
>> - detect the name  for the "notebook" (filename.ipnb)
> 
> I'm not sure this can be easily accessible, but it could be a request for enhancement,
> what you can do is : 
> IPython.notebook.notebook_name
> To lowercasse, and space to _ then add ipynb.


i didn't understand this steps :/


> 
>> - clean all the output of the "notebook"
> 
> this can be done progrmatically on server side with a custom script on the ipynb file.

i saw the examples on the wiki page, that's great i can follow the example "remove_outputs" but this still require to low the notebook name

> 
>> 
>> - save the "notebook"
> 
> IPython.notebook.save_notebook()
> 
>> 
>> - commit the saved file (here should prompt a js text input box where to add the commit notes, but i can live without it as a start)
> 
> it is doable, just a question of time to code it.
> 
>> 
>> 
>> 		IPython.toolbar.add_buttons_group([
>> 		    {
>> 		         'label'   : 'Save & Commit Notebook',
>> 		         'icon'    : 'ui-icon-circle-arrow-n', 
>> 		         'callback': function(){IPython.notebook.kernel.execute('detect the notebook name'),
>> 				                        IPython.notebook.kernel.execute('clean notebook'),
>> 							IPython.notebook.kernel.execute('save notebook'),
>> 							IPython.notebook.kernel.execute('svn commit notebook')}
>> 		    }
>> 		    ]);
>> 
>> 
>> i need an help to learn how to use "IPython.notebook.kernel.execute" from inside a JS code
>> i was looking in `docs/examples/widgets/directview` but i got lost :(
> 
> 
> So kernel.execute take callbacks, which get the data back, it's a little tricky.
> 
> let's do it for git : 
> 
> IPython.notebook.kernel.execute(
> 	'a=!git status', 
> 		{'execute_reply': function(data){console.log('data:',data)}},
> 	 	{'user_variables':['a']}
>     )
> 
> ( Note for later, to do this properly, it should be done with user_expression, but there is an issue, i don't now why)
> 
> function(data){console.log('data:',data)}
> 
> will be called with the response as parameter, 
> here login the data to the js console.
> 
> Data will be an object with a few parameters ( i'll let you explore), the one that interests us is user_variable ( we requested 'a' above) 
> 
> which itself as a 'a' key.
> 
> data.user_variables.a is the following strig :
> "['# On branch master', '# Changes not staged for commit:', '#   (use "git add <file>..." to update what will be committed)', '#   (use "git checkout -- <file>..." to discard?.."
> (using user_expression would prevnt polluting the kernel with the 'a' variable but it raises.
> then you can do whatever you wish with the variable in the callback.


i changed the code of the svninfo button to :


IPython.toolbar.add_buttons_group([
	{
		'label'   : 'SVN Info',
		'icon'    : 'ui-icon-info', 
		'callback': function(){IPython.notebook.kernel.execute('a = !svn info'), 
		                       IPython.notebook.kernel.execute('svninfo = {i.split(": ")[0]:i.split(": ")[1] for i in a[:-1]}',
							                                          {'execute_reply': function(data){console.log('data:',data)}},
												  {'user_variables':['svninfo']})
							  }
	}
]);


i'm not yet able to print out the dict in a new cell ? 

the svninfo dictionary is available in the notebook's code cell (after i press the info button), 
so the IPython.notebook.kernel.execute works as aspected and the 

svninfo is also available in the JS console.

but trying to add the svninfo in the "last" cell adding a line in the button like :

#
?
{'user_variables':['svninfo']}),
IPython.notebook.get_cell(-1).set_text(svninfo)
#

but i have errors .. i guess i should not pass the python dict to set_text()

Uncaught ReferenceError: svninfo is not defined 

i tried to give the JS object :

#
?
{'user_variables':['svninfo']}),
IPython.notebook.get_cell(-1).set_text(data.user_variables.svninfo)
#

but it gave me 


Uncaught ReferenceError: data is not defined custom.js:32
IPython.toolbar.add_buttons_group.callbackcustom.js:32
f.event.dispatchjquery-1.7.1.min.js:3
h.handle.ijquery-1.7.1.min.js:3
data: 
Object {status: "ok", execution_count: 0, user_variables: Object, payload: Array[0], user_expressions: Object}
 custom.js:30




> 
> see 
> http://wiki.ipython.org/Notebook_feedback
> to strip notebook programatically ( in the gist at the end of the page) 
> 
> 
>> 
>> have you any hints on how can i use the ipython.notebook.kernel  in order to :
>> 
>> - print out some text in a new notebook cell 
> 
> IPython.notebook.get_cell(i).set_text(?)
> 
> IPython.notebook.select(n)
> IPython.notebook.insert_cell_above()
> ?
> 
> should be of help.


i can print a string (not yet a python object) in the last cell using something like :

IPython.notebook.get_cell(-1).set_text('?some text?.')

but i don't know how to "add" a new cell,

let's say i want add a cell at the end, i should select the n=-1 with :

> IPython.notebook.select(n)

then :

> IPython.notebook.insert_cell_above()

and at the end :


IPython.notebook.get_cell(-1).set_text('?some text?.')

but the cell is not added


> 
>> - detect the name for the active notebook
>> - clear all the output for the active notebook
>> 
>> 
>> Thanks a lot for you help,
> 
> click on button -> callback a
> 
> callback a fetch info on the kernel trigger callback b on the response
> 
> callback b -> show a dialog to ask for commit message
> 
> click on ok trigger the correct information to be send on the kernel to commit.
> 

does this means i can have more "callback"  executed when the same "button" is triggered ?


> 
> Does it help ? 
> We probably want to do something like that in a more generic way embedded into IPython notebook. 
> 
> We'll be happy to get your feedback on that !
> 
> -- 
> Matthias

 i'm a bit slow in learning how the system works (my fault)
it is helping me a lot!

Thanks!

--MAssimo.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121209/80852306/attachment.html>

From bussonniermatthias at gmail.com  Sun Dec  9 06:44:30 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 9 Dec 2012 12:44:30 +0100
Subject: [IPython-dev] custom JS and IPython.notebook.kernel usage (svn
	example)
In-Reply-To: <FF011CE5-C95E-4B73-B5A1-4D695D79D522@gmail.com>
References: <315615DE-AF96-45C5-9ABE-221217B1F43A@gmail.com>
	<4784D0DA-F8A4-45DC-9B0B-06A7652CC2DF@gmail.com>
	<FF011CE5-C95E-4B73-B5A1-4D695D79D522@gmail.com>
Message-ID: <499BDE3F-DF84-466A-B398-E2ACE32961BA@gmail.com>

Le 9 d?c. 2012 ? 07:26, epi a ?crit :
> 
> Thanks a lot Matthias,
> 
> So for now i'll try to work on a "svn info" button as test case (no input required and no needs to check the notebook name)
> 
> what it should do :
> 
> 1) run a system call that print some text as log
> 2) parse the log in a dict
> 3) have the dict printed in a new cell 
> 
> i'm aware that's my lack of JS knowledge, below some questions to better understand what you suggest me :
> 
> Il giorno 05/dic/2012, alle ore 10:23, Matthias BUSSONNIER <bussonniermatthias at gmail.com> ha scritto:

>> I'm not sure this can be easily accessible, but it could be a request for enhancement,
>> what you can do is : 
>> IPython.notebook.notebook_name
>> To lowercasse, and space to _ then add ipynb.
> 
> i didn't understand this steps :/

Ok,
This, should give you the name of the ipynb file (more or less)

IPython.notebook.notebook_name.toLowerCase().replace(' ','_')+'.ipynb'

>> 
>>> - clean all the output of the "notebook"
>> 
>> this can be done progrmatically on server side with a custom script on the ipynb file.
> 
> I saw the examples on the wiki page, that's great i can follow the example "remove_outputs" but this still require to know the notebook name
> 
> 
> i changed the code of the svninfo button to :
> 
> 
> IPython.toolbar.add_buttons_group([
> 	{
> 		'label'   : 'SVN Info',
> 		'icon'    : 'ui-icon-info', 
> 		'callback': function(){IPython.notebook.kernel.execute('a = !svn info'), 
> 		                       IPython.notebook.kernel.execute('svninfo = {i.split(": ")[0]:i.split(": ")[1] for i in a[:-1]}',
> 							                                          {'execute_reply': function(data){console.log('data:',data)}},
> 												  {'user_variables':['svninfo']})
> 							  }
> 	}
> ]);
> 
> 
> i'm not yet able to print out the dict in a new cell ? 
> 
> the svninfo dictionary is available in the notebook's code cell (after i press the info button), 
> so the IPython.notebook.kernel.execute works as aspected and the 
> 
> svninfo is also available in the JS console.
> 
> but trying to add the svninfo in the "last" cell adding a line in the button like :
> 
> #
> ?
> {'user_variables':['svninfo']}),
> IPython.notebook.get_cell(-1).set_text(svninfo)
> #
> 

IPython.notebook.insert_cell_at_bottom()
c = IPython.notebook.get_cell(-1)
c.set_text('foo')

does work to create a cell with 'foo' inside. 

you will have to put this in place of 

>  function(data){console.log('data:',data)}

and the text for the cell will be in data.user_variable.svninfo.


IPython.toolbar.add_buttons_group([
	{
		'label'   : 'SVN Info',
		'icon'    : 'ui-icon-info', 
		'callback': function(){IPython.notebook.kernel.execute('a = !svn info;svninfo = {i.split(": ")[0]:i.split(": ")[1] for i in a[:-1]}', 
							                                          {'execute_reply': function(data){
															console.log('data:',data)
															IPython.notebook.insert_cell_at_bottom()
															c = IPython.notebook.get_cell(-1)
															c.set_text(data.user_variable.svninfo)
															}},
												  {'user_variables':['svninfo']})
							  }
	}
]);



> but i have errors .. i guess i should not pass the python dict to set_text()
> 
> Uncaught ReferenceError: svninfo is not defined 
> 
> i tried to give the JS object :
> 
> #
> ?
> {'user_variables':['svninfo']}),
> IPython.notebook.get_cell(-1).set_text(data.user_variables.svninfo)


> ...
>> 
>>> - detect the name for the active notebook
>>> - clear all the output for the active notebook
>>> 
>>> 
>>> Thanks a lot for you help,
>> 
>> click on button -> callback a
>> 
>> callback a fetch info on the kernel trigger callback b on the response
>> 
>> callback b -> show a dialog to ask for commit message
>> 
>> click on ok trigger the correct information to be send on the kernel to commit.
>> 
> 
> does this means i can have more "callback"  executed when the same "button" is triggered ?

It is possible, but not what I was meaning. 


>> 
>> Does it help ? 
>> We probably want to do something like that in a more generic way embedded into IPython notebook. 
>> 
>> We'll be happy to get your feedback on that !
>> 
>> -- 
>> Matthias
> 
>  i'm a bit slow in learning how the system works (my fault)
> it is helping me a lot!

No it's not your fault,  the way javascript works is quite a bit different from python. 
It took me quite some time to figure out this also. 

The problem is you are not used to use unnamed callback which people called 'callback hell'.
It is quite possible to do those with named function, even if it is sometime more convenient to do them anonymously for closure. 

What you can do is open an issue or even a Pull request, so that we can see your full code, 
and comment inline on how to do things. 

Cheers, 
-- 
Matthias

> 
> Thanks!
> 
> --MAssimo.
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From carl.input at gmail.com  Mon Dec 10 20:48:44 2012
From: carl.input at gmail.com (Carl Smith)
Date: Tue, 11 Dec 2012 01:48:44 +0000
Subject: [IPython-dev] Some Thoughts on Notebook Security
Message-ID: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>

The IPython Notebook's vulnerability to cross-site scripting and
cross-site request forgery, XSS and XSRF, is a serious problem that
provides baddies with a range of attack vectors, each with almost
unlimited potential for harm to the user. Attempting to find a single
solution to so many problems is overwhelming and almost bound to fail.
Therefore, we will likely benefit from breaking the problem down,
insofar as we're able.

The most obvious place to start is to distinguish between static views
of a notebook and a notebook that has a running kernel, which I'll
call static notebooks and kernelled notebooks for lack of a
convention.

A static notebook is just a webpage, so it should ideally behave like
one. Any webpage can execute arbitrary JavaScript, so the fact that
static notebooks have this ability is not a concern in itself.

Serving all static notebooks from a separate domain should prevent XSS
and XSRF attacks because of the Same Origin Policy.

Static notebooks, served from a different domain, could be rendered
inside iframes, enabling us to embed them inside other webpages and
applications. These notebooks would still be superficially served by
our own servers, so the UX wouldn't be effected.

Using randomised URLs, or some other scheme that does not validate
requests by stored credentials, may allow us to serve notebooks from
the separate domain while keeping access to the notebooks private.
This may be useful when users wish to share a static notebook
selectively.

Other approaches all seem to rest on attempts to cripple JavaScript
execution, by either rendering the JavaScript source as text, else
removing it altogether. This seems like a bad idea as many static
notebooks, particularly in the long run, will need to be able to use
JavaScript to work properly. Will we refuse to render a user's graph
or widget in a static notebook because it uses JavaScript? This is a
never-ending spiral.

I think we need to build on browser security, and therefore trust it,
rather than build a heap of nasty hacks, which may be circumvented
anyway.

Just taking it back to basics for a moment: If Malory signs up for a
dating site, she might decide to put some dodgy JS inside some script
tags, and submit that as part of her profile. If the site doesn't
sanitise that HTML, we all know how it plays out; any poor chap that
thinks she's a hotty gets pwned. Sanitising her input is textbook.

Static notebooks are a totally different scenario. User's will want to
include JS and have it work when publishing a static notebook. This is
a feature of the Notebook. Furthermore, there's a number of ways to
get JS into a static notebook, some more subtle than others, while a
dating site is only dealing with text from an input field.

I don't have much to say about kernelled notebooks right now, but on
statics, I'd like to ask: If we serve a static notebook from a domain
which has zero features beyond hosting static notebooks, would those
notebooks be able to do anything to a user that a regular webpage
couldn't? I'm not a security guy, so I really don't know, but it seems
that we'd be totally free to let users publish whatever they liked,
with no more restrictions than browsers impose already.

I just don't want to see hours of work ploughed into crippling a super
powerful feature, if we could just an use extra domain with a simple
API instead.

Carl


From carl.input at gmail.com  Mon Dec 10 21:26:05 2012
From: carl.input at gmail.com (Carl Smith)
Date: Tue, 11 Dec 2012 02:26:05 +0000
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
Message-ID: <CAP-uhDf+E7OT_V=sTBZV5GWeLGKzZ0u1Hb0h+9YCOT5ziLVBYg@mail.gmail.com>

Just read back what I'd posted and needed to add: This would not
prevent XSRF attacks in general. You'd still need to check referrers
and so on. It would only prevent the Notebook being used to circumvent
those protections with XSS.

The main point is that we gain nothing by trying to cripple and
sanitise JavaScript in notebooks. I think??


From ellisonbg at gmail.com  Mon Dec 10 23:12:02 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 10 Dec 2012 20:12:02 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
Message-ID: <CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>

Carl,

I appreciate your thinking about this.  It is really important.  But I
think it *may* simple to fix:

There are two issues we have:

* In markdown cells users can put arbitrary HTML and JS.

To fix this, we need to enable the HTML sanitizer that comes with the
JS Markdown rendered that we are using.  This is what StackOverflow
uses to sanitize their markdown and should completely remove any
security risks coming from within markdown cells.

* In CodeCell output, the Javascript repr is dynamically passed
into eval.  This only happens when code is run, not when the notebook
is loaded, so it is less critical, but still needs to be fixed.

To fix this, we need to disable the Javascript representation of
objects altogether.

Will these two things not completely fix the security problems we
currently have?

Now the question is how to enable all of the nice things you can do
with Javascript.  I think the answer is Javascript plugins, JSON reprs
and JSON handlers:

https://github.com/ipython/ipython/pull/2518

The idea is that the extra Javascript cool-stuff will be installed by
the person who runs the notebook server once and for all notebooks on
that server.  Similar to how python packages are installed = you do
this before you start python.  To get data from python to the
Javascript plugins we will use JSON objects and trigger the callbacks
to handle them.

Unless I am misunderstanding the nature of the security risks, I think
this is what we should do.

Cheers,

Brian


On Mon, Dec 10, 2012 at 5:48 PM, Carl Smith <carl.input at gmail.com> wrote:
> The IPython Notebook's vulnerability to cross-site scripting and
> cross-site request forgery, XSS and XSRF, is a serious problem that
> provides baddies with a range of attack vectors, each with almost
> unlimited potential for harm to the user. Attempting to find a single
> solution to so many problems is overwhelming and almost bound to fail.
> Therefore, we will likely benefit from breaking the problem down,
> insofar as we're able.
>
> The most obvious place to start is to distinguish between static views
> of a notebook and a notebook that has a running kernel, which I'll
> call static notebooks and kernelled notebooks for lack of a
> convention.
>
> A static notebook is just a webpage, so it should ideally behave like
> one. Any webpage can execute arbitrary JavaScript, so the fact that
> static notebooks have this ability is not a concern in itself.
>
> Serving all static notebooks from a separate domain should prevent XSS
> and XSRF attacks because of the Same Origin Policy.
>
> Static notebooks, served from a different domain, could be rendered
> inside iframes, enabling us to embed them inside other webpages and
> applications. These notebooks would still be superficially served by
> our own servers, so the UX wouldn't be effected.
>
> Using randomised URLs, or some other scheme that does not validate
> requests by stored credentials, may allow us to serve notebooks from
> the separate domain while keeping access to the notebooks private.
> This may be useful when users wish to share a static notebook
> selectively.
>
> Other approaches all seem to rest on attempts to cripple JavaScript
> execution, by either rendering the JavaScript source as text, else
> removing it altogether. This seems like a bad idea as many static
> notebooks, particularly in the long run, will need to be able to use
> JavaScript to work properly. Will we refuse to render a user's graph
> or widget in a static notebook because it uses JavaScript? This is a
> never-ending spiral.
>
> I think we need to build on browser security, and therefore trust it,
> rather than build a heap of nasty hacks, which may be circumvented
> anyway.
>
> Just taking it back to basics for a moment: If Malory signs up for a
> dating site, she might decide to put some dodgy JS inside some script
> tags, and submit that as part of her profile. If the site doesn't
> sanitise that HTML, we all know how it plays out; any poor chap that
> thinks she's a hotty gets pwned. Sanitising her input is textbook.
>
> Static notebooks are a totally different scenario. User's will want to
> include JS and have it work when publishing a static notebook. This is
> a feature of the Notebook. Furthermore, there's a number of ways to
> get JS into a static notebook, some more subtle than others, while a
> dating site is only dealing with text from an input field.
>
> I don't have much to say about kernelled notebooks right now, but on
> statics, I'd like to ask: If we serve a static notebook from a domain
> which has zero features beyond hosting static notebooks, would those
> notebooks be able to do anything to a user that a regular webpage
> couldn't? I'm not a security guy, so I really don't know, but it seems
> that we'd be totally free to let users publish whatever they liked,
> with no more restrictions than browsers impose already.
>
> I just don't want to see hours of work ploughed into crippling a super
> powerful feature, if we could just an use extra domain with a simple
> API instead.
>
> Carl
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From aron.ahmadia at gmail.com  Mon Dec 10 23:18:23 2012
From: aron.ahmadia at gmail.com (Aron Ahmadia)
Date: Mon, 10 Dec 2012 23:18:23 -0500
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
Message-ID: <67CF674D-E62B-427D-B9DC-6896E7B0570F@gmail.com>

+1

Sent from my iPhone

On Dec 10, 2012, at 11:12 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Carl,
> 
> I appreciate your thinking about this.  It is really important.  But I
> think it *may* simple to fix:
> 
> There are two issues we have:
> 
> * In markdown cells users can put arbitrary HTML and JS.
> 
> To fix this, we need to enable the HTML sanitizer that comes with the
> JS Markdown rendered that we are using.  This is what StackOverflow
> uses to sanitize their markdown and should completely remove any
> security risks coming from within markdown cells.
> 
> * In CodeCell output, the Javascript repr is dynamically passed
> into eval.  This only happens when code is run, not when the notebook
> is loaded, so it is less critical, but still needs to be fixed.
> 
> To fix this, we need to disable the Javascript representation of
> objects altogether.
> 
> Will these two things not completely fix the security problems we
> currently have?
> 
> Now the question is how to enable all of the nice things you can do
> with Javascript.  I think the answer is Javascript plugins, JSON reprs
> and JSON handlers:
> 
> https://github.com/ipython/ipython/pull/2518
> 
> The idea is that the extra Javascript cool-stuff will be installed by
> the person who runs the notebook server once and for all notebooks on
> that server.  Similar to how python packages are installed = you do
> this before you start python.  To get data from python to the
> Javascript plugins we will use JSON objects and trigger the callbacks
> to handle them.
> 
> Unless I am misunderstanding the nature of the security risks, I think
> this is what we should do.
> 
> Cheers,
> 
> Brian
> 
> 
> On Mon, Dec 10, 2012 at 5:48 PM, Carl Smith <carl.input at gmail.com> wrote:
>> The IPython Notebook's vulnerability to cross-site scripting and
>> cross-site request forgery, XSS and XSRF, is a serious problem that
>> provides baddies with a range of attack vectors, each with almost
>> unlimited potential for harm to the user. Attempting to find a single
>> solution to so many problems is overwhelming and almost bound to fail.
>> Therefore, we will likely benefit from breaking the problem down,
>> insofar as we're able.
>> 
>> The most obvious place to start is to distinguish between static views
>> of a notebook and a notebook that has a running kernel, which I'll
>> call static notebooks and kernelled notebooks for lack of a
>> convention.
>> 
>> A static notebook is just a webpage, so it should ideally behave like
>> one. Any webpage can execute arbitrary JavaScript, so the fact that
>> static notebooks have this ability is not a concern in itself.
>> 
>> Serving all static notebooks from a separate domain should prevent XSS
>> and XSRF attacks because of the Same Origin Policy.
>> 
>> Static notebooks, served from a different domain, could be rendered
>> inside iframes, enabling us to embed them inside other webpages and
>> applications. These notebooks would still be superficially served by
>> our own servers, so the UX wouldn't be effected.
>> 
>> Using randomised URLs, or some other scheme that does not validate
>> requests by stored credentials, may allow us to serve notebooks from
>> the separate domain while keeping access to the notebooks private.
>> This may be useful when users wish to share a static notebook
>> selectively.
>> 
>> Other approaches all seem to rest on attempts to cripple JavaScript
>> execution, by either rendering the JavaScript source as text, else
>> removing it altogether. This seems like a bad idea as many static
>> notebooks, particularly in the long run, will need to be able to use
>> JavaScript to work properly. Will we refuse to render a user's graph
>> or widget in a static notebook because it uses JavaScript? This is a
>> never-ending spiral.
>> 
>> I think we need to build on browser security, and therefore trust it,
>> rather than build a heap of nasty hacks, which may be circumvented
>> anyway.
>> 
>> Just taking it back to basics for a moment: If Malory signs up for a
>> dating site, she might decide to put some dodgy JS inside some script
>> tags, and submit that as part of her profile. If the site doesn't
>> sanitise that HTML, we all know how it plays out; any poor chap that
>> thinks she's a hotty gets pwned. Sanitising her input is textbook.
>> 
>> Static notebooks are a totally different scenario. User's will want to
>> include JS and have it work when publishing a static notebook. This is
>> a feature of the Notebook. Furthermore, there's a number of ways to
>> get JS into a static notebook, some more subtle than others, while a
>> dating site is only dealing with text from an input field.
>> 
>> I don't have much to say about kernelled notebooks right now, but on
>> statics, I'd like to ask: If we serve a static notebook from a domain
>> which has zero features beyond hosting static notebooks, would those
>> notebooks be able to do anything to a user that a regular webpage
>> couldn't? I'm not a security guy, so I really don't know, but it seems
>> that we'd be totally free to let users publish whatever they liked,
>> with no more restrictions than browsers impose already.
>> 
>> I just don't want to see hours of work ploughed into crippling a super
>> powerful feature, if we could just an use extra domain with a simple
>> API instead.
>> 
>> Carl
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From brad.froehle at gmail.com  Tue Dec 11 00:22:04 2012
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Mon, 10 Dec 2012 21:22:04 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
Message-ID: <1A71AFBE321A44169B9FAC319314AE2D@gmail.com>

Regardless of the approach taken, I think that Carl's suggestion of using separate domains for viewing & editing notebooks should be taken to heart.  It does help eliminate one vector of attack.

-Brad

On Monday, December 10, 2012 at 8:12 PM, Brian Granger wrote: 
> Carl,
> 
> I appreciate your thinking about this. It is really important. But I
> think it *may* simple to fix:
> 
> There are two issues we have:
> 
> * In markdown cells users can put arbitrary HTML and JS.
> 
> To fix this, we need to enable the HTML sanitizer that comes with the
> JS Markdown rendered that we are using. This is what StackOverflow
> uses to sanitize their markdown and should completely remove any
> security risks coming from within markdown cells.
> 
> * In CodeCell output, the Javascript repr is dynamically passed
> into eval. This only happens when code is run, not when the notebook
> is loaded, so it is less critical, but still needs to be fixed.
> 
> To fix this, we need to disable the Javascript representation of
> objects altogether.
> 
> Will these two things not completely fix the security problems we
> currently have?
> 
> Now the question is how to enable all of the nice things you can do
> with Javascript. I think the answer is Javascript plugins, JSON reprs
> and JSON handlers:
> 
> https://github.com/ipython/ipython/pull/2518
> 
> The idea is that the extra Javascript cool-stuff will be installed by
> the person who runs the notebook server once and for all notebooks on
> that server. Similar to how python packages are installed = you do
> this before you start python. To get data from python to the
> Javascript plugins we will use JSON objects and trigger the callbacks
> to handle them.
> 
> Unless I am misunderstanding the nature of the security risks, I think
> this is what we should do.
> 
> Cheers,
> 
> Brian
> 
> 
> On Mon, Dec 10, 2012 at 5:48 PM, Carl Smith <carl.input at gmail.com (mailto:carl.input at gmail.com)> wrote:
> > The IPython Notebook's vulnerability to cross-site scripting and
> > cross-site request forgery, XSS and XSRF, is a serious problem that
> > provides baddies with a range of attack vectors, each with almost
> > unlimited potential for harm to the user. Attempting to find a single
> > solution to so many problems is overwhelming and almost bound to fail.
> > Therefore, we will likely benefit from breaking the problem down,
> > insofar as we're able.
> > 
> > The most obvious place to start is to distinguish between static views
> > of a notebook and a notebook that has a running kernel, which I'll
> > call static notebooks and kernelled notebooks for lack of a
> > convention.
> > 
> > A static notebook is just a webpage, so it should ideally behave like
> > one. Any webpage can execute arbitrary JavaScript, so the fact that
> > static notebooks have this ability is not a concern in itself.
> > 
> > Serving all static notebooks from a separate domain should prevent XSS
> > and XSRF attacks because of the Same Origin Policy.
> > 
> > Static notebooks, served from a different domain, could be rendered
> > inside iframes, enabling us to embed them inside other webpages and
> > applications. These notebooks would still be superficially served by
> > our own servers, so the UX wouldn't be effected.
> > 
> > Using randomised URLs, or some other scheme that does not validate
> > requests by stored credentials, may allow us to serve notebooks from
> > the separate domain while keeping access to the notebooks private.
> > This may be useful when users wish to share a static notebook
> > selectively.
> > 
> > Other approaches all seem to rest on attempts to cripple JavaScript
> > execution, by either rendering the JavaScript source as text, else
> > removing it altogether. This seems like a bad idea as many static
> > notebooks, particularly in the long run, will need to be able to use
> > JavaScript to work properly. Will we refuse to render a user's graph
> > or widget in a static notebook because it uses JavaScript? This is a
> > never-ending spiral.
> > 
> > I think we need to build on browser security, and therefore trust it,
> > rather than build a heap of nasty hacks, which may be circumvented
> > anyway.
> > 
> > Just taking it back to basics for a moment: If Malory signs up for a
> > dating site, she might decide to put some dodgy JS inside some script
> > tags, and submit that as part of her profile. If the site doesn't
> > sanitise that HTML, we all know how it plays out; any poor chap that
> > thinks she's a hotty gets pwned. Sanitising her input is textbook.
> > 
> > Static notebooks are a totally different scenario. User's will want to
> > include JS and have it work when publishing a static notebook. This is
> > a feature of the Notebook. Furthermore, there's a number of ways to
> > get JS into a static notebook, some more subtle than others, while a
> > dating site is only dealing with text from an input field.
> > 
> > I don't have much to say about kernelled notebooks right now, but on
> > statics, I'd like to ask: If we serve a static notebook from a domain
> > which has zero features beyond hosting static notebooks, would those
> > notebooks be able to do anything to a user that a regular webpage
> > couldn't? I'm not a security guy, so I really don't know, but it seems
> > that we'd be totally free to let users publish whatever they liked,
> > with no more restrictions than browsers impose already.
> > 
> > I just don't want to see hours of work ploughed into crippling a super
> > powerful feature, if we could just an use extra domain with a simple
> > API instead.
> > 
> > Carl
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu (mailto:bgranger at calpoly.edu) and ellisonbg at gmail.com (mailto:ellisonbg at gmail.com)
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
> http://mail.scipy.org/mailman/listinfo/ipython-dev





From ellisonbg at gmail.com  Tue Dec 11 00:27:59 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 10 Dec 2012 21:27:59 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <1A71AFBE321A44169B9FAC319314AE2D@gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
	<1A71AFBE321A44169B9FAC319314AE2D@gmail.com>
Message-ID: <CAH4pYpRmF82YO1ir8dFhDB8psTr+wqYV=8hjsDVU5QWz_9wtzA@mail.gmail.com>

But am I not correct that with the measures I proposed, that attack
vector is also rendered null?  I don't want to have to tell people -
you have to run multiple domains when using the notebook.  In many
cases, people ar running the notebook without any DNS whatsoever...

Cheers,

Brian

On Mon, Dec 10, 2012 at 9:22 PM, Bradley M. Froehle
<brad.froehle at gmail.com> wrote:
> Regardless of the approach taken, I think that Carl's suggestion of using separate domains for viewing & editing notebooks should be taken to heart.  It does help eliminate one vector of attack.
>
> -Brad
>
> On Monday, December 10, 2012 at 8:12 PM, Brian Granger wrote:
>> Carl,
>>
>> I appreciate your thinking about this. It is really important. But I
>> think it *may* simple to fix:
>>
>> There are two issues we have:
>>
>> * In markdown cells users can put arbitrary HTML and JS.
>>
>> To fix this, we need to enable the HTML sanitizer that comes with the
>> JS Markdown rendered that we are using. This is what StackOverflow
>> uses to sanitize their markdown and should completely remove any
>> security risks coming from within markdown cells.
>>
>> * In CodeCell output, the Javascript repr is dynamically passed
>> into eval. This only happens when code is run, not when the notebook
>> is loaded, so it is less critical, but still needs to be fixed.
>>
>> To fix this, we need to disable the Javascript representation of
>> objects altogether.
>>
>> Will these two things not completely fix the security problems we
>> currently have?
>>
>> Now the question is how to enable all of the nice things you can do
>> with Javascript. I think the answer is Javascript plugins, JSON reprs
>> and JSON handlers:
>>
>> https://github.com/ipython/ipython/pull/2518
>>
>> The idea is that the extra Javascript cool-stuff will be installed by
>> the person who runs the notebook server once and for all notebooks on
>> that server. Similar to how python packages are installed = you do
>> this before you start python. To get data from python to the
>> Javascript plugins we will use JSON objects and trigger the callbacks
>> to handle them.
>>
>> Unless I am misunderstanding the nature of the security risks, I think
>> this is what we should do.
>>
>> Cheers,
>>
>> Brian
>>
>>
>> On Mon, Dec 10, 2012 at 5:48 PM, Carl Smith <carl.input at gmail.com (mailto:carl.input at gmail.com)> wrote:
>> > The IPython Notebook's vulnerability to cross-site scripting and
>> > cross-site request forgery, XSS and XSRF, is a serious problem that
>> > provides baddies with a range of attack vectors, each with almost
>> > unlimited potential for harm to the user. Attempting to find a single
>> > solution to so many problems is overwhelming and almost bound to fail.
>> > Therefore, we will likely benefit from breaking the problem down,
>> > insofar as we're able.
>> >
>> > The most obvious place to start is to distinguish between static views
>> > of a notebook and a notebook that has a running kernel, which I'll
>> > call static notebooks and kernelled notebooks for lack of a
>> > convention.
>> >
>> > A static notebook is just a webpage, so it should ideally behave like
>> > one. Any webpage can execute arbitrary JavaScript, so the fact that
>> > static notebooks have this ability is not a concern in itself.
>> >
>> > Serving all static notebooks from a separate domain should prevent XSS
>> > and XSRF attacks because of the Same Origin Policy.
>> >
>> > Static notebooks, served from a different domain, could be rendered
>> > inside iframes, enabling us to embed them inside other webpages and
>> > applications. These notebooks would still be superficially served by
>> > our own servers, so the UX wouldn't be effected.
>> >
>> > Using randomised URLs, or some other scheme that does not validate
>> > requests by stored credentials, may allow us to serve notebooks from
>> > the separate domain while keeping access to the notebooks private.
>> > This may be useful when users wish to share a static notebook
>> > selectively.
>> >
>> > Other approaches all seem to rest on attempts to cripple JavaScript
>> > execution, by either rendering the JavaScript source as text, else
>> > removing it altogether. This seems like a bad idea as many static
>> > notebooks, particularly in the long run, will need to be able to use
>> > JavaScript to work properly. Will we refuse to render a user's graph
>> > or widget in a static notebook because it uses JavaScript? This is a
>> > never-ending spiral.
>> >
>> > I think we need to build on browser security, and therefore trust it,
>> > rather than build a heap of nasty hacks, which may be circumvented
>> > anyway.
>> >
>> > Just taking it back to basics for a moment: If Malory signs up for a
>> > dating site, she might decide to put some dodgy JS inside some script
>> > tags, and submit that as part of her profile. If the site doesn't
>> > sanitise that HTML, we all know how it plays out; any poor chap that
>> > thinks she's a hotty gets pwned. Sanitising her input is textbook.
>> >
>> > Static notebooks are a totally different scenario. User's will want to
>> > include JS and have it work when publishing a static notebook. This is
>> > a feature of the Notebook. Furthermore, there's a number of ways to
>> > get JS into a static notebook, some more subtle than others, while a
>> > dating site is only dealing with text from an input field.
>> >
>> > I don't have much to say about kernelled notebooks right now, but on
>> > statics, I'd like to ask: If we serve a static notebook from a domain
>> > which has zero features beyond hosting static notebooks, would those
>> > notebooks be able to do anything to a user that a regular webpage
>> > couldn't? I'm not a security guy, so I really don't know, but it seems
>> > that we'd be totally free to let users publish whatever they liked,
>> > with no more restrictions than browsers impose already.
>> >
>> > I just don't want to see hours of work ploughed into crippling a super
>> > powerful feature, if we could just an use extra domain with a simple
>> > API instead.
>> >
>> > Carl
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu (mailto:bgranger at calpoly.edu) and ellisonbg at gmail.com (mailto:ellisonbg at gmail.com)
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Tue Dec 11 01:05:59 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 11 Dec 2012 00:05:59 -0600
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
Message-ID: <50C6CD47.2070600@creativetrax.com>

On 12/10/12 10:12 PM, Brian Granger wrote:
> * In CodeCell output, the Javascript repr is dynamically passed
> into eval.  This only happens when code is run, not when the notebook
> is loaded, so it is less critical, but still needs to be fixed.
>
> To fix this, we need to disable the Javascript representation of
> objects altogether.
>
> Will these two things not completely fix the security problems we
> currently have?

It appears that IPython.core.display.HTML() allows <script> tags in the 
html the user submits:

import IPython
IPython.core.display.HTML('<script>alert("hi")</script>')

Thanks,

Jason



From ellisonbg at gmail.com  Tue Dec 11 01:11:47 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 10 Dec 2012 22:11:47 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <50C6CD47.2070600@creativetrax.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<CAH4pYpQtkUAHSnCmi4UCw9fGYfz022FfOejmc3rx+R3H5eKL8Q@mail.gmail.com>
	<50C6CD47.2070600@creativetrax.com>
Message-ID: <CAH4pYpRj=qDtw6XhLhKXwrxTzPj=Nr+OLQaK2s-jTqi-zUh-og@mail.gmail.com>

Oh, yes forgot about that, we will have to clean that HTML as well.

On Mon, Dec 10, 2012 at 10:05 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> On 12/10/12 10:12 PM, Brian Granger wrote:
>> * In CodeCell output, the Javascript repr is dynamically passed
>> into eval.  This only happens when code is run, not when the notebook
>> is loaded, so it is less critical, but still needs to be fixed.
>>
>> To fix this, we need to disable the Javascript representation of
>> objects altogether.
>>
>> Will these two things not completely fix the security problems we
>> currently have?
>
> It appears that IPython.core.display.HTML() allows <script> tags in the
> html the user submits:
>
> import IPython
> IPython.core.display.HTML('<script>alert("hi")</script>')
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From david.trem at gmail.com  Tue Dec 11 04:53:33 2012
From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=)
Date: Tue, 11 Dec 2012 10:53:33 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
Message-ID: <50C7029D.2040908@gmail.com>

Hi,

Finally, thanks to Thomas entry point, I gave a try at implementing
some Fortran cell magic based on f2py. This is cell_magic only with
almost no argument available. But this is fully functional :-)

I've asked for a pull request available here:
https://github.com/ipython/ipython/pull/2674

A demo notebook is available here:
http://nbviewer.ipython.org/4257286/

Comments, suggestions and feature requests are very welcome :-)

Regards,

David

Le 07/12/12 14:15, Thomas Kluyver a ?crit :
> On 7 December 2012 12:49, David Tremouilles <david.trem at gmail.com 
> <mailto:david.trem at gmail.com>> wrote:
>
>     I'm wondering if there is any %%fortran magic available out there ?
>     Does anybody tried to implement this ?
>
>
> I don't think anyone's done it yet. If you want to have a go, a good 
> starting point would be the %%cython magic we ship as an extension:
> https://github.com/ipython/ipython/blob/master/IPython/extensions/cythonmagic.py
>
> Best wishes,
> Thomas
>
>
> _______________________________________________
> IPython-User mailing list
> IPython-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/8f8cdd15/attachment.html>

From klonuo at gmail.com  Tue Dec 11 07:16:33 2012
From: klonuo at gmail.com (klo uo)
Date: Tue, 11 Dec 2012 13:16:33 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <50C7029D.2040908@gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
Message-ID: <CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>

Thanks for working on this.
On Windows it won't run:

========================================
Could not locate executable C:Python27pythonw.exe
Executable C:Python27pythonw.exe does not exist
========================================

Obvious issue with backslash at some point

Is distutil.exec_command only way to do this?
On Windows it will look "ugly" as it pops command line window on command
execution


On Tue, Dec 11, 2012 at 10:53 AM, Tr?mouilles David <david.trem at gmail.com>wrote:

>  Hi,
>
> Finally, thanks to Thomas entry point, I gave a try at implementing
> some Fortran cell magic based on f2py. This is cell_magic only with
> almost no argument available. But this is fully functional :-)
>
> I've asked for a pull request available here:
> https://github.com/ipython/ipython/pull/2674
>
> A demo notebook is available here:
> http://nbviewer.ipython.org/4257286/
>
> Comments, suggestions and feature requests are very welcome :-)
>
> Regards,
>
> David
>
> Le 07/12/12 14:15, Thomas Kluyver a ?crit :
>
>  On 7 December 2012 12:49, David Tremouilles <david.trem at gmail.com> wrote:
>
>> I'm wondering if there is any %%fortran magic available out there ?
>> Does anybody tried to implement this ?
>
>
> I don't think anyone's done it yet. If you want to have a go, a good
> starting point would be the %%cython magic we ship as an extension:
>
> https://github.com/ipython/ipython/blob/master/IPython/extensions/cythonmagic.py
>
> Best wishes,
> Thomas
>
>
> _______________________________________________
> IPython-User mailing listIPython-User at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-user
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/dd0fe9f8/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 11 09:31:48 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 11 Dec 2012 15:31:48 +0100
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
Message-ID: <E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>

Long response to many messages.

tl;dr :
 lots of good idea, some misconception, a few good point raised.

Le 11 d?c. 2012 ? 02:48, Carl Smith a ?crit :

> A static notebook is just a webpage, so it should ideally behave like
> one. Any webpage can execute arbitrary JavaScript, so the fact that
> static notebooks have this ability is not a concern in itself.


I think it is. In nbviewer we would like to allow people to comment, 
which obviously require them to be logged in? so it is not **exactly** static.

> Static notebooks, served from a different domain, could be rendered
> inside iframes, enabling us to embed them inside other webpages and
> applications. These notebooks would still be superficially served by
> our own servers, so the UX wouldn't be effected.

keep in mind that iframe are not sandboxed, and you can inject js on parent frame. 
Unless you use the sandbox attributes, which is part of html5 but not implemented in every
browser? And not yet infallible, it is more a "we'll help you embed other pages by providing a separate 
js namespace but we don't guaranty yes that the VM is unbreachable"


> I don't have much to say about kernelled notebooks right now, but on
> statics, I'd like to ask: If we serve a static notebook from a domain
> which has zero features beyond hosting static notebooks, would those
> notebooks be able to do anything to a user that a regular webpage
> couldn't? I'm not a security guy, so I really don't know, but it seems
> that we'd be totally free to let users publish whatever they liked,
> with no more restrictions than browsers impose already.

We still have some responsibility of what is displayed.

Responsible disclosure don't want to say much more but but having a statically display 
notebook is often link to having a "sharing/import" button which is dangerous.
And could lead to self propagating notebook through account that can infect other 
notebooks, or share itself on twitter...


> The main point is that we gain nothing by trying to cripple and
> sanitise JavaScript in notebooks. I think??

I know that Brian does not agree with that, but I am on your side with this point.
Except that preventing javascript might force people to do things "the right way".
This will lead to project that will still works across IPython version, whereas displaying JS have great chance to break.


> To fix this, we need to disable the Javascript representation of
> objects altogether.
> Unless I am misunderstanding the nature of the security risks, I think
> this is what we should do.


I disagree with the direct implication, but agree with the fact that disabling javascript repr can be done and will lead to cleaner code.
Flaws are **not** in repr. Flaws are mostly in assumption made on ipynb as a format and our html frontend.   And please stop considering that the "ipython notebook" is 
the only tool that generate ipynb files and use them.

We really have to choose a more appropriate vocabulary to speak of 
* ipynb as file
* ipynb as a format
* A notebook as the repr of a given ipynb files view throughout **our** html frontend.
* **Our** notebook server.

We should **not** consider that danger come from html/js repr but from **forged** ipynb file.

I challenge you to find a PDF generated from a tex file with pdflatex that trigger a breach in adobe reader. 
Anyone ?

Did anybody have a jpeg that came directly from his/her camera that trigger privileges escalation on windows ?
No ?

Corrupted mp3 that crash your computer from audacity ?

Do you consider that a script that ask for your root password is a virus/malware.. ?

Example :

#!/bin/bash 
sudo rm -rf  --no-preserve-root /

No, you just give the ability to do anything, you just make the ability to do bad stuff painful.
So my take on this one is :

Allow javascript only if:

server is **started** with some painfull flag  --yes-i-know-what-i-do-i-want-to-enable-js-in-output
When Js arrives to **frontend** in an output (see i didn't  tell where it came from? it could be at load time)
If the above flag has been set, 
	modal dialog 
		"something want to display js, allows for this session ? " (5 sec count down for before ok button becomes clickable.)

> Regardless of the approach taken, I think that Carl's suggestion of using separate domains for viewing & editing notebooks should be taken to heart.  It does help eliminate one vector of attack.

Multi domain is a real good idea. I have a clear view in my head on how we could use that in a way close to OAuth to allow javascript by still having "logged-in" users.
It wouldn't be as seamless a something like github, but close.

> But am I not correct that with the measures I proposed, that attack
> vector is also rendered null?  I don't want to have to tell people -
> you have to run multiple domains when using the notebook.  In many
> cases, people ar running the notebook without any DNS whatsoever?

The **big** question is:
Are viewer logged in (in any way) to the given server, and if so do they have the right to do anything else with those credentials ?
If it is just a public notebook viewer, then it's fine.

If you want something more "interactive" (sharing/ permissions?etc, and the display any JS ) you won't have much choice.
Or you will have a painful multi-login.

In any way this I thing should not diminish the fact that we should tell IPython notebook user to only 
view and even more download ipynb files from people they **trust**.

One good plugin for IPython notebook might be gpg-notebook-signing and chain of trust.


> It appears that IPython.core.display.HTML() allows <script> tags in the 
> html the user submits:
> 
> import IPython
> IPython.core.display.HTML('<script>alert("hi")</script>')

Yes this the issues are that in particular **those** are executed at load time.

To conclude I would say that Nobody did bring the question of security while collaborating, which is a much more difficult question.

also I've already written some stuff on js/security in IPep 3, feel free to complete or even make it  a separate IPep only focused on security.

https://github.com/ipython/ipython/wiki/IPEP-3:-Multiuser-support-in-the-notebook


Cheers.
-- 
Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/7122d42f/attachment.html>

From carl.input at gmail.com  Tue Dec 11 10:10:04 2012
From: carl.input at gmail.com (Carl Smith)
Date: Tue, 11 Dec 2012 15:10:04 +0000
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
Message-ID: <CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>

Hi Brian

> * In markdown cells users can put arbitrary HTML and JS.
>
> To fix this, we need to enable the HTML sanitizer that comes with the
> JS Markdown rendered that we are using.  This is what StackOverflow
> uses to sanitize their markdown and should completely remove any
> security risks coming from within markdown cells.

StackOverflow don't want any user-provided JavaScript to be rendered
inside there forums. Obviously it would create the same security
issues we're discussing here, but it's not a feature of StackOverflow.
User's only provide rich text, so sanitising posts makes perfect
sense. I think we should bare in mind that we're trying to support
arbitrary JS, while almost everyone else is trying to prevent it, not
for security reasons, but to prevent their site from becoming Myspace.

> * In CodeCell output, the Javascript repr is dynamically passed
> into eval.  This only happens when code is run, not when the notebook
> is loaded, so it is less critical, but still needs to be fixed.
>
> To fix this, we need to disable the Javascript representation of
> objects altogether.
>
> Will these two things not completely fix the security problems we
> currently have?

> Now the question is how to enable all of the nice things you can do
> with Javascript.  I think the answer is Javascript plugins, JSON reprs
> and JSON handlers:
>
> https://github.com/ipython/ipython/pull/2518
>
> The idea is that the extra Javascript cool-stuff will be installed by
> the person who runs the notebook server once and for all notebooks on
> that server.  Similar to how python packages are installed = you do
> this before you start python.  To get data from python to the
> Javascript plugins we will use JSON objects and trigger the callbacks
> to handle them.

This seems to be dependent on a kernel, which static notebooks don't
have? If I generate a static notebook, which is just a web page, then
post that page to a hosting service, or email it to someone, how would
the plugins work? Maybe we're looking at two slightly different
scenarios. I'm focussed on static views only. The host should not have
to allow anything more than posting and getting HTML documents.

When creating statics, I see IPython Notebook as essentially a webpage
editor. It should be able to just use JS as freely as any page, then
be distributed like any page.

All XSS attacks, as I understand it, depend on the baddy being able to
have their malicious JS served from another domain that is trusted,
circumventing the Same Origin Policy. Simple solution: don't serve
user's notebooks from trusted domains.

================

Hi Matthias

> Static notebooks, served from a different domain, could be rendered
> inside iframes, enabling us to embed them inside other webpages and
> applications. These notebooks would still be superficially served by
> our own servers, so the UX wouldn't be effected.
>
>
> keep in mind that iframe are not sandboxed, and you can inject js on parent
> frame.
> Unless you use the sandbox attributes, which is part of html5 but not
> implemented in every
> browser? And not yet infallible, it is more a "we'll help you embed other
> pages by providing a separate
> js namespace but we don't guaranty yes that the VM is unbreachable"

I pretty sure iframes are sandboxed in the sense that a parent page
and an iframe can not communicate unless they have the same origin,
and this is an old feature. The new sandbox attribute in HTML5 is for
a different purpose.


> Responsible disclosure don't want to say much more but but having a
> statically display
> notebook is often link to having a "sharing/import" button which is
> dangerous.
> And could lead to self propagating notebook through account that can infect
> other
> notebooks, or share itself on twitter...

Any buttons, like for importing a notebook, would live in the parent
page and would have no access to, nor allow access from, the iframe.
The parent page would know which static notebook it embeds in the
iframe though, so it could provide buttons that connect to the actual
notebook in question, which is a totally different file to the static
notebook being rendered in the iframe anyway.

> Multi domain is a real good idea. I have a clear view in my head on how we
> could use that in a way close to OAuth to allow javascript by still having
> "logged-in" users.
> It wouldn't be as seamless a something like github, but close.

I think we're looking at things differently: You seem to be
considering static views as something generated on the fly and on
demand, nbviewer style. I'm thinking about running nbconvert on a
notebook, then keeping the output as a webpage to be copied and passed
around freely. Once the static notebook exists, it's a done deal.
There's no chance of any changes to IPython breaking it. It's a
independent webpage. Updating it would amount to deleting it and
replacing it with a new version.

> The **big** question is:
> Are viewer logged in (in any way) to the given server, and if so do they
> have the right to do anything else with those credentials ?
> If it is just a public notebook viewer, then it's fine.
>
> If you want something more "interactive" (sharing/ permissions?etc, and the
> display any JS ) you won't have much choice.
> Or you will have a painful multi-login.

I'm very much against hosting user submitted notebooks on any domain
with cookie based authentication. It needs to be divided into 'trusted
domain', where no user's JS will ever be served, and 'hosting domain'
that has no account system of it's own. The trusted domain would
control the hosting domain, as a kind of slave.

That's just my take on all this.

Cheers

Carl


From brad.froehle at gmail.com  Tue Dec 11 11:32:55 2012
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Tue, 11 Dec 2012 08:32:55 -0800
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
Message-ID: <32A45AD050C3485C91584666E1C22450@gmail.com>

I would have suggested using the `numpy.f2py.compile` function directly: https://github.com/numpy/numpy/blob/master/numpy/f2py/__init__.py

? although it appears to use exec_command as well.

-Brad


On Tuesday, December 11, 2012 at 4:16 AM, klo uo wrote:

> Thanks for working on this.
> On Windows it won't run:
>  
> ========================================
> Could not locate executable C:Python27pythonw.exe
> Executable C:Python27pythonw.exe does not exist
> ========================================
>  
> Obvious issue with backslash at some point
>  
> Is distutil.exec_command only way to do this?
> On Windows it will look "ugly" as it pops command line window on command execution





From bussonniermatthias at gmail.com  Tue Dec 11 12:15:59 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 11 Dec 2012 18:15:59 +0100
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
Message-ID: <7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>


Le 11 d?c. 2012 ? 16:10, Carl Smith a ?crit :

> Hi Brian
>> 
>> The idea is that the extra Javascript cool-stuff will be installed by
>> the person who runs the notebook server once and for all notebooks on
>> that server.  Similar to how python packages are installed = you do
>> this before you start python.  To get data from python to the
>> Javascript plugins we will use JSON objects and trigger the callbacks
>> to handle them.
> 
> This seems to be dependent on a kernel, which static notebooks don't
> have? If I generate a static notebook, which is just a web page, then
> post that page to a hosting service, or email it to someone, how would
> the plugins work? Maybe we're looking at two slightly different
> scenarios. I'm focussed on static views only. The host should not have
> to allow anything more than posting and getting HTML documents.

IIRC, you can embed several repr in the ipynb file. 
So you could provide a plugin that can "render" object on static view. 
(like d3.js graph, you don't need the kernel to do that)

> ================
> 
> Hi Matthias
> 
>> Static notebooks, served from a different domain, could be rendered
>> inside iframes, enabling us to embed them inside other webpages and
>> applications. These notebooks would still be superficially served by
>> our own servers, so the UX wouldn't be effected.
>> 
>> keep in mind that iframe are not sandboxed, and you can inject js on parent
>> frame.
>> Unless you use the sandbox attributes, which is part of html5 but not
>> implemented in every
>> browser? And not yet infallible, it is more a "we'll help you embed other
>> pages by providing a separate
>> js namespace but we don't guaranty yes that the VM is unbreachable"
> 

> I pretty sure iframes are sandboxed in the sense that a parent page
> and an iframe can not communicate unless they have the same origin,
> and this is an old feature. The new sandbox attribute in HTML5 is for
> a different purpose.

But this is still kind of a problem, as usually the static view will be served from the "same origin" 
as the rest of the website. 

If you don't want to make a notebook fully public, you have to have some kind of authentication 
that allows you to load it. 

I'm still doubt a little about what frames are supposed to do and what they actually do.
I'm not an expert on that, but it is still worth digging.

>> Responsible disclosure don't want to say much more but but having a
>> statically display
>> notebook is often link to having a "sharing/import" button which is
>> dangerous.
>> And could lead to self propagating notebook through account that can infect
>> other
>> notebooks, or share itself on twitter...
> 
> Any buttons, like for importing a notebook, would live in the parent
> page and would have no access to, nor allow access from, the iframe.
> The parent page would know which static notebook it embeds in the
> iframe though, so it could provide buttons that connect to the actual
> notebook in question, which is a totally different file to the static
> notebook being rendered in the iframe anyway.

I understand what you want to do, I guess the definition of "static" is blurry. 
If you want a perfectly static version (does it make sense in html) you can go with iframe. 
If you want the ability to comment on a particular cell, then you have to build iframe for 
every cell. 
and you lose the ability to comment "inline" as github does.


> 
>> Multi domain is a real good idea. I have a clear view in my head on how we
>> could use that in a way close to OAuth to allow javascript by still having
>> "logged-in" users.
>> It wouldn't be as seamless a something like github, but close.
> 
> I think we're looking at things differently: You seem to be
> considering static views as something generated on the fly and on
> demand, nbviewer style. I'm thinking about running nbconvert on a
> notebook, then keeping the output as a webpage to be copied and passed
> around freely. Once the static notebook exists, it's a done deal.
> There's no chance of any changes to IPython breaking it. It's a
> independent webpage. Updating it would amount to deleting it and
> replacing it with a new version.

I don't think those are quite different.
You can have a "perfectly static" version that embeds bad js and require some kind of authentication to be seen. 
The line between "on the fly" and static is thin.

> 
>> The **big** question is:
>> Are viewer logged in (in any way) to the given server, and if so do they
>> have the right to do anything else with those credentials ?
>> If it is just a public notebook viewer, then it's fine.
>> 
>> If you want something more "interactive" (sharing/ permissions?etc, and the
>> display any JS ) you won't have much choice.
>> Or you will have a painful multi-login.
> 
> I'm very much against hosting user submitted notebooks on any domain
> with cookie based authentication. It needs to be divided into 'trusted
> domain', where no user's JS will ever be served, and 'hosting domain'
> that has no account system of it's own. The trusted domain would
> control the hosting domain, as a kind of slave.

Yep, kind of what I have in mind. 
The hosting domain can have "tokens" 
Publish this comment on this notebook on the behalf of ...
The you have to "validate" those action on the "trusted domain".
-- 
Matthias



> 
> That's just my take on all this.
> 
> Cheers
> 
> Carl
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From david.trem at gmail.com  Tue Dec 11 14:25:28 2012
From: david.trem at gmail.com (=?windows-1252?Q?Tr=E9mouilles_David?=)
Date: Tue, 11 Dec 2012 20:25:28 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <32A45AD050C3485C91584666E1C22450@gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
	<32A45AD050C3485C91584666E1C22450@gmail.com>
Message-ID: <50C788A8.3090308@gmail.com>

Indeed using f2py.compile will not solve the problem.
Actually the code was directly inspired form compile.

Any other other suggestion to improve that point ?

David

Le 11/12/12 17:32, Bradley M. Froehle a ?crit :
> I would have suggested using the `numpy.f2py.compile` function directly: https://github.com/numpy/numpy/blob/master/numpy/f2py/__init__.py
>
> ? although it appears to use exec_command as well.
>
> -Brad
>
>
> On Tuesday, December 11, 2012 at 4:16 AM, klo uo wrote:
>
>> Thanks for working on this.
>> On Windows it won't run:
>>   
>> ========================================
>> Could not locate executable C:Python27pythonw.exe
>> Executable C:Python27pythonw.exe does not exist
>> ========================================
>>   
>> Obvious issue with backslash at some point
>>   
>> Is distutil.exec_command only way to do this?
>> On Windows it will look "ugly" as it pops command line window on command execution
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From brad.froehle at gmail.com  Tue Dec 11 14:29:10 2012
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Tue, 11 Dec 2012 11:29:10 -0800
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <50C788A8.3090308@gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
	<32A45AD050C3485C91584666E1C22450@gmail.com>
	<50C788A8.3090308@gmail.com>
Message-ID: <CAHXv-MjPH=RckyQcQTeJBWSc-PxG=i+No53QHJ7EzCQVJhFpVA@mail.gmail.com>

I don't think there is much we can do on our end.  Does f2py even work in
Windows???

-Brad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/2665393a/attachment.html>

From klonuo at gmail.com  Tue Dec 11 14:56:58 2012
From: klonuo at gmail.com (klo uo)
Date: Tue, 11 Dec 2012 20:56:58 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <50C788A8.3090308@gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
	<32A45AD050C3485C91584666E1C22450@gmail.com>
	<50C788A8.3090308@gmail.com>
Message-ID: <CAA-8Ld9ksbjM-aCo2vJZf5oMqyeAjhCSiVP9Dt8tqZtNjMe5KQ@mail.gmail.com>

If you could employ subprocess for execution it can solve the problem


On Tue, Dec 11, 2012 at 8:25 PM, Tr?mouilles David <david.trem at gmail.com>wrote:

> Indeed using f2py.compile will not solve the problem.
> Actually the code was directly inspired form compile.
>
> Any other other suggestion to improve that point ?
>
> David
>
> Le 11/12/12 17:32, Bradley M. Froehle a ?crit :
> > I would have suggested using the `numpy.f2py.compile` function directly:
> https://github.com/numpy/numpy/blob/master/numpy/f2py/__init__.py
> >
> > ? although it appears to use exec_command as well.
> >
> > -Brad
> >
> >
> > On Tuesday, December 11, 2012 at 4:16 AM, klo uo wrote:
> >
> >> Thanks for working on this.
> >> On Windows it won't run:
> >>
> >> ========================================
> >> Could not locate executable C:Python27pythonw.exe
> >> Executable C:Python27pythonw.exe does not exist
> >> ========================================
> >>
> >> Obvious issue with backslash at some point
> >>
> >> Is distutil.exec_command only way to do this?
> >> On Windows it will look "ugly" as it pops command line window on
> command execution
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/b4ad9a09/attachment.html>

From klonuo at gmail.com  Tue Dec 11 14:57:34 2012
From: klonuo at gmail.com (klo uo)
Date: Tue, 11 Dec 2012 20:57:34 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <CAHXv-MjPH=RckyQcQTeJBWSc-PxG=i+No53QHJ7EzCQVJhFpVA@mail.gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
	<32A45AD050C3485C91584666E1C22450@gmail.com>
	<50C788A8.3090308@gmail.com>
	<CAHXv-MjPH=RckyQcQTeJBWSc-PxG=i+No53QHJ7EzCQVJhFpVA@mail.gmail.com>
Message-ID: <CAA-8Ld8UpBTpuTdi3vD1ruxPjqLKYNPT9JAWvvtuAY7QuZHp=Q@mail.gmail.com>

On Tue, Dec 11, 2012 at 8:29 PM, Bradley M. Froehle
<brad.froehle at gmail.com>wrote:

> I don't think there is much we can do on our end.  Does f2py even work in
> Windows???
>


Why do you think it wouldn't work?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/9381b534/attachment.html>

From cekees at gmail.com  Tue Dec 11 16:07:58 2012
From: cekees at gmail.com (Chris Kees)
Date: Tue, 11 Dec 2012 15:07:58 -0600
Subject: [IPython-dev] nbconvert to posters
Message-ID: <CAOVFbFh_RDRLEA-1vTUcLhMD9L3dh2LJoYOTAdjtXMGHzax83Q@mail.gmail.com>

Hi, is anybody using nbconvert or a modified version thereof to generate
notebooks in a  poster format (i.e. for large touch screen displays or to
just print on plotters)?  Also, can nbconvert properly convert animations
for slide presentations? I noticed that nbviewer will correctly load mp4s,
which is nice! -Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/af1a1b1e/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 11 16:11:44 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 11 Dec 2012 22:11:44 +0100
Subject: [IPython-dev] nbconvert to posters
In-Reply-To: <CAOVFbFh_RDRLEA-1vTUcLhMD9L3dh2LJoYOTAdjtXMGHzax83Q@mail.gmail.com>
References: <CAOVFbFh_RDRLEA-1vTUcLhMD9L3dh2LJoYOTAdjtXMGHzax83Q@mail.gmail.com>
Message-ID: <A55B977A-6D0E-4A09-BCD9-03F62C0763EC@gmail.com>


Le 11 d?c. 2012 ? 22:07, Chris Kees a ?crit :

> Hi, is anybody using nbconvert or a modified version thereof to generate notebooks in a  poster format (i.e. for large touch screen displays or to just print on plotters)?  
Html/svg should scale properly, but I haven't tried 

> Also, can nbconvert properly convert animations for slide presentations?

You can have a look at this PR, even if it requires some works to have it working.
https://github.com/ipython/nbconvert/pull/63

> I noticed that nbviewer will correctly load mp4s, which is nice! -Chris


-- 
Matthias





From damianavila at gmail.com  Tue Dec 11 16:24:16 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Tue, 11 Dec 2012 18:24:16 -0300
Subject: [IPython-dev] nbconvert to posters
In-Reply-To: <CAH+mRR2uSnkeUjGHqL51Hy9Qoh0d6J1DuF5aA=Tx4iRpCjdGow@mail.gmail.com>
References: <CAOVFbFh_RDRLEA-1vTUcLhMD9L3dh2LJoYOTAdjtXMGHzax83Q@mail.gmail.com>
	<A55B977A-6D0E-4A09-BCD9-03F62C0763EC@gmail.com>
	<CAH+mRR3c3XQCirVeT5dqZScD91VvcVOykCMk5nZrRHykiG=ocA@mail.gmail.com>
	<CAH+mRR2uSnkeUjGHqL51Hy9Qoh0d6J1DuF5aA=Tx4iRpCjdGow@mail.gmail.com>
Message-ID: <CAH+mRR1UqP+p86wG+TFnv2GmNo-fs-6Dei=ZdwgJFXTLSB8PNQ@mail.gmail.com>

Up to my knowledge, there are some problems getting reveal.js to play well
some videos because off the transitions between slides, but I have to look
deeper (or maybe not use transitions). I will play a little when I get home
and let you know...

Btw, reveal option is now working with with metadata delimiters, so we are
closer than before :-)

Regards.

Dami?n.

El dic 11, 2012 6:11 p.m., "Matthias BUSSONNIER" <
bussonniermatthias at gmail.com> escribi?:


Le 11 d?c. 2012 ? 22:07, Chris Kees a ?crit :


> Hi, is anybody using nbconvert or a modified version thereof to generate
notebooks in a poster f...
Html/svg should scale properly, but I haven't tried


> Also, can nbconvert properly convert animations for slide presentations?
You can have a look at this PR, even if it requires some works to have it
working.
https://github.com/ipython/nbconvert/pull/63


> I noticed that nbviewer will correctly load mp4s, which is nice! -Chris

--
Matthias



_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/77eec5ea/attachment.html>

From fperez.net at gmail.com  Tue Dec 11 17:34:11 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 11 Dec 2012 17:34:11 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
Message-ID: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>

Hi all,

we wanted to give you all some absolutely fantastic news regarding
IPython that we just received...  This also partly explains my silence
during much of this semester, as I was focused on looking for
long-term funding for IPython.

We are proud to announce that we've received funding from the Alfred
P. Sloan foundation (http://sloan.org) that will support IPython
development for the next two years.  Thanks to this $1.15M grant,
Brian Granger and I will be working roughly 3/4 of our time on
IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
Ivanov and Thomas Kluyver will be working as postdocs on the project.
Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
part-time on the development of notebooks for applied statistics in
collaboration with Jonathan Taylor from Stanford (these materials will
be used by Jonathan in the teaching of his applied stats course, the
STAT 191 course - http://www.stanford.edu/class/stats191).  While
Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
located at Berkeley (with frequent visits to Stanford for the
statistics work with Jonathan).

This will also give us resources to support two annual all-hands
development sprints at Berkeley where I hope all our core developers
will be able to attend, as well as funding for cloud resources so that
we can provide paid support for our CI hosting and nbviewer beyond
what the free plans allow (and further services that may arise over
time).

We'd like to particularly thank Josh Greenberg, our program director
at the Sloan foundation, for the phenomenal guidance and support he
provided during the grant proposal preparation and detailed review
process.

Needless to say, we are extremely excited about these news.  We'll
have more announcements to make over the next few weeks, but now I
need to get a lot of administrative wheels in motion quickly, as the
funding officially starts Jan 1, 2013!

All the best,

f


From gael.varoquaux at normalesup.org  Tue Dec 11 17:36:51 2012
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 11 Dec 2012 23:36:51 +0100
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <20121211223651.GF6089@phare.normalesup.org>

Nice achievement! We'll really have an IPython that absolutely rocks.
Congratulations.

Ga?l


From cekees at gmail.com  Tue Dec 11 18:44:31 2012
From: cekees at gmail.com (Chris Kees)
Date: Tue, 11 Dec 2012 17:44:31 -0600
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <20121211223651.GF6089@phare.normalesup.org>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<20121211223651.GF6089@phare.normalesup.org>
Message-ID: <CAOVFbFibqZ+2fJczSkQYLwWZQ+4sR26Z6+fDwSD3F5yHdY8Tnw@mail.gmail.com>

Great news, Fernando! Now if we could just get you a scientific python
distribution that rocks to go along with it.


On Tue, Dec 11, 2012 at 4:36 PM, Gael Varoquaux <
gael.varoquaux at normalesup.org> wrote:

> Nice achievement! We'll really have an IPython that absolutely rocks.
> Congratulations.
>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/755571ae/attachment.html>

From jrocher at enthought.com  Tue Dec 11 18:49:13 2012
From: jrocher at enthought.com (Jonathan Rocher)
Date: Tue, 11 Dec 2012 17:49:13 -0600
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAOVFbFibqZ+2fJczSkQYLwWZQ+4sR26Z6+fDwSD3F5yHdY8Tnw@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<20121211223651.GF6089@phare.normalesup.org>
	<CAOVFbFibqZ+2fJczSkQYLwWZQ+4sR26Z6+fDwSD3F5yHdY8Tnw@mail.gmail.com>
Message-ID: <CAOzk5QfakMch=0rHEHq+CPR719Q5i3JY8PFYBFx88oDBVW_LOA@mail.gmail.com>

This is awesome. Congrats to all of you!

Jonathan


On Tue, Dec 11, 2012 at 5:44 PM, Chris Kees <cekees at gmail.com> wrote:

> Great news, Fernando! Now if we could just get you a scientific python
> distribution that rocks to go along with it.
>
>
> On Tue, Dec 11, 2012 at 4:36 PM, Gael Varoquaux <
> gael.varoquaux at normalesup.org> wrote:
>
>> Nice achievement! We'll really have an IPython that absolutely rocks.
>> Congratulations.
>>
>> Ga?l
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Jonathan Rocher, PhD
Scientific software developer
Enthought, Inc.
jrocher at enthought.com
1-512-536-1057
http://www.enthought.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/82e37b8d/attachment.html>

From damianavila at gmail.com  Tue Dec 11 19:25:57 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Tue, 11 Dec 2012 21:25:57 -0300
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <50C7CF15.7020408@gmail.com>

El 11/12/12 19:34, Fernando Perez escribi?:
> Hi all,
>
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.
>
> We are proud to announce that we've received funding from the Alfred
> P. Sloan foundation (http://sloan.org) that will support IPython
> development for the next two years.  Thanks to this $1.15M grant,
> Brian Granger and I will be working roughly 3/4 of our time on
> IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
> Ivanov and Thomas Kluyver will be working as postdocs on the project.
> Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
> part-time on the development of notebooks for applied statistics in
> collaboration with Jonathan Taylor from Stanford (these materials will
> be used by Jonathan in the teaching of his applied stats course, the
> STAT 191 course -http://www.stanford.edu/class/stats191).  While
> Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
> located at Berkeley (with frequent visits to Stanford for the
> statistics work with Jonathan).
>
> This will also give us resources to support two annual all-hands
> development sprints at Berkeley where I hope all our core developers
> will be able to attend, as well as funding for cloud resources so that
> we can provide paid support for our CI hosting and nbviewer beyond
> what the free plans allow (and further services that may arise over
> time).
>
> We'd like to particularly thank Josh Greenberg, our program director
> at the Sloan foundation, for the phenomenal guidance and support he
> provided during the grant proposal preparation and detailed review
> process.
>
> Needless to say, we are extremely excited about these news.  We'll
> have more announcements to make over the next few weeks, but now I
> need to get a lot of administrative wheels in motion quickly, as the
> funding officially starts Jan 1, 2013!
>
> All the best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Congratulations to all the team!

Regards.

Dami?n.


From jason-sage at creativetrax.com  Tue Dec 11 19:49:33 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 11 Dec 2012 18:49:33 -0600
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <50C7D49D.2050209@creativetrax.com>

On 12/11/12 4:34 PM, Fernando Perez wrote:
> Hi all,
>
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.


Awesome!  Congrats!  I can't wait to see what the goals are--that's 
quite a team, based on the work you guys have done.  Is this part of the 
funding that you envisioned for the multi-user notebook?

Thanks,

Jason




From scopatz at gmail.com  Tue Dec 11 21:08:02 2012
From: scopatz at gmail.com (Anthony Scopatz)
Date: Tue, 11 Dec 2012 20:08:02 -0600
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <50C7D49D.2050209@creativetrax.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C7D49D.2050209@creativetrax.com>
Message-ID: <CAPk-6T6DMOZLRR4X9n0QiBXiTP9Z2Q4dF-J=p=u3GuW5K8cnGA@mail.gmail.com>

Woohoo! Congrats!


On Tue, Dec 11, 2012 at 6:49 PM, Jason Grout <jason-sage at creativetrax.com>wrote:

> On 12/11/12 4:34 PM, Fernando Perez wrote:
> > Hi all,
> >
> > we wanted to give you all some absolutely fantastic news regarding
> > IPython that we just received...  This also partly explains my silence
> > during much of this semester, as I was focused on looking for
> > long-term funding for IPython.
>
>
> Awesome!  Congrats!  I can't wait to see what the goals are--that's
> quite a team, based on the work you guys have done.  Is this part of the
> funding that you envisioned for the multi-user notebook?
>
> Thanks,
>
> Jason
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/d14c657a/attachment.html>

From massimodisasha at gmail.com  Tue Dec 11 21:21:11 2012
From: massimodisasha at gmail.com (epi)
Date: Tue, 11 Dec 2012 21:21:11 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
	stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAOzk5QfakMch=0rHEHq+CPR719Q5i3JY8PFYBFx88oDBVW_LOA@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<20121211223651.GF6089@phare.normalesup.org>
	<CAOVFbFibqZ+2fJczSkQYLwWZQ+4sR26Z6+fDwSD3F5yHdY8Tnw@mail.gmail.com>
	<CAOzk5QfakMch=0rHEHq+CPR719Q5i3JY8PFYBFx88oDBVW_LOA@mail.gmail.com>
Message-ID: <7D4DF50C-5268-46D8-A201-8DDC3A77D8DE@gmail.com>


That's a good new for a Better Science!
Congratulation to all of You my friends!!!

Il giorno 11/dic/2012, alle ore 18:49, Jonathan Rocher <jrocher at enthought.com> ha scritto:

> This is awesome. Congrats to all of you!
> 
> Jonathan
> 
> 
> On Tue, Dec 11, 2012 at 5:44 PM, Chris Kees <cekees at gmail.com> wrote:
> Great news, Fernando! Now if we could just get you a scientific python distribution that rocks to go along with it. 
> 
> 
> On Tue, Dec 11, 2012 at 4:36 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:
> Nice achievement! We'll really have an IPython that absolutely rocks.
> Congratulations.
> 
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> 
> -- 
> Jonathan Rocher, PhD
> Scientific software developer
> Enthought, Inc.
> jrocher at enthought.com
> 1-512-536-1057
> http://www.enthought.com
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/ec064bd1/attachment.html>

From d.warde.farley at gmail.com  Tue Dec 11 22:25:03 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Tue, 11 Dec 2012 22:25:03 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <CALc6Xo75Udd1Ccm7neE6frri1rV6qrNuM+MZKRv_X1c3eFi4hQ@mail.gmail.com>

Utterly fantastic news! Congratulations to everyone, and I look
forward to watching the magic unfold (and pitching in where I can).

On Tue, Dec 11, 2012 at 5:34 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.
>
> We are proud to announce that we've received funding from the Alfred
> P. Sloan foundation (http://sloan.org) that will support IPython
> development for the next two years.  Thanks to this $1.15M grant,
> Brian Granger and I will be working roughly 3/4 of our time on
> IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
> Ivanov and Thomas Kluyver will be working as postdocs on the project.
> Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
> part-time on the development of notebooks for applied statistics in
> collaboration with Jonathan Taylor from Stanford (these materials will
> be used by Jonathan in the teaching of his applied stats course, the
> STAT 191 course - http://www.stanford.edu/class/stats191).  While
> Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
> located at Berkeley (with frequent visits to Stanford for the
> statistics work with Jonathan).
>
> This will also give us resources to support two annual all-hands
> development sprints at Berkeley where I hope all our core developers
> will be able to attend, as well as funding for cloud resources so that
> we can provide paid support for our CI hosting and nbviewer beyond
> what the free plans allow (and further services that may arise over
> time).
>
> We'd like to particularly thank Josh Greenberg, our program director
> at the Sloan foundation, for the phenomenal guidance and support he
> provided during the grant proposal preparation and detailed review
> process.
>
> Needless to say, we are extremely excited about these news.  We'll
> have more announcements to make over the next few weeks, but now I
> need to get a lot of administrative wheels in motion quickly, as the
> funding officially starts Jan 1, 2013!
>
> All the best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From satra at mit.edu  Tue Dec 11 22:33:38 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Tue, 11 Dec 2012 22:33:38 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <CA+A4wOk2ve2QLWPLPC84c3_NGHFcVtgxRexN5AtBF4JzMogYnQ@mail.gmail.com>

hi fernando,

it was great to hear the news in person and many congratulations to the
entire ipython team.

cheers,

satra

On Tue, Dec 11, 2012 at 5:34 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi all,
>
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.
>
> We are proud to announce that we've received funding from the Alfred
> P. Sloan foundation (http://sloan.org) that will support IPython
> development for the next two years.  Thanks to this $1.15M grant,
> Brian Granger and I will be working roughly 3/4 of our time on
> IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
> Ivanov and Thomas Kluyver will be working as postdocs on the project.
> Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
> part-time on the development of notebooks for applied statistics in
> collaboration with Jonathan Taylor from Stanford (these materials will
> be used by Jonathan in the teaching of his applied stats course, the
> STAT 191 course - http://www.stanford.edu/class/stats191).  While
> Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
> located at Berkeley (with frequent visits to Stanford for the
> statistics work with Jonathan).
>
> This will also give us resources to support two annual all-hands
> development sprints at Berkeley where I hope all our core developers
> will be able to attend, as well as funding for cloud resources so that
> we can provide paid support for our CI hosting and nbviewer beyond
> what the free plans allow (and further services that may arise over
> time).
>
> We'd like to particularly thank Josh Greenberg, our program director
> at the Sloan foundation, for the phenomenal guidance and support he
> provided during the grant proposal preparation and detailed review
> process.
>
> Needless to say, we are extremely excited about these news.  We'll
> have more announcements to make over the next few weeks, but now I
> need to get a lot of administrative wheels in motion quickly, as the
> funding officially starts Jan 1, 2013!
>
> All the best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121211/2dbb1ff3/attachment.html>

From pasokan at gmail.com  Tue Dec 11 22:56:09 2012
From: pasokan at gmail.com (Asokan Pichai)
Date: Wed, 12 Dec 2012 09:26:09 +0530
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CA+A4wOk2ve2QLWPLPC84c3_NGHFcVtgxRexN5AtBF4JzMogYnQ@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<CA+A4wOk2ve2QLWPLPC84c3_NGHFcVtgxRexN5AtBF4JzMogYnQ@mail.gmail.com>
Message-ID: <CAB3TsChauZ2sP4XngN1rK2d=b2+mOur5eMcN+0DJFUwfiO9vZA@mail.gmail.com>

Congrats Fernando and the iPy team



On 12 December 2012 09:03, Satrajit Ghosh <satra at mit.edu> wrote:

> hi fernando,
>
> it was great to hear the news in person and many congratulations to the
> entire ipython team.
>
> cheers,
>
> satra
>
> On Tue, Dec 11, 2012 at 5:34 PM, Fernando Perez <fperez.net at gmail.com>wrote:
>
>> Hi all,
>>
>> we wanted to give you all some absolutely fantastic news regarding
>> IPython that we just received...  This also partly explains my silence
>> during much of this semester, as I was focused on looking for
>> long-term funding for IPython.
>>
>> We are proud to announce that we've received funding from the Alfred
>> P. Sloan foundation (http://sloan.org) that will support IPython
>> development for the next two years.  Thanks to this $1.15M grant,
>> Brian Granger and I will be working roughly 3/4 of our time on
>> IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
>> Ivanov and Thomas Kluyver will be working as postdocs on the project.
>> Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
>> part-time on the development of notebooks for applied statistics in
>> collaboration with Jonathan Taylor from Stanford (these materials will
>> be used by Jonathan in the teaching of his applied stats course, the
>> STAT 191 course - http://www.stanford.edu/class/stats191).  While
>> Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
>> located at Berkeley (with frequent visits to Stanford for the
>> statistics work with Jonathan).
>>
>> This will also give us resources to support two annual all-hands
>> development sprints at Berkeley where I hope all our core developers
>> will be able to attend, as well as funding for cloud resources so that
>> we can provide paid support for our CI hosting and nbviewer beyond
>> what the free plans allow (and further services that may arise over
>> time).
>>
>> We'd like to particularly thank Josh Greenberg, our program director
>> at the Sloan foundation, for the phenomenal guidance and support he
>> provided during the grant proposal preparation and detailed review
>> process.
>>
>> Needless to say, we are extremely excited about these news.  We'll
>> have more announcements to make over the next few weeks, but now I
>> need to get a lot of administrative wheels in motion quickly, as the
>> funding officially starts Jan 1, 2013!
>>
>> All the best,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Asokan Pichai
*-------------------*
We will find a way. Or, make one. (Hannibal)

*To find everything profound ? that is an inconvenient trait.* It makes one
strain one's eyes all the time, and in the end one finds more than one
might have wished. -- Nietzsche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121212/f531976a/attachment.html>

From Nicolas.Rougier at inria.fr  Wed Dec 12 02:26:48 2012
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Wed, 12 Dec 2012 08:26:48 +0100
Subject: [IPython-dev] Some excellent news for IPython: we've received
	stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAB3TsChauZ2sP4XngN1rK2d=b2+mOur5eMcN+0DJFUwfiO9vZA@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<CA+A4wOk2ve2QLWPLPC84c3_NGHFcVtgxRexN5AtBF4JzMogYnQ@mail.gmail.com>
	<CAB3TsChauZ2sP4XngN1rK2d=b2+mOur5eMcN+0DJFUwfiO9vZA@mail.gmail.com>
Message-ID: <50207C6F-35D9-488B-97DD-9ADA7CA8B0EC@inria.fr>



Fantastic ! Congratulations !

Looking at the great work you've already done without such grant, and I can't even imagine what will come out of it !

Cheers,
Nicolas

On Dec 12, 2012, at 4:56 , Asokan Pichai wrote:

> Congrats Fernando and the iPy team
> 
> 
> 
> On 12 December 2012 09:03, Satrajit Ghosh <satra at mit.edu> wrote:
> hi fernando,
> 
> it was great to hear the news in person and many congratulations to the entire ipython team.
> 
> cheers,
> 
> satra
> 
> On Tue, Dec 11, 2012 at 5:34 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
> 
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.
> 
> We are proud to announce that we've received funding from the Alfred
> P. Sloan foundation (http://sloan.org) that will support IPython
> development for the next two years.  Thanks to this $1.15M grant,
> Brian Granger and I will be working roughly 3/4 of our time on
> IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
> Ivanov and Thomas Kluyver will be working as postdocs on the project.
> Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
> part-time on the development of notebooks for applied statistics in
> collaboration with Jonathan Taylor from Stanford (these materials will
> be used by Jonathan in the teaching of his applied stats course, the
> STAT 191 course - http://www.stanford.edu/class/stats191).  While
> Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
> located at Berkeley (with frequent visits to Stanford for the
> statistics work with Jonathan).
> 
> This will also give us resources to support two annual all-hands
> development sprints at Berkeley where I hope all our core developers
> will be able to attend, as well as funding for cloud resources so that
> we can provide paid support for our CI hosting and nbviewer beyond
> what the free plans allow (and further services that may arise over
> time).
> 
> We'd like to particularly thank Josh Greenberg, our program director
> at the Sloan foundation, for the phenomenal guidance and support he
> provided during the grant proposal preparation and detailed review
> process.
> 
> Needless to say, we are extremely excited about these news.  We'll
> have more announcements to make over the next few weeks, but now I
> need to get a lot of administrative wheels in motion quickly, as the
> funding officially starts Jan 1, 2013!
> 
> All the best,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> 
> -- 
> Asokan Pichai
> *-------------------*
> We will find a way. Or, make one. (Hannibal)
> 
> To find everything profound ? that is an inconvenient trait. It makes one strain one's eyes all the time, and in the end one finds more than one might have wished. -- Nietzsche
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From nicolas at pettiaux.be  Wed Dec 12 02:54:01 2012
From: nicolas at pettiaux.be (Nicolas Pettiaux)
Date: Wed, 12 Dec 2012 08:54:01 +0100
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <50207C6F-35D9-488B-97DD-9ADA7CA8B0EC@inria.fr>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<CA+A4wOk2ve2QLWPLPC84c3_NGHFcVtgxRexN5AtBF4JzMogYnQ@mail.gmail.com>
	<CAB3TsChauZ2sP4XngN1rK2d=b2+mOur5eMcN+0DJFUwfiO9vZA@mail.gmail.com>
	<50207C6F-35D9-488B-97DD-9ADA7CA8B0EC@inria.fr>
Message-ID: <CAEHX+g=WTkBijpvA02fyMCFp3vHJ9uuzWz+N3EPbek6dKAKN0g@mail.gmail.com>

Congratulations and much thanks for sharing this excellent news with us.

I propose to celebrate with some good Belgian beers and chocolate at
next EuroScipy in Brussels in August 2013.

I'll make myself the chocolate cake.

Nicolas P

--
Nicolas Pettiaux, dr. sc - gsm : 0496 24 55 01 - rmll.be - wlsm.be
Rencontres mondiales du logiciel et de la culture libre - Bruxelles, 6-11/7/2013
World libre software and culture meeting - Brussels,  July 6 to 11, 2013
Lepacte.be - ? promouvoir les libert?s num?riques en Belgique ? - hetpact.be


From jason-sage at creativetrax.com  Wed Dec 12 09:37:55 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 12 Dec 2012 08:37:55 -0600
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <50C896C3.1090209@creativetrax.com>

On 12/11/12 4:34 PM, Fernando Perez wrote:
> Hi all,
>
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.

It looks like you guys are getting some good publicity on this:

http://news.ycombinator.com/item?id=4909070

Has anyone submitted this to slashdot?  Are there plans to put up a 
press release on the front page of ipython.org (listing the goals, 
etc.)?  This is the sort of announcement you can publicize quite a bit, 
but of course the publicizing should be done right away while the news 
is fresh.

Thanks,

Jason



From jason-sage at creativetrax.com  Wed Dec 12 10:50:13 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 12 Dec 2012 09:50:13 -0600
Subject: [IPython-dev] IPEP 8: Custom messages and message handlers
Message-ID: <50C8A7B5.7020204@creativetrax.com>

Hi everyone,

I just posted an initial draft of an IPEP about adding custom message 
handlers on the server side (which also necessitates custom messages).

https://github.com/ipython/ipython/wiki/IPEP-8:-Custom-messages-and-message-handlers

With that, I'm curious about the workflow for IPEPs.  Some IPEPs seem to 
use wiki pages, while others just directly use issues.  The nice thing 
about issues is that there is a way to have a conversation directly on 
the IPEP.  The bad thing is that the changes to an issue IPEP are not 
recorded.

Regardless, comments are welcome!

Thanks,

Jason


From jorgen.stenarson at kroywen.se  Wed Dec 12 11:27:38 2012
From: jorgen.stenarson at kroywen.se (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Wed, 12 Dec 2012 17:27:38 +0100
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <50C8B07A.2040905@kroywen.se>

Congratulations! I imagine it was a lot of work to put the proposal 
together. I'm looking forward to hearing more about the plans.

/J?rgen


From takowl at gmail.com  Wed Dec 12 11:32:26 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 12 Dec 2012 16:32:26 +0000
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <50C896C3.1090209@creativetrax.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C896C3.1090209@creativetrax.com>
Message-ID: <CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>

On 12 December 2012 14:37, Jason Grout <jason-sage at creativetrax.com> wrote:

> Has anyone submitted this to slashdot?  Are there plans to put up a
> press release on the front page of ipython.org (listing the goals,
> etc.)?  This is the sort of announcement you can publicize quite a bit,
> but of course the publicizing should be done right away while the news
> is fresh.
>

Good point. Fernando, do you want to write something for the website? Or I
can draft something for you to check - I don't feel familiar enough with
the content of the grant application to just write the announcement.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121212/48a6c0e4/attachment.html>

From ellisonbg at gmail.com  Wed Dec 12 11:42:12 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 12 Dec 2012 08:42:12 -0800
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C896C3.1090209@creativetrax.com>
	<CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>
Message-ID: <CAH4pYpQMkTWtpTAZ-nLyEiQ8_w3E5zvrjB0fP95VhwFMHKQ9wA@mail.gmail.com>

Fernando and I will try to draft something for the website today.

Cheers,

Brian

On Wed, Dec 12, 2012 at 8:32 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 12 December 2012 14:37, Jason Grout <jason-sage at creativetrax.com> wrote:
>>
>> Has anyone submitted this to slashdot?  Are there plans to put up a
>> press release on the front page of ipython.org (listing the goals,
>> etc.)?  This is the sort of announcement you can publicize quite a bit,
>> but of course the publicizing should be done right away while the news
>> is fresh.
>
>
> Good point. Fernando, do you want to write something for the website? Or I
> can draft something for you to check - I don't feel familiar enough with the
> content of the grant application to just write the announcement.
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Dec 12 11:44:59 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 12 Dec 2012 08:44:59 -0800
Subject: [IPython-dev] IPEP 8: Custom messages and message handlers
In-Reply-To: <50C8A7B5.7020204@creativetrax.com>
References: <50C8A7B5.7020204@creativetrax.com>
Message-ID: <CAH4pYpSdEJODzydUo0e5bL0Z2+V23YQa+rOykoxGf7At2wL_Sw@mail.gmail.com>

Great, thanks for getting this started.  This is one of the main tasks
of the grant, so we will start to work on this a lot in Jan.

As far as the IPEP, I am not very happy with how we are running IPEP
discussions.  I think that having them on the mailing list makes them
difficult to find and track along with the IPEP text.  I would
probably prefer to use github issues to track the discussion and the
wiki for the document.

On Wed, Dec 12, 2012 at 7:50 AM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> Hi everyone,
>
> I just posted an initial draft of an IPEP about adding custom message
> handlers on the server side (which also necessitates custom messages).
>
> https://github.com/ipython/ipython/wiki/IPEP-8:-Custom-messages-and-message-handlers
>
> With that, I'm curious about the workflow for IPEPs.  Some IPEPs seem to
> use wiki pages, while others just directly use issues.  The nice thing
> about issues is that there is a way to have a conversation directly on
> the IPEP.  The bad thing is that the changes to an issue IPEP are not
> recorded.
>
> Regardless, comments are welcome!
>
> Thanks,
>
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Dec 12 11:48:03 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 12 Dec 2012 08:48:03 -0800
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAH4pYpQMkTWtpTAZ-nLyEiQ8_w3E5zvrjB0fP95VhwFMHKQ9wA@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C896C3.1090209@creativetrax.com>
	<CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>
	<CAH4pYpQMkTWtpTAZ-nLyEiQ8_w3E5zvrjB0fP95VhwFMHKQ9wA@mail.gmail.com>
Message-ID: <CAH4pYpSuF9Xfd6zGzD5PmO8=kYFs6pU3UG5x+razoMfhBEGbZg@mail.gmail.com>

Also, thanks everyone for your support, we deeply appreciate it and
are thrilled about this opportunity.

Cheers,

Brian

On Wed, Dec 12, 2012 at 8:42 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> Fernando and I will try to draft something for the website today.
>
> Cheers,
>
> Brian
>
> On Wed, Dec 12, 2012 at 8:32 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> On 12 December 2012 14:37, Jason Grout <jason-sage at creativetrax.com> wrote:
>>>
>>> Has anyone submitted this to slashdot?  Are there plans to put up a
>>> press release on the front page of ipython.org (listing the goals,
>>> etc.)?  This is the sort of announcement you can publicize quite a bit,
>>> but of course the publicizing should be done right away while the news
>>> is fresh.
>>
>>
>> Good point. Fernando, do you want to write something for the website? Or I
>> can draft something for you to check - I don't feel familiar enough with the
>> content of the grant application to just write the announcement.
>>
>> Thanks,
>> Thomas
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Dec 12 12:46:25 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 12 Dec 2012 09:46:25 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
Message-ID: <CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>

OK trying to catch up with this thread...

* Matthias brings up a great point that we need to consider
forged/hostile notebooks that were not created with our notebook
server.  The solution to this is to make sure that our notebook server
can "clean" notebooks before displaying them.  The solutions that I
posted above will address this completely and allow people to simply
open notebooks from the web without worrying about them.

* I still feel like I don't have a straight answer to my question:
will the solutions I proposed solve all of the security problems while
allowing us to serve authenticated notebooks on single domains?  If
now, what security problems would remain?

* I am confused about the multiple domain solution that is being
proposed.  The idea is that notebooks containing arbitrary Javascript
would be served from a separate domain that does not offer any
authentication.  The authenticated notebooks would live on a second
domain that doesn't allow Javascript.  But then how is a user supposed
to author a notebook with Javascript and have that notebook not be
anonymous?  What if a user wants to author a notebook with Javascript
that needs to be private?  But my biggest complaint about this is that
it completely cripples one of the most important features of the
notebook - the possibility of sharing notebooks+kernels with other
people.  These features are infinitely more important that allowing
arbitrary Javascript in the notebook.

* I don't see any fundamental different between "static notebooks" and
"notebooks with kernels" - both can have Javascript and the new
Javascript plugins will work on both.  Both have the same overall
security issues and I don't think it makes sense to try and handle
them separately.

Cheers,

Brian




On Tue, Dec 11, 2012 at 9:15 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 11 d?c. 2012 ? 16:10, Carl Smith a ?crit :
>
>> Hi Brian
>>>
>>> The idea is that the extra Javascript cool-stuff will be installed by
>>> the person who runs the notebook server once and for all notebooks on
>>> that server.  Similar to how python packages are installed = you do
>>> this before you start python.  To get data from python to the
>>> Javascript plugins we will use JSON objects and trigger the callbacks
>>> to handle them.
>>
>> This seems to be dependent on a kernel, which static notebooks don't
>> have? If I generate a static notebook, which is just a web page, then
>> post that page to a hosting service, or email it to someone, how would
>> the plugins work? Maybe we're looking at two slightly different
>> scenarios. I'm focussed on static views only. The host should not have
>> to allow anything more than posting and getting HTML documents.
>
> IIRC, you can embed several repr in the ipynb file.
> So you could provide a plugin that can "render" object on static view.
> (like d3.js graph, you don't need the kernel to do that)
>
>> ================
>>
>> Hi Matthias
>>
>>> Static notebooks, served from a different domain, could be rendered
>>> inside iframes, enabling us to embed them inside other webpages and
>>> applications. These notebooks would still be superficially served by
>>> our own servers, so the UX wouldn't be effected.
>>>
>>> keep in mind that iframe are not sandboxed, and you can inject js on parent
>>> frame.
>>> Unless you use the sandbox attributes, which is part of html5 but not
>>> implemented in every
>>> browser? And not yet infallible, it is more a "we'll help you embed other
>>> pages by providing a separate
>>> js namespace but we don't guaranty yes that the VM is unbreachable"
>>
>
>> I pretty sure iframes are sandboxed in the sense that a parent page
>> and an iframe can not communicate unless they have the same origin,
>> and this is an old feature. The new sandbox attribute in HTML5 is for
>> a different purpose.
>
> But this is still kind of a problem, as usually the static view will be served from the "same origin"
> as the rest of the website.
>
> If you don't want to make a notebook fully public, you have to have some kind of authentication
> that allows you to load it.
>
> I'm still doubt a little about what frames are supposed to do and what they actually do.
> I'm not an expert on that, but it is still worth digging.
>
>>> Responsible disclosure don't want to say much more but but having a
>>> statically display
>>> notebook is often link to having a "sharing/import" button which is
>>> dangerous.
>>> And could lead to self propagating notebook through account that can infect
>>> other
>>> notebooks, or share itself on twitter...
>>
>> Any buttons, like for importing a notebook, would live in the parent
>> page and would have no access to, nor allow access from, the iframe.
>> The parent page would know which static notebook it embeds in the
>> iframe though, so it could provide buttons that connect to the actual
>> notebook in question, which is a totally different file to the static
>> notebook being rendered in the iframe anyway.
>
> I understand what you want to do, I guess the definition of "static" is blurry.
> If you want a perfectly static version (does it make sense in html) you can go with iframe.
> If you want the ability to comment on a particular cell, then you have to build iframe for
> every cell.
> and you lose the ability to comment "inline" as github does.
>
>
>>
>>> Multi domain is a real good idea. I have a clear view in my head on how we
>>> could use that in a way close to OAuth to allow javascript by still having
>>> "logged-in" users.
>>> It wouldn't be as seamless a something like github, but close.
>>
>> I think we're looking at things differently: You seem to be
>> considering static views as something generated on the fly and on
>> demand, nbviewer style. I'm thinking about running nbconvert on a
>> notebook, then keeping the output as a webpage to be copied and passed
>> around freely. Once the static notebook exists, it's a done deal.
>> There's no chance of any changes to IPython breaking it. It's a
>> independent webpage. Updating it would amount to deleting it and
>> replacing it with a new version.
>
> I don't think those are quite different.
> You can have a "perfectly static" version that embeds bad js and require some kind of authentication to be seen.
> The line between "on the fly" and static is thin.
>
>>
>>> The **big** question is:
>>> Are viewer logged in (in any way) to the given server, and if so do they
>>> have the right to do anything else with those credentials ?
>>> If it is just a public notebook viewer, then it's fine.
>>>
>>> If you want something more "interactive" (sharing/ permissions?etc, and the
>>> display any JS ) you won't have much choice.
>>> Or you will have a painful multi-login.
>>
>> I'm very much against hosting user submitted notebooks on any domain
>> with cookie based authentication. It needs to be divided into 'trusted
>> domain', where no user's JS will ever be served, and 'hosting domain'
>> that has no account system of it's own. The trusted domain would
>> control the hosting domain, as a kind of slave.
>
> Yep, kind of what I have in mind.
> The hosting domain can have "tokens"
> Publish this comment on this notebook on the behalf of ...
> The you have to "validate" those action on the "trusted domain".
> --
> Matthias
>
>
>
>>
>> That's just my take on all this.
>>
>> Cheers
>>
>> Carl
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Dec 12 12:55:03 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 12 Dec 2012 09:55:03 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
Message-ID: <CAH4pYpQnzK4XM448pF1hkmTuXAkPZ=Y-OpaVbWDeLJfU7HWBFQ@mail.gmail.com>

Another comment about the current Javascript repr implementation that
uses eval.  I have written a number of Javascript reprs that use this
older approach and it is a horrific experience - trying to debug code
passed to eval is almost impossible - all you know is that it fails,
you don't get any traceback to useful information.  The fact that very
few people have used this reinforces this data - it is just too hard
for users.  This was a major reason for my developing the JS plugin
idea - it is a *much* saner way of developing custom JS code that
doesn't have any of the security problems.

Cheers,

Brian

On Wed, Dec 12, 2012 at 9:46 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> OK trying to catch up with this thread...
>
> * Matthias brings up a great point that we need to consider
> forged/hostile notebooks that were not created with our notebook
> server.  The solution to this is to make sure that our notebook server
> can "clean" notebooks before displaying them.  The solutions that I
> posted above will address this completely and allow people to simply
> open notebooks from the web without worrying about them.
>
> * I still feel like I don't have a straight answer to my question:
> will the solutions I proposed solve all of the security problems while
> allowing us to serve authenticated notebooks on single domains?  If
> now, what security problems would remain?
>
> * I am confused about the multiple domain solution that is being
> proposed.  The idea is that notebooks containing arbitrary Javascript
> would be served from a separate domain that does not offer any
> authentication.  The authenticated notebooks would live on a second
> domain that doesn't allow Javascript.  But then how is a user supposed
> to author a notebook with Javascript and have that notebook not be
> anonymous?  What if a user wants to author a notebook with Javascript
> that needs to be private?  But my biggest complaint about this is that
> it completely cripples one of the most important features of the
> notebook - the possibility of sharing notebooks+kernels with other
> people.  These features are infinitely more important that allowing
> arbitrary Javascript in the notebook.
>
> * I don't see any fundamental different between "static notebooks" and
> "notebooks with kernels" - both can have Javascript and the new
> Javascript plugins will work on both.  Both have the same overall
> security issues and I don't think it makes sense to try and handle
> them separately.
>
> Cheers,
>
> Brian
>
>
>
>
> On Tue, Dec 11, 2012 at 9:15 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>>
>> Le 11 d?c. 2012 ? 16:10, Carl Smith a ?crit :
>>
>>> Hi Brian
>>>>
>>>> The idea is that the extra Javascript cool-stuff will be installed by
>>>> the person who runs the notebook server once and for all notebooks on
>>>> that server.  Similar to how python packages are installed = you do
>>>> this before you start python.  To get data from python to the
>>>> Javascript plugins we will use JSON objects and trigger the callbacks
>>>> to handle them.
>>>
>>> This seems to be dependent on a kernel, which static notebooks don't
>>> have? If I generate a static notebook, which is just a web page, then
>>> post that page to a hosting service, or email it to someone, how would
>>> the plugins work? Maybe we're looking at two slightly different
>>> scenarios. I'm focussed on static views only. The host should not have
>>> to allow anything more than posting and getting HTML documents.
>>
>> IIRC, you can embed several repr in the ipynb file.
>> So you could provide a plugin that can "render" object on static view.
>> (like d3.js graph, you don't need the kernel to do that)
>>
>>> ================
>>>
>>> Hi Matthias
>>>
>>>> Static notebooks, served from a different domain, could be rendered
>>>> inside iframes, enabling us to embed them inside other webpages and
>>>> applications. These notebooks would still be superficially served by
>>>> our own servers, so the UX wouldn't be effected.
>>>>
>>>> keep in mind that iframe are not sandboxed, and you can inject js on parent
>>>> frame.
>>>> Unless you use the sandbox attributes, which is part of html5 but not
>>>> implemented in every
>>>> browser? And not yet infallible, it is more a "we'll help you embed other
>>>> pages by providing a separate
>>>> js namespace but we don't guaranty yes that the VM is unbreachable"
>>>
>>
>>> I pretty sure iframes are sandboxed in the sense that a parent page
>>> and an iframe can not communicate unless they have the same origin,
>>> and this is an old feature. The new sandbox attribute in HTML5 is for
>>> a different purpose.
>>
>> But this is still kind of a problem, as usually the static view will be served from the "same origin"
>> as the rest of the website.
>>
>> If you don't want to make a notebook fully public, you have to have some kind of authentication
>> that allows you to load it.
>>
>> I'm still doubt a little about what frames are supposed to do and what they actually do.
>> I'm not an expert on that, but it is still worth digging.
>>
>>>> Responsible disclosure don't want to say much more but but having a
>>>> statically display
>>>> notebook is often link to having a "sharing/import" button which is
>>>> dangerous.
>>>> And could lead to self propagating notebook through account that can infect
>>>> other
>>>> notebooks, or share itself on twitter...
>>>
>>> Any buttons, like for importing a notebook, would live in the parent
>>> page and would have no access to, nor allow access from, the iframe.
>>> The parent page would know which static notebook it embeds in the
>>> iframe though, so it could provide buttons that connect to the actual
>>> notebook in question, which is a totally different file to the static
>>> notebook being rendered in the iframe anyway.
>>
>> I understand what you want to do, I guess the definition of "static" is blurry.
>> If you want a perfectly static version (does it make sense in html) you can go with iframe.
>> If you want the ability to comment on a particular cell, then you have to build iframe for
>> every cell.
>> and you lose the ability to comment "inline" as github does.
>>
>>
>>>
>>>> Multi domain is a real good idea. I have a clear view in my head on how we
>>>> could use that in a way close to OAuth to allow javascript by still having
>>>> "logged-in" users.
>>>> It wouldn't be as seamless a something like github, but close.
>>>
>>> I think we're looking at things differently: You seem to be
>>> considering static views as something generated on the fly and on
>>> demand, nbviewer style. I'm thinking about running nbconvert on a
>>> notebook, then keeping the output as a webpage to be copied and passed
>>> around freely. Once the static notebook exists, it's a done deal.
>>> There's no chance of any changes to IPython breaking it. It's a
>>> independent webpage. Updating it would amount to deleting it and
>>> replacing it with a new version.
>>
>> I don't think those are quite different.
>> You can have a "perfectly static" version that embeds bad js and require some kind of authentication to be seen.
>> The line between "on the fly" and static is thin.
>>
>>>
>>>> The **big** question is:
>>>> Are viewer logged in (in any way) to the given server, and if so do they
>>>> have the right to do anything else with those credentials ?
>>>> If it is just a public notebook viewer, then it's fine.
>>>>
>>>> If you want something more "interactive" (sharing/ permissions?etc, and the
>>>> display any JS ) you won't have much choice.
>>>> Or you will have a painful multi-login.
>>>
>>> I'm very much against hosting user submitted notebooks on any domain
>>> with cookie based authentication. It needs to be divided into 'trusted
>>> domain', where no user's JS will ever be served, and 'hosting domain'
>>> that has no account system of it's own. The trusted domain would
>>> control the hosting domain, as a kind of slave.
>>
>> Yep, kind of what I have in mind.
>> The hosting domain can have "tokens"
>> Publish this comment on this notebook on the behalf of ...
>> The you have to "validate" those action on the "trusted domain".
>> --
>> Matthias
>>
>>
>>
>>>
>>> That's just my take on all this.
>>>
>>> Cheers
>>>
>>> Carl
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Wed Dec 12 14:10:52 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 12 Dec 2012 14:10:52 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAH4pYpSuF9Xfd6zGzD5PmO8=kYFs6pU3UG5x+razoMfhBEGbZg@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C896C3.1090209@creativetrax.com>
	<CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>
	<CAH4pYpQMkTWtpTAZ-nLyEiQ8_w3E5zvrjB0fP95VhwFMHKQ9wA@mail.gmail.com>
	<CAH4pYpSuF9Xfd6zGzD5PmO8=kYFs6pU3UG5x+razoMfhBEGbZg@mail.gmail.com>
Message-ID: <CAHAreOoi3ing1B_8g3La2ucJnakUQ7mQ3fiy9EvUZ059yu_yPQ@mail.gmail.com>

I'm at an NSF workshop in Providence and have near-zero bandwidth
today, so it would  be great if Thomas can give us a hand with the
site upload.  We just shared the draft with you on gdocs...

thanks!!


From bussonniermatthias at gmail.com  Wed Dec 12 14:37:51 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 12 Dec 2012 20:37:51 +0100
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
Message-ID: <40D15F76-8012-4350-9135-060C9F352A51@gmail.com>


Le 12 d?c. 2012 ? 18:46, Brian Granger a ?crit :

> OK trying to catch up with this thread...
> 
> * Matthias brings up a great point that we need to consider
> forged/hostile notebooks that were not created with our notebook
> server.  The solution to this is to make sure that our notebook server
> can "clean" notebooks before displaying them.  The solutions that I
> posted above will address this completely and allow people to simply
> open notebooks from the web without worrying about them.
> 
> * I still feel like I don't have a straight answer to my question:
> will the solutions I proposed solve all of the security problems while
> allowing us to serve authenticated notebooks on single domains?  If
> now, what security problems would remain?
> 
>> 
>> To fix this, we need to enable the HTML sanitizer that comes with the
>> JS Markdown rendered that we are using.  This is what StackOverflow
>> uses to sanitize their markdown and should completely remove any
>> security risks coming from within markdown cells.
> 
> 
>> * In CodeCell output, the Javascript repr is dynamically passed
>> into eval.  This only happens when code is run, not when the notebook
>> is loaded, so it is less critical, but still needs to be fixed.
> 
>> Oh, yes forgot about that, we will have to clean that HTML as well.
> 
Which can be resumes in clean html/js in all output. 
(as rendered md cell are actually "output")

Then yes.


> 
> * I am confused about the multiple domain solution that is being
> proposed.  The idea is that notebooks containing arbitrary Javascript
> would be served from a separate domain that does not offer any
> authentication.  The authenticated notebooks would live on a second
> domain that doesn't allow Javascript.  But then how is a user supposed
> to author a notebook with Javascript and have that notebook not be
> anonymous?  What if a user wants to author a notebook with Javascript
> that needs to be private? 

let see. 
Multi domain just allow to have fine-grained privileges in cookies.

You can login to wordpress with your google account right ? 
But then it only have access to your identity, not your mails right ?

Then you do the same under the hood with subdomains. 
(for the sake of simplicity i'll use domains names that have meanings, and a simplified logic.) 

user logs in https://iamroot.ipython.org.
He/She have acces to his/her dashboard. 

Get to notification "Other User want you to see a notebook QUX [open]"

click on Open opens the notebook in another tabs.

https://comment-only.ipython.org/view/QUX.ipynb
The user is still "logged in", but any request to https://iamroot.ipython.org will fail because of xss policies. 
And obviously the only thing malicious JS could do is post comment on the current notebook.

The point is to restrain the number of actions available from a certain domain name. 
As you control the authorization of a domain name server side, nothing prevent a user to grant more rights to a specific "domain" or notebook. 

It does not prevent collaborating in anyways, 
But if you share kernel you share filesystem, so there are no point in really using subdomain at this stage.
Still it gives another layer of security as if  there is a way to inject javascript, this javascript will not find any
malicious things to do.

Or it allows you "test" untrusted plugins..

I let you imagine other stuff. 
-- 
Matthias



> * I don't see any fundamental different between "static notebooks" and
> "notebooks with kernels" - both can have Javascript and the new
> Javascript plugins will work on both.  Both have the same overall
> security issues and I don't think it makes sense to try and handle
> them separately.
> 
> Cheers,
> 
> Brian
> 
> 
> 
> 
> On Tue, Dec 11, 2012 at 9:15 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> 
>> Le 11 d?c. 2012 ? 16:10, Carl Smith a ?crit :
>> 
>>> Hi Brian
>>>> 
>>>> The idea is that the extra Javascript cool-stuff will be installed by
>>>> the person who runs the notebook server once and for all notebooks on
>>>> that server.  Similar to how python packages are installed = you do
>>>> this before you start python.  To get data from python to the
>>>> Javascript plugins we will use JSON objects and trigger the callbacks
>>>> to handle them.
>>> 
>>> This seems to be dependent on a kernel, which static notebooks don't
>>> have? If I generate a static notebook, which is just a web page, then
>>> post that page to a hosting service, or email it to someone, how would
>>> the plugins work? Maybe we're looking at two slightly different
>>> scenarios. I'm focussed on static views only. The host should not have
>>> to allow anything more than posting and getting HTML documents.
>> 
>> IIRC, you can embed several repr in the ipynb file.
>> So you could provide a plugin that can "render" object on static view.
>> (like d3.js graph, you don't need the kernel to do that)
>> 
>>> ================
>>> 
>>> Hi Matthias
>>> 
>>>> Static notebooks, served from a different domain, could be rendered
>>>> inside iframes, enabling us to embed them inside other webpages and
>>>> applications. These notebooks would still be superficially served by
>>>> our own servers, so the UX wouldn't be effected.
>>>> 
>>>> keep in mind that iframe are not sandboxed, and you can inject js on parent
>>>> frame.
>>>> Unless you use the sandbox attributes, which is part of html5 but not
>>>> implemented in every
>>>> browser? And not yet infallible, it is more a "we'll help you embed other
>>>> pages by providing a separate
>>>> js namespace but we don't guaranty yes that the VM is unbreachable"
>>> 
>> 
>>> I pretty sure iframes are sandboxed in the sense that a parent page
>>> and an iframe can not communicate unless they have the same origin,
>>> and this is an old feature. The new sandbox attribute in HTML5 is for
>>> a different purpose.
>> 
>> But this is still kind of a problem, as usually the static view will be served from the "same origin"
>> as the rest of the website.
>> 
>> If you don't want to make a notebook fully public, you have to have some kind of authentication
>> that allows you to load it.
>> 
>> I'm still doubt a little about what frames are supposed to do and what they actually do.
>> I'm not an expert on that, but it is still worth digging.
>> 
>>>> Responsible disclosure don't want to say much more but but having a
>>>> statically display
>>>> notebook is often link to having a "sharing/import" button which is
>>>> dangerous.
>>>> And could lead to self propagating notebook through account that can infect
>>>> other
>>>> notebooks, or share itself on twitter...
>>> 
>>> Any buttons, like for importing a notebook, would live in the parent
>>> page and would have no access to, nor allow access from, the iframe.
>>> The parent page would know which static notebook it embeds in the
>>> iframe though, so it could provide buttons that connect to the actual
>>> notebook in question, which is a totally different file to the static
>>> notebook being rendered in the iframe anyway.
>> 
>> I understand what you want to do, I guess the definition of "static" is blurry.
>> If you want a perfectly static version (does it make sense in html) you can go with iframe.
>> If you want the ability to comment on a particular cell, then you have to build iframe for
>> every cell.
>> and you lose the ability to comment "inline" as github does.
>> 
>> 
>>> 
>>>> Multi domain is a real good idea. I have a clear view in my head on how we
>>>> could use that in a way close to OAuth to allow javascript by still having
>>>> "logged-in" users.
>>>> It wouldn't be as seamless a something like github, but close.
>>> 
>>> I think we're looking at things differently: You seem to be
>>> considering static views as something generated on the fly and on
>>> demand, nbviewer style. I'm thinking about running nbconvert on a
>>> notebook, then keeping the output as a webpage to be copied and passed
>>> around freely. Once the static notebook exists, it's a done deal.
>>> There's no chance of any changes to IPython breaking it. It's a
>>> independent webpage. Updating it would amount to deleting it and
>>> replacing it with a new version.
>> 
>> I don't think those are quite different.
>> You can have a "perfectly static" version that embeds bad js and require some kind of authentication to be seen.
>> The line between "on the fly" and static is thin.
>> 
>>> 
>>>> The **big** question is:
>>>> Are viewer logged in (in any way) to the given server, and if so do they
>>>> have the right to do anything else with those credentials ?
>>>> If it is just a public notebook viewer, then it's fine.
>>>> 
>>>> If you want something more "interactive" (sharing/ permissions?etc, and the
>>>> display any JS ) you won't have much choice.
>>>> Or you will have a painful multi-login.
>>> 
>>> I'm very much against hosting user submitted notebooks on any domain
>>> with cookie based authentication. It needs to be divided into 'trusted
>>> domain', where no user's JS will ever be served, and 'hosting domain'
>>> that has no account system of it's own. The trusted domain would
>>> control the hosting domain, as a kind of slave.
>> 
>> Yep, kind of what I have in mind.
>> The hosting domain can have "tokens"
>> Publish this comment on this notebook on the behalf of ...
>> The you have to "validate" those action on the "trusted domain".
>> --
>> Matthias
>> 
>> 
>> 
>>> 
>>> That's just my take on all this.
>>> 
>>> Cheers
>>> 
>>> Carl
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From stanton at haas.berkeley.edu  Wed Dec 12 14:43:46 2012
From: stanton at haas.berkeley.edu (Richard Stanton)
Date: Wed, 12 Dec 2012 11:43:46 -0800
Subject: [IPython-dev] What is "Print View" supposed to do?
Message-ID: <40C7B1BFC291ED4E9D10436D07736A33470B8F2F9E@EXMAIL7.haas.uc.berkeley.edu>

I'm trying to share an IPython notebook with someone who doesn't use IPython (or Python, for that matter). I figured a PDF file would do the trick, and tried selecting "Print View", but all this displayed was the first cell in the worksheet, then nothing else. What is this supposed to do...?

By the way, this is under Windows, running EPD 7.3 and the latest version of IPython 0.14.dev (downloaded this morning).

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121212/051e8159/attachment.html>

From bussonniermatthias at gmail.com  Wed Dec 12 14:48:39 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 12 Dec 2012 20:48:39 +0100
Subject: [IPython-dev] What is "Print View" supposed to do?
In-Reply-To: <40C7B1BFC291ED4E9D10436D07736A33470B8F2F9E@EXMAIL7.haas.uc.berkeley.edu>
References: <40C7B1BFC291ED4E9D10436D07736A33470B8F2F9E@EXMAIL7.haas.uc.berkeley.edu>
Message-ID: <4B8905EA-2713-4E31-A034-5F6CC081A3E9@gmail.com>


Le 12 d?c. 2012 ? 20:43, Richard Stanton a ?crit :

> I?m trying to share an IPython notebook with someone who doesn?t use IPython (or Python, for that matter). I figured a PDF file would do the trick, and tried selecting ?Print View?, but all this displayed was the first cell in the worksheet, then nothing else. What is this supposed to do??
>  
> By the way, this is under Windows, running EPD 7.3 and the latest version of IPython 0.14.dev (downloaded this morning).

Try printing directly without going through the print view, it should work.

You could also share through http://nbviewer.ipython.org/

We are working on improving static rendering. 
-- 
Matthias



From wen.g.gong at gmail.com  Wed Dec 12 15:00:36 2012
From: wen.g.gong at gmail.com (W Gong)
Date: Wed, 12 Dec 2012 15:00:36 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOoi3ing1B_8g3La2ucJnakUQ7mQ3fiy9EvUZ059yu_yPQ@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C896C3.1090209@creativetrax.com>
	<CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>
	<CAH4pYpQMkTWtpTAZ-nLyEiQ8_w3E5zvrjB0fP95VhwFMHKQ9wA@mail.gmail.com>
	<CAH4pYpSuF9Xfd6zGzD5PmO8=kYFs6pU3UG5x+razoMfhBEGbZg@mail.gmail.com>
	<CAHAreOoi3ing1B_8g3La2ucJnakUQ7mQ3fiy9EvUZ059yu_yPQ@mail.gmail.com>
Message-ID: <CAJzFRJEhSYQ1Ar0S+i1jYw-L_xh1X0ge5kH3H+Y6Y-giaQgM0Q@mail.gmail.com>

Wow, congratulations!
Great software deserves great support!

On Wed, Dec 12, 2012 at 2:10 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> I'm at an NSF workshop in Providence and have near-zero bandwidth
> today, so it would  be great if Thomas can give us a hand with the
> site upload.  We just shared the draft with you on gdocs...
>
> thanks!!
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 

Thanks,

- Wen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121212/8eda87e1/attachment.html>

From gokhansever at gmail.com  Wed Dec 12 15:05:27 2012
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Wed, 12 Dec 2012 13:05:27 -0700
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
Message-ID: <CAE5kuyjSEqB-b5RHOCoScQEeBEucRhfG0-dk3QSyaY=NQzdHew@mail.gmail.com>

Congratulations for the new funding IPython team.

You are very determined putting IPython at the heart of computational
scientists' workflow.


On Tue, Dec 11, 2012 at 3:34 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi all,
>
> we wanted to give you all some absolutely fantastic news regarding
> IPython that we just received...  This also partly explains my silence
> during much of this semester, as I was focused on looking for
> long-term funding for IPython.
>
> We are proud to announce that we've received funding from the Alfred
> P. Sloan foundation (http://sloan.org) that will support IPython
> development for the next two years.  Thanks to this $1.15M grant,
> Brian Granger and I will be working roughly 3/4 of our time on
> IPython, Min Ragan-Kelley will be our lead project engineer, and Paul
> Ivanov and Thomas Kluyver will be working as postdocs on the project.
> Furthermore,  Matthew Brett and JB Poline of Nipy fame will be working
> part-time on the development of notebooks for applied statistics in
> collaboration with Jonathan Taylor from Stanford (these materials will
> be used by Jonathan in the teaching of his applied stats course, the
> STAT 191 course - http://www.stanford.edu/class/stats191).  While
> Brian is at Cal Poly San Luis Obispo,  the rest of our team will be
> located at Berkeley (with frequent visits to Stanford for the
> statistics work with Jonathan).
>
> This will also give us resources to support two annual all-hands
> development sprints at Berkeley where I hope all our core developers
> will be able to attend, as well as funding for cloud resources so that
> we can provide paid support for our CI hosting and nbviewer beyond
> what the free plans allow (and further services that may arise over
> time).
>
> We'd like to particularly thank Josh Greenberg, our program director
> at the Sloan foundation, for the phenomenal guidance and support he
> provided during the grant proposal preparation and detailed review
> process.
>
> Needless to say, we are extremely excited about these news.  We'll
> have more announcements to make over the next few weeks, but now I
> need to get a lot of administrative wheels in motion quickly, as the
> funding officially starts Jan 1, 2013!
>
> All the best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121212/aeb131de/attachment.html>

From takowl at gmail.com  Wed Dec 12 15:30:24 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 12 Dec 2012 20:30:24 +0000
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOoi3ing1B_8g3La2ucJnakUQ7mQ3fiy9EvUZ059yu_yPQ@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C896C3.1090209@creativetrax.com>
	<CAOvn4qjt8GwzbDuscpsEq=mn49a0V7jZU083SrHmk8t4GMJ1cg@mail.gmail.com>
	<CAH4pYpQMkTWtpTAZ-nLyEiQ8_w3E5zvrjB0fP95VhwFMHKQ9wA@mail.gmail.com>
	<CAH4pYpSuF9Xfd6zGzD5PmO8=kYFs6pU3UG5x+razoMfhBEGbZg@mail.gmail.com>
	<CAHAreOoi3ing1B_8g3La2ucJnakUQ7mQ3fiy9EvUZ059yu_yPQ@mail.gmail.com>
Message-ID: <CAOvn4qj=mEcd8KuT0pFxaSvrUjmd1UjrYxXhQ7o5Sq2kZ5C00A@mail.gmail.com>

On 12 December 2012 19:10, Fernando Perez <fperez.net at gmail.com> wrote:

> I'm at an NSF workshop in Providence and have near-zero bandwidth
> today, so it would  be great if Thomas can give us a hand with the
> site upload.  We just shared the draft with you on gdocs...
>

Thanks. I've quickly drafted a summary of that and made a pull request:

https://github.com/ipython/ipython-website/pull/15

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121212/ecf1c269/attachment.html>

From jason-sage at creativetrax.com  Thu Dec 13 00:16:56 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 12 Dec 2012 23:16:56 -0600
Subject: [IPython-dev] IPEP 8: Custom messages and message handlers
In-Reply-To: <CAH4pYpSdEJODzydUo0e5bL0Z2+V23YQa+rOykoxGf7At2wL_Sw@mail.gmail.com>
References: <50C8A7B5.7020204@creativetrax.com>
	<CAH4pYpSdEJODzydUo0e5bL0Z2+V23YQa+rOykoxGf7At2wL_Sw@mail.gmail.com>
Message-ID: <50C964C8.4010308@creativetrax.com>

On 12/12/12 10:44 AM, Brian Granger wrote:
> Great, thanks for getting this started.  This is one of the main tasks
> of the grant, so we will start to work on this a lot in Jan.
>

Awesome.  That is really great to hear.

I've thought a lot about the future of the Sage notebook, and how we 
might transition to a multi-user IPython notebook instead of pushing 
forward with the current Sage notebook design.  I'm excited to hear that 
doing a good multi-user IPython notebook is one of the top priorities.


> As far as the IPEP, I am not very happy with how we are running IPEP
> discussions.  I think that having them on the mailing list makes them
> difficult to find and track along with the IPEP text.  I would
> probably prefer to use github issues to track the discussion and the
> wiki for the document.

The thing about the issues is that the discussion isn't threaded.  If it 
was, and if it was easy to easy to quote parts of a post to reply, then 
I think it would do a good job of replacing an email discussion.

Thanks,

Jason



From fperez.net at gmail.com  Thu Dec 13 01:43:16 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 13 Dec 2012 01:43:16 -0500
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <50C7D49D.2050209@creativetrax.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C7D49D.2050209@creativetrax.com>
Message-ID: <CAHAreOoNnpTm-1Ps=sENc714o_U+TVTdy7rp8-wcN1=3k+4y_A@mail.gmail.com>

Hey Jason,

On Tue, Dec 11, 2012 at 7:49 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> Awesome!  Congrats!  I can't wait to see what the goals are--that's
> quite a team, based on the work you guys have done.  Is this part of the
> funding that you envisioned for the multi-user notebook?

Indeed, and thanks for the kind words.  Both to answer this and
following your advice of providing a press-relase-like info piece,
here are some more details:

http://ipython.org/sloan-grant.html

It's a bummer you couldn't make it to the ICERM workshop, it's been a
blast and we've had a ton of interesting discussions around the
NotebookS (IPython & Sage).  I've also had a good chance to talk a lot
about these things directly with William, and some good ideas have
come out.

Best,

f


From david.trem at gmail.com  Thu Dec 13 05:22:17 2012
From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=)
Date: Thu, 13 Dec 2012 11:22:17 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
Message-ID: <50C9AC59.50606@gmail.com>

I pushed an updated version that does not use "exec_command"
and that should work on windows...

May you give it a try ?

Regards,

David

Le 11/12/12 13:16, klo uo a ?crit :
> Thanks for working on this.
> On Windows it won't run:
>
> ========================================
> Could not locate executable C:Python27pythonw.exe
> Executable C:Python27pythonw.exe does not exist
> ========================================
>
> Obvious issue with backslash at some point
>
> Is distutil.exec_command only way to do this?
> On Windows it will look "ugly" as it pops command line window on 
> command execution
>
>
> On Tue, Dec 11, 2012 at 10:53 AM, Tr?mouilles David 
> <david.trem at gmail.com <mailto:david.trem at gmail.com>> wrote:
>
>     Hi,
>
>     Finally, thanks to Thomas entry point, I gave a try at implementing
>     some Fortran cell magic based on f2py. This is cell_magic only with
>     almost no argument available. But this is fully functional :-)
>
>     I've asked for a pull request available here:
>     https://github.com/ipython/ipython/pull/2674
>
>     A demo notebook is available here:
>     http://nbviewer.ipython.org/4257286/
>
>     Comments, suggestions and feature requests are very welcome :-)
>
>     Regards,
>
>     David
>
>     Le 07/12/12 14:15, Thomas Kluyver a ?crit :
>>     On 7 December 2012 12:49, David Tremouilles <david.trem at gmail.com
>>     <mailto:david.trem at gmail.com>> wrote:
>>
>>         I'm wondering if there is any %%fortran magic available out
>>         there ?
>>         Does anybody tried to implement this ?
>>
>>
>>     I don't think anyone's done it yet. If you want to have a go, a
>>     good starting point would be the %%cython magic we ship as an
>>     extension:
>>     https://github.com/ipython/ipython/blob/master/IPython/extensions/cythonmagic.py
>>
>>     Best wishes,
>>     Thomas
>>
>>
>>     _______________________________________________
>>     IPython-User mailing list
>>     IPython-User at scipy.org  <mailto:IPython-User at scipy.org>
>>     http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121213/d725f3db/attachment.html>

From carl.input at gmail.com  Thu Dec 13 08:47:10 2012
From: carl.input at gmail.com (Carl Smith)
Date: Thu, 13 Dec 2012 13:47:10 +0000
Subject: [IPython-dev] Some excellent news for IPython: we've received
 stable funding from the Alfred P. Sloan Foundation.
In-Reply-To: <CAHAreOoNnpTm-1Ps=sENc714o_U+TVTdy7rp8-wcN1=3k+4y_A@mail.gmail.com>
References: <CAHAreOrq-xk0yhke6J8KrkUK4=jeavEwRDwGei9-OOFJ=ww2-w@mail.gmail.com>
	<50C7D49D.2050209@creativetrax.com>
	<CAHAreOoNnpTm-1Ps=sENc714o_U+TVTdy7rp8-wcN1=3k+4y_A@mail.gmail.com>
Message-ID: <CAP-uhDfiqUJxuXE47PRhrUXTgJ6_4zTRhO5QAMj36KTaOZhQVQ@mail.gmail.com>

Fantastic news. Thanks for all the hard work over the years guys.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121213/ed86b283/attachment.html>

From klonuo at gmail.com  Thu Dec 13 09:21:33 2012
From: klonuo at gmail.com (klo)
Date: Thu, 13 Dec 2012 15:21:33 +0100
Subject: [IPython-dev] Embryo of Fortran magic based on f2py
In-Reply-To: <50C9AC59.50606@gmail.com>
References: <CAFHqy=aKj4+Qr4t80ZAUSiyvr8TBv76QOdtuJ50hLcBU0UFZGg@mail.gmail.com>
	<CAOvn4qhROg3a1kbdxKZjcAY3_uyFDb3e5Q7w+UQRgmM0YiDn9Q@mail.gmail.com>
	<50C7029D.2040908@gmail.com>
	<CAA-8Ld9dECH+Vi_pf-61BUFVKHPu4bJe5-CBMZPTHury5C3psg@mail.gmail.com>
	<50C9AC59.50606@gmail.com>
Message-ID: <op.wo9d570ursc1x7@home>

On Thu, 13 Dec 2012 11:22:17 +0100, Tr?mouilles David  
<david.trem at gmail.com> wrote:

> I pushed an updated version that does not use "exec_command"
> and that should work on windows...
>
> May you give it a try ?

Thanks David,

I commented here:  
https://github.com/davidtrem/ipython/commit/436d2443770b44a7a0245e069cc500967316132a#commitcomment-2303002


From scopatz at gmail.com  Thu Dec 13 11:15:06 2012
From: scopatz at gmail.com (Anthony Scopatz)
Date: Thu, 13 Dec 2012 10:15:06 -0600
Subject: [IPython-dev] Chrome 22 and loading a notebook with too many images
Message-ID: <CAPk-6T7ump066J-mDjYXvNHrL6KJXpF2eHHEUsGgqF3Jx4OtVw@mail.gmail.com>

Hello All,

I want to report an issue have have be having on Chrome
version 22.0.1229.94 on Ubuntu 11.10 on IPython 0.14-dev.
It involves loading a saved notebook with many matplotlib Figures
in a single output cell. I have done all of the normal debugging steps
a million times (cleared browser cache, re-installed fresh local mathjax,
updated master, did a clean re-install of IPython).

Two sample notebooks are attached.  In pos.ipynb, there are 15 images
in a single output cell at the bottom.  This fails to load in chrome but
works just fine in firefox.  In pos1.ipynb, I have reduced the number of
images down to 8, where the notebook now loads.  Note that 8 seems
to be the maximum because 9+ fails.

Also, when it fails to load, it doesn't do so gracefully.  Instead the page
hangs after creating the menu bars and the CPU usage shoots up to
100% and remains there *ad infinitum*.  Annoying! (Though it is keeping
my lap warm in the cold Chicago winter...)

I do not know what it causing this issue, but I thought that you should
know.

Be Well
Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121213/68488475/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pos.ipynb
Type: application/octet-stream
Size: 755945 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121213/68488475/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pos1.ipynb
Type: application/octet-stream
Size: 376513 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121213/68488475/attachment-0001.obj>

From ellisonbg at gmail.com  Thu Dec 13 22:55:45 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 13 Dec 2012 19:55:45 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
Message-ID: <CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>

Matthias,

On Wed, Dec 12, 2012 at 11:37 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 12 d?c. 2012 ? 18:46, Brian Granger a ?crit :
>
>> OK trying to catch up with this thread...
>>
>> * Matthias brings up a great point that we need to consider
>> forged/hostile notebooks that were not created with our notebook
>> server.  The solution to this is to make sure that our notebook server
>> can "clean" notebooks before displaying them.  The solutions that I
>> posted above will address this completely and allow people to simply
>> open notebooks from the web without worrying about them.
>>
>> * I still feel like I don't have a straight answer to my question:
>> will the solutions I proposed solve all of the security problems while
>> allowing us to serve authenticated notebooks on single domains?  If
>> now, what security problems would remain?
>>
>>>
>>> To fix this, we need to enable the HTML sanitizer that comes with the
>>> JS Markdown rendered that we are using.  This is what StackOverflow
>>> uses to sanitize their markdown and should completely remove any
>>> security risks coming from within markdown cells.
>>
>>
>>> * In CodeCell output, the Javascript repr is dynamically passed
>>> into eval.  This only happens when code is run, not when the notebook
>>> is loaded, so it is less critical, but still needs to be fixed.
>>
>>> Oh, yes forgot about that, we will have to clean that HTML as well.
>>
> Which can be resumes in clean html/js in all output.
> (as rendered md cell are actually "output")
>
> Then yes.
>
>
>>
>> * I am confused about the multiple domain solution that is being
>> proposed.  The idea is that notebooks containing arbitrary Javascript
>> would be served from a separate domain that does not offer any
>> authentication.  The authenticated notebooks would live on a second
>> domain that doesn't allow Javascript.  But then how is a user supposed
>> to author a notebook with Javascript and have that notebook not be
>> anonymous?  What if a user wants to author a notebook with Javascript
>> that needs to be private?
>
> let see.
> Multi domain just allow to have fine-grained privileges in cookies.
>
> You can login to wordpress with your google account right ?
> But then it only have access to your identity, not your mails right ?
>
> Then you do the same under the hood with subdomains.
> (for the sake of simplicity i'll use domains names that have meanings, and a simplified logic.)
>
> user logs in https://iamroot.ipython.org.
> He/She have acces to his/her dashboard.
>
> Get to notification "Other User want you to see a notebook QUX [open]"
>
> click on Open opens the notebook in another tabs.
>
> https://comment-only.ipython.org/view/QUX.ipynb
> The user is still "logged in", but any request to https://iamroot.ipython.org will fail because of xss policies.
> And obviously the only thing malicious JS could do is post comment on the current notebook.
>
> The point is to restrain the number of actions available from a certain domain name.
> As you control the authorization of a domain name server side, nothing prevent a user to grant more rights to a specific "domain" or notebook.

I understand that multiple domains allow a fine grained approach to
authentication.

Most people running notebooks are just going to throw the notebook
server up on a port and point people to it.  Many will not even create
an actual domain, just use a raw IP address.  This will be true even
when the notebook server gains multiuser capabilities.  I don't want
to have to  tell people - to run the notebook server, you have to run
servers on two separate domains with an infrastructure that allows
those servers to work together.  We want the multiuser support to be
as simply as:

ipython notebook --multiuser ...

on a single machine.

But even if we forced everyone to use a multiple domain notebook
server I don't understand how it would work.  Here is my
understanding:

* There would be a secure domain that would not allow any dynamic
Javascript in notebooks.  We would simply clean it out everytime in
the notebook app.
* Because this domain is "safe" this is where people would want to
author and keep all of their notebooks.
* Because dynamic Javascript is cleaned out every time, this domain
could not be used to create Javascript containing notebooks.
* In my mind, life on this domain is exactly as I propose above = we
simply don't allow JS because it could do bad things.
* Notebooks on this domain can safely be given access to the users
regular kernels - IOW, these notebooks are completely equivalent to
having shell access to the users files.

* When people want to create or view notebooks containing Javascript
code, they have to go to a separate insecure domain.  This domain has
to be completely sandboxed.
* The unsecure domain can't be allowed to edit/remove/etc. notebooks
from the secure domain, other wise hostile Javascript code could
completely destroy the safe notebooks.
* Notebooks authored on this domain have to stay on this domain, so
users will have to manage two sets of notebooks.
* Kernels on this domain have to be completely sandboxed as
potentially hostile Javascript code could use Kernel.execute to do
anything on the kernel.  These kernels could not even be allowed
network connectivity as users could inject Js code that write python
code that launches DOS attacked on the server or other hosts on the
net.
* Notebooks that users share with each other have to remain on this
domain as they might contain hostile JS code.
* Hostile JS code on this domain would have the ability to completely
screw up notebooks on this domain.  So I could write a notebook that
will delete all of your other JS containing notebooks if you open it?

Is this a correct summary of how the multiple domains would work?  Am
I missing or misunderstanding things?  Would we give notebooks on the
insecure domain more capabilities or access to the safe domain side of
things?  Is dynamic Javascript less risky than I am understanding so
we wouldn't have to isolate kernels?

Cheers,

Brian



> It does not prevent collaborating in anyways,
> But if you share kernel you share filesystem, so there are no point in really using subdomain at this stage.
> Still it gives another layer of security as if  there is a way to inject javascript, this javascript will not find any
> malicious things to do.
>
> Or it allows you "test" untrusted plugins..
>
> I let you imagine other stuff.
> --
> Matthias
>
>
>
>> * I don't see any fundamental different between "static notebooks" and
>> "notebooks with kernels" - both can have Javascript and the new
>> Javascript plugins will work on both.  Both have the same overall
>> security issues and I don't think it makes sense to try and handle
>> them separately.
>>
>> Cheers,
>>
>> Brian
>>
>>
>>
>>
>> On Tue, Dec 11, 2012 at 9:15 AM, Matthias BUSSONNIER
>> <bussonniermatthias at gmail.com> wrote:
>>>
>>> Le 11 d?c. 2012 ? 16:10, Carl Smith a ?crit :
>>>
>>>> Hi Brian
>>>>>
>>>>> The idea is that the extra Javascript cool-stuff will be installed by
>>>>> the person who runs the notebook server once and for all notebooks on
>>>>> that server.  Similar to how python packages are installed = you do
>>>>> this before you start python.  To get data from python to the
>>>>> Javascript plugins we will use JSON objects and trigger the callbacks
>>>>> to handle them.
>>>>
>>>> This seems to be dependent on a kernel, which static notebooks don't
>>>> have? If I generate a static notebook, which is just a web page, then
>>>> post that page to a hosting service, or email it to someone, how would
>>>> the plugins work? Maybe we're looking at two slightly different
>>>> scenarios. I'm focussed on static views only. The host should not have
>>>> to allow anything more than posting and getting HTML documents.
>>>
>>> IIRC, you can embed several repr in the ipynb file.
>>> So you could provide a plugin that can "render" object on static view.
>>> (like d3.js graph, you don't need the kernel to do that)
>>>
>>>> ================
>>>>
>>>> Hi Matthias
>>>>
>>>>> Static notebooks, served from a different domain, could be rendered
>>>>> inside iframes, enabling us to embed them inside other webpages and
>>>>> applications. These notebooks would still be superficially served by
>>>>> our own servers, so the UX wouldn't be effected.
>>>>>
>>>>> keep in mind that iframe are not sandboxed, and you can inject js on parent
>>>>> frame.
>>>>> Unless you use the sandbox attributes, which is part of html5 but not
>>>>> implemented in every
>>>>> browser? And not yet infallible, it is more a "we'll help you embed other
>>>>> pages by providing a separate
>>>>> js namespace but we don't guaranty yes that the VM is unbreachable"
>>>>
>>>
>>>> I pretty sure iframes are sandboxed in the sense that a parent page
>>>> and an iframe can not communicate unless they have the same origin,
>>>> and this is an old feature. The new sandbox attribute in HTML5 is for
>>>> a different purpose.
>>>
>>> But this is still kind of a problem, as usually the static view will be served from the "same origin"
>>> as the rest of the website.
>>>
>>> If you don't want to make a notebook fully public, you have to have some kind of authentication
>>> that allows you to load it.
>>>
>>> I'm still doubt a little about what frames are supposed to do and what they actually do.
>>> I'm not an expert on that, but it is still worth digging.
>>>
>>>>> Responsible disclosure don't want to say much more but but having a
>>>>> statically display
>>>>> notebook is often link to having a "sharing/import" button which is
>>>>> dangerous.
>>>>> And could lead to self propagating notebook through account that can infect
>>>>> other
>>>>> notebooks, or share itself on twitter...
>>>>
>>>> Any buttons, like for importing a notebook, would live in the parent
>>>> page and would have no access to, nor allow access from, the iframe.
>>>> The parent page would know which static notebook it embeds in the
>>>> iframe though, so it could provide buttons that connect to the actual
>>>> notebook in question, which is a totally different file to the static
>>>> notebook being rendered in the iframe anyway.
>>>
>>> I understand what you want to do, I guess the definition of "static" is blurry.
>>> If you want a perfectly static version (does it make sense in html) you can go with iframe.
>>> If you want the ability to comment on a particular cell, then you have to build iframe for
>>> every cell.
>>> and you lose the ability to comment "inline" as github does.
>>>
>>>
>>>>
>>>>> Multi domain is a real good idea. I have a clear view in my head on how we
>>>>> could use that in a way close to OAuth to allow javascript by still having
>>>>> "logged-in" users.
>>>>> It wouldn't be as seamless a something like github, but close.
>>>>
>>>> I think we're looking at things differently: You seem to be
>>>> considering static views as something generated on the fly and on
>>>> demand, nbviewer style. I'm thinking about running nbconvert on a
>>>> notebook, then keeping the output as a webpage to be copied and passed
>>>> around freely. Once the static notebook exists, it's a done deal.
>>>> There's no chance of any changes to IPython breaking it. It's a
>>>> independent webpage. Updating it would amount to deleting it and
>>>> replacing it with a new version.
>>>
>>> I don't think those are quite different.
>>> You can have a "perfectly static" version that embeds bad js and require some kind of authentication to be seen.
>>> The line between "on the fly" and static is thin.
>>>
>>>>
>>>>> The **big** question is:
>>>>> Are viewer logged in (in any way) to the given server, and if so do they
>>>>> have the right to do anything else with those credentials ?
>>>>> If it is just a public notebook viewer, then it's fine.
>>>>>
>>>>> If you want something more "interactive" (sharing/ permissions?etc, and the
>>>>> display any JS ) you won't have much choice.
>>>>> Or you will have a painful multi-login.
>>>>
>>>> I'm very much against hosting user submitted notebooks on any domain
>>>> with cookie based authentication. It needs to be divided into 'trusted
>>>> domain', where no user's JS will ever be served, and 'hosting domain'
>>>> that has no account system of it's own. The trusted domain would
>>>> control the hosting domain, as a kind of slave.
>>>
>>> Yep, kind of what I have in mind.
>>> The hosting domain can have "tokens"
>>> Publish this comment on this notebook on the behalf of ...
>>> The you have to "validate" those action on the "trusted domain".
>>> --
>>> Matthias
>>>
>>>
>>>
>>>>
>>>> That's just my take on all this.
>>>>
>>>> Cheers
>>>>
>>>> Carl
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Fri Dec 14 01:34:07 2012
From: carl.input at gmail.com (Carl Smith)
Date: Fri, 14 Dec 2012 06:34:07 +0000
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
	<CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
Message-ID: <CAP-uhDdKW7sNggOEV6x3ioWCu4LkSKmP_j93Z3UYZQuNahFLsQ@mail.gmail.com>

Hi Brian

I think we're discussing different things altogether. The reason it
makes sense to distinguish between static views and everything else is
that static notebooks are not notebooks, they're just webpages. I know
you disagree with treating them differently when dealing with
security, but I know you're happy to hear other people's views too :)

> Most people running notebooks are just going to throw the notebook
> server up on a port and point people to it.  Many will not even create
> an actual domain, just use a raw IP address.  This will be true even
> when the notebook server gains multiuser capabilities.  I don't want
> to have to  tell people - to run the notebook server, you have to run
> servers on two separate domains with an infrastructure that allows
> those servers to work together.  We want the multiuser support to be
> as simply as:
>
> ipython notebook --multiuser ...
>
> on a single machine.

There wouldn't be two servers in that scenario. When a user wanted to
create a static view, IPython would save the notebook, run nbconvert
on it, producing HTML output, drop that into a template to create a
single HTML document with the images all embedded as base64 or SVG
literals and so on. That webpage can be passed around like any file
and viewed in any browser.

An (optional) hosting service, which would operate externally to any
of this, would be a simple server that could just host these static
views for people to look at, and people could easily embed them in
other webpages.

An (optional) web service with user accounts and the ability to share
actual notebooks and stuff like that, would never host user created
static views, or any user generated HTML, to prevent XSS attacks on
the main web service. It would instead use a hosting service to serve
notebooks from a different domain, and embed them wherever I wanted
them to appear, inside iframes.

> * When people want to create or view notebooks containing Javascript
> code, they have to go to a separate insecure domain.  This domain has
> to be completely sandboxed.

No one would create anything at the hosting domain, just post things
to it and view stuff from it. There's no sandbox at all. It's more or
less a file server.

The alternative, ripping out all the dangerous HTML, will likely
include removing all onhover, onload type events and any img tags, and
other stuff that makes requests to URLs on the user's behalf. It'll be
hard to get right.

Any plugin system, as I see it, can only provide generic code, so each
notebook that uses JS will contain some JS specific to that notebook
which will have to be checked by hand before it's trusted. You can't
just use common libraries, you need some unique code to put those
libraries to use, often fairly large and complex code. Maybe I don't
understand how this will work??

If we deal with static views of notebooks as just HTML documents, we
can deal with this side of things as a relatively simple, commonplace,
web security issue, and not have to take a baseball bat to user's
code.

Actual notebook files are a radically different thing, but these files
are not dangerous because they contain JS, they're dangerous because
they allow JS, Python and other code to forget the browser and execute
straight on the system. There's little point looking to web/browser
security on that front.

If the user knows a notebook file's JS is safe, by whatever means,
they still have to be very careful about the code cells, which can use
a range of languages, and IPython syntax, to make one thing look like
another. You only need to edit one character to make totally honest
code become deadly. I could register a GitHub username that's one
character different to a trusted one, then include a line in a
notebook that appears to grab a bit of handy code from the friendly
guys at IPython, but actually clones my repo. The obvious next two
lines are `cd newrepo` and `sudo python setup.py install`, where
setup.py is whatever I want it to be. They'll happily provide their
root password and it's all over.

Even if all notebooks have safe JS, it doesn't make them safe by a
long shot, and we can't sanitise code cells.

In any case, I think trust should be primarily based on people. If you
sent me some executable and told me to run it, I'd run it. At the end
of the day, you're Brain Granger; you don't go around sending people
malicious code (or at least, if you did, it'd be funny).

If I whitelist IPython's GitHub account as trustworthy, I should be
able to run any code from IPython's GitHub repos without any further
confirmation. It's from people that I trust. Whitelisting sources,
based on people and organisations, is more natural than whitelisting
plugins, which could always be hosted by trustworthy people anyway.

Security is not the same thing as trust, I know, but trust is another
key element in sharing code safely.

It's an open problem, and one we should be willing to clash over a
bit, so I hope you don't mind the rant. It's just after 6am here (up
with the kids) so excuse me if I'm a bit blunt. I need to go get
another cuppa.

P.S. All the best, and congratulations with securing funding. IPython
has a really exciting future, with or without JS in static views :)
I'm sure you'll have an awesome new year.

Carl


From bussonniermatthias at gmail.com  Fri Dec 14 03:12:22 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 14 Dec 2012 09:12:22 +0100
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAP-uhDdKW7sNggOEV6x3ioWCu4LkSKmP_j93Z3UYZQuNahFLsQ@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
	<CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
	<CAP-uhDdKW7sNggOEV6x3ioWCu4LkSKmP_j93Z3UYZQuNahFLsQ@mail.gmail.com>
Message-ID: <1FA84142-E068-4BF2-A012-9E0A6DDF0BD8@gmail.com>

I only saw one thing :

Le 14 d?c. 2012 ? 07:34, Carl Smith a ?crit :
> 
> [...]. At the end
> of the day, you're **Brain** Granger; [...]

(emphase mine) 

-- 
Matthias

From bussonniermatthias at gmail.com  Fri Dec 14 03:15:54 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 14 Dec 2012 09:15:54 +0100
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
	<CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
Message-ID: <3CDF8C23-F569-4B35-8AE1-ABB702DE9458@gmail.com>


> 
> I understand that multiple domains allow a fine grained approach to
> authentication.
> 
> Most people running notebooks are just going to throw the notebook
> server up on a port and point people to it.  Many will not even create
> an actual domain, just use a raw IP address.  This will be true even
> when the notebook server gains multiuser capabilities.  I don't want
> to have to  tell people - to run the notebook server, you have to run
> servers on two separate domains with an infrastructure that allows
> those servers to work together.  We want the multiuser support to be
> as simply as:
> 
> ipython notebook --multiuser ...
> 
> on a single machine.
> 
> But even if we forced everyone to use a multiple domain notebook
> server I don't understand how it would work.  Here is my
> understanding:
> 
> * There would be a secure domain that would not allow any dynamic
> Javascript in notebooks.  We would simply clean it out everytime in
> the notebook app.
> * Because this domain is "safe" this is where people would want to
> author and keep all of their notebooks.
> * Because dynamic Javascript is cleaned out every time, this domain
> could not be used to create Javascript containing notebooks.
> * In my mind, life on this domain is exactly as I propose above = we
> simply don't allow JS because it could do bad things.
> * Notebooks on this domain can safely be given access to the users
> regular kernels - IOW, these notebooks are completely equivalent to
> having shell access to the users files.
> 
> * When people want to create or view notebooks containing Javascript
> code, they have to go to a separate insecure domain.  This domain has
> to be completely sandboxed.
> * The unsecure domain can't be allowed to edit/remove/etc. notebooks
> from the secure domain, other wise hostile Javascript code could
> completely destroy the safe notebooks.
> * Notebooks authored on this domain have to stay on this domain, so
> users will have to manage two sets of notebooks.
> * Kernels on this domain have to be completely sandboxed as
> potentially hostile Javascript code could use Kernel.execute to do
> anything on the kernel.  These kernels could not even be allowed
> network connectivity as users could inject Js code that write python
> code that launches DOS attacked on the server or other hosts on the
> net.
> * Notebooks that users share with each other have to remain on this
> domain as they might contain hostile JS code.
> * Hostile JS code on this domain would have the ability to completely
> screw up notebooks on this domain.  So I could write a notebook that
> will delete all of your other JS containing notebooks if you open it?
> 
> Is this a correct summary of how the multiple domains would work?  Am
> I missing or misunderstanding things?  Would we give notebooks on the
> insecure domain more capabilities or access to the safe domain side of
> things?  Is dynamic Javascript less risky than I am understanding so
> we wouldn't have to isolate kernels?
> 

I think this is a correct summary. 
You just have to consider the fact that as the server(s) have control of the two domains
you can move thins from one to the other under the hood and have seamless links between the two.
-- 
Matthias


> Cheers,
> 
> Brian
> 
> 
> 
>> It does not prevent collaborating in anyways,
>> But if you share kernel you share filesystem, so there are no point in really using subdomain at this stage.
>> Still it gives another layer of security as if  there is a way to inject javascript, this javascript will not find any
>> malicious things to do.
>> 
>> Or it allows you "test" untrusted plugins..
>> 
>> I let you imagine other stuff.
>> --
>> Matthias
>> 
>> 
>> 
>>> * I don't see any fundamental different between "static notebooks" and
>>> "notebooks with kernels" - both can have Javascript and the new
>>> Javascript plugins will work on both.  Both have the same overall
>>> security issues and I don't think it makes sense to try and handle
>>> them separately.
>>> 
>>> Cheers,
>>> 
>>> Brian
>>> 
>>> 
>>> 
>>> 
>>> On Tue, Dec 11, 2012 at 9:15 AM, Matthias BUSSONNIER
>>> <bussonniermatthias at gmail.com> wrote:
>>>> 
>>>> Le 11 d?c. 2012 ? 16:10, Carl Smith a ?crit :
>>>> 
>>>>> Hi Brian
>>>>>> 
>>>>>> The idea is that the extra Javascript cool-stuff will be installed by
>>>>>> the person who runs the notebook server once and for all notebooks on
>>>>>> that server.  Similar to how python packages are installed = you do
>>>>>> this before you start python.  To get data from python to the
>>>>>> Javascript plugins we will use JSON objects and trigger the callbacks
>>>>>> to handle them.
>>>>> 
>>>>> This seems to be dependent on a kernel, which static notebooks don't
>>>>> have? If I generate a static notebook, which is just a web page, then
>>>>> post that page to a hosting service, or email it to someone, how would
>>>>> the plugins work? Maybe we're looking at two slightly different
>>>>> scenarios. I'm focussed on static views only. The host should not have
>>>>> to allow anything more than posting and getting HTML documents.
>>>> 
>>>> IIRC, you can embed several repr in the ipynb file.
>>>> So you could provide a plugin that can "render" object on static view.
>>>> (like d3.js graph, you don't need the kernel to do that)
>>>> 
>>>>> ================
>>>>> 
>>>>> Hi Matthias
>>>>> 
>>>>>> Static notebooks, served from a different domain, could be rendered
>>>>>> inside iframes, enabling us to embed them inside other webpages and
>>>>>> applications. These notebooks would still be superficially served by
>>>>>> our own servers, so the UX wouldn't be effected.
>>>>>> 
>>>>>> keep in mind that iframe are not sandboxed, and you can inject js on parent
>>>>>> frame.
>>>>>> Unless you use the sandbox attributes, which is part of html5 but not
>>>>>> implemented in every
>>>>>> browser? And not yet infallible, it is more a "we'll help you embed other
>>>>>> pages by providing a separate
>>>>>> js namespace but we don't guaranty yes that the VM is unbreachable"
>>>>> 
>>>> 
>>>>> I pretty sure iframes are sandboxed in the sense that a parent page
>>>>> and an iframe can not communicate unless they have the same origin,
>>>>> and this is an old feature. The new sandbox attribute in HTML5 is for
>>>>> a different purpose.
>>>> 
>>>> But this is still kind of a problem, as usually the static view will be served from the "same origin"
>>>> as the rest of the website.
>>>> 
>>>> If you don't want to make a notebook fully public, you have to have some kind of authentication
>>>> that allows you to load it.
>>>> 
>>>> I'm still doubt a little about what frames are supposed to do and what they actually do.
>>>> I'm not an expert on that, but it is still worth digging.
>>>> 
>>>>>> Responsible disclosure don't want to say much more but but having a
>>>>>> statically display
>>>>>> notebook is often link to having a "sharing/import" button which is
>>>>>> dangerous.
>>>>>> And could lead to self propagating notebook through account that can infect
>>>>>> other
>>>>>> notebooks, or share itself on twitter...
>>>>> 
>>>>> Any buttons, like for importing a notebook, would live in the parent
>>>>> page and would have no access to, nor allow access from, the iframe.
>>>>> The parent page would know which static notebook it embeds in the
>>>>> iframe though, so it could provide buttons that connect to the actual
>>>>> notebook in question, which is a totally different file to the static
>>>>> notebook being rendered in the iframe anyway.
>>>> 
>>>> I understand what you want to do, I guess the definition of "static" is blurry.
>>>> If you want a perfectly static version (does it make sense in html) you can go with iframe.
>>>> If you want the ability to comment on a particular cell, then you have to build iframe for
>>>> every cell.
>>>> and you lose the ability to comment "inline" as github does.
>>>> 
>>>> 
>>>>> 
>>>>>> Multi domain is a real good idea. I have a clear view in my head on how we
>>>>>> could use that in a way close to OAuth to allow javascript by still having
>>>>>> "logged-in" users.
>>>>>> It wouldn't be as seamless a something like github, but close.
>>>>> 
>>>>> I think we're looking at things differently: You seem to be
>>>>> considering static views as something generated on the fly and on
>>>>> demand, nbviewer style. I'm thinking about running nbconvert on a
>>>>> notebook, then keeping the output as a webpage to be copied and passed
>>>>> around freely. Once the static notebook exists, it's a done deal.
>>>>> There's no chance of any changes to IPython breaking it. It's a
>>>>> independent webpage. Updating it would amount to deleting it and
>>>>> replacing it with a new version.
>>>> 
>>>> I don't think those are quite different.
>>>> You can have a "perfectly static" version that embeds bad js and require some kind of authentication to be seen.
>>>> The line between "on the fly" and static is thin.
>>>> 
>>>>> 
>>>>>> The **big** question is:
>>>>>> Are viewer logged in (in any way) to the given server, and if so do they
>>>>>> have the right to do anything else with those credentials ?
>>>>>> If it is just a public notebook viewer, then it's fine.
>>>>>> 
>>>>>> If you want something more "interactive" (sharing/ permissions?etc, and the
>>>>>> display any JS ) you won't have much choice.
>>>>>> Or you will have a painful multi-login.
>>>>> 
>>>>> I'm very much against hosting user submitted notebooks on any domain
>>>>> with cookie based authentication. It needs to be divided into 'trusted
>>>>> domain', where no user's JS will ever be served, and 'hosting domain'
>>>>> that has no account system of it's own. The trusted domain would
>>>>> control the hosting domain, as a kind of slave.
>>>> 
>>>> Yep, kind of what I have in mind.
>>>> The hosting domain can have "tokens"
>>>> Publish this comment on this notebook on the behalf of ...
>>>> The you have to "validate" those action on the "trusted domain".
>>>> --
>>>> Matthias
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> That's just my take on all this.
>>>>> 
>>>>> Cheers
>>>>> 
>>>>> Carl
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> 
>>> 
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From carl.input at gmail.com  Fri Dec 14 03:20:04 2012
From: carl.input at gmail.com (Carl Smith)
Date: Fri, 14 Dec 2012 08:20:04 +0000
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <1FA84142-E068-4BF2-A012-9E0A6DDF0BD8@gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
	<CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
	<CAP-uhDdKW7sNggOEV6x3ioWCu4LkSKmP_j93Z3UYZQuNahFLsQ@mail.gmail.com>
	<1FA84142-E068-4BF2-A012-9E0A6DDF0BD8@gmail.com>
Message-ID: <CAP-uhDfDn3HRZwckBMFQT3C8gK=tkSuy_3Q_yQ5CPbmejdOq3g@mail.gmail.com>

Opps - sorry about that. It was really early.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121214/ed3f96c2/attachment.html>

From nick.bollweg at gmail.com  Fri Dec 14 08:13:25 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Fri, 14 Dec 2012 08:13:25 -0500
Subject: [IPython-dev] custom web notebook cells
Message-ID: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>

hi everybody! longtime user, first time poster.

i've been kicking around a lot of ideas of things to do with ipython
notebook: while the bulk of stuff is clientside, to get there, i think i
need to do custom cells, which is a little more than ellisonbg's
"jsonhandlers" branch does. further, as the specific capabilities exposed
will require some pretty big depndencies (pdf generation, numpy, etc.) I am
thinking about doing everything as python modules that are installed into
the environment, rather than as purely drop-in javascript assets.

i've started the work on github here:
https://github.com/bollwyvl/ipython/tree/custom_cells

and a strawman cell provider here (does not contain working code, though it
does load!):
https://github.com/bollwyvl/ipython/tree/blockly

any insight is welcome!

to get my stuff in, i have used the setuptools iter_entry_points method I
am familiar with... and defined a new entry_point, ipnotebook. plugins
provide their list of assets and where one might find their static paths.
can anyone comment on how this is (potentially) different from The IPython
Way? The extensions stuff in the doc didn't look like they would be useful,
so i gave up on trying to use that.

the structure of working with cells in master is pretty rigid right now,
with the enumerated list of cells type in the code explicitly in the front
end in several places, js and html. my initial work (in a addition to
inadvertently recreating some of ellisonbg's work on the tornado app) has
been to refactor all of those cases of adding buttons or whatever into
something that looks at a window.IPython-scope plugin registry.... this is
defined in cell.js right now, but maybe should be someplace else.

after i did discover ellisonbg's stuff, i saw the most recent work was on
dependencies... during my stuff, i will look into frontend and backend
asset managers: require.js and webassets come to mind. this would have the
pleasant benefit of clearing up some of the mathjax complications, and
would make plugin development a lot more robust, if not simpler... also, if
a user installed the kitchen sink of plugins, which started adding a lot of
javascript assets, ideally a user should not see significant notebook
slowdown until they were actually using new features.

looking forward to working on this over the holidays, and would love to
have some fun things to show come 2013!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121214/3f683198/attachment.html>

From bussonniermatthias at gmail.com  Fri Dec 14 08:43:21 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 14 Dec 2012 14:43:21 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
Message-ID: <7CFCBCFE-BBD5-4840-8546-6E885EAED2AD@gmail.com>


Le 14 d?c. 2012 ? 14:13, Nicholas Bollweg a ?crit :

> hi everybody! longtime user, first time poster.
> 
> I've been kicking around a lot of ideas of things to do with ipython notebook: while the bulk of stuff is clientside, to get there, i think i need to do custom cells, which is a little more than ellisonbg's "jsonhandlers" branch does. further, as the specific capabilities exposed will require some pretty big depndencies (pdf generation, numpy, etc.) I am thinking about doing everything as python modules that are installed into the environment, rather than as purely drop-in javascript assets.
> 
> i've started the work on github here:
> https://github.com/bollwyvl/ipython/tree/custom_cells
> 
> and a strawman cell provider here (does not contain working code, though it does load!):
> https://github.com/bollwyvl/ipython/tree/blockly
> 
> any insight is welcome!

I'm curious of what kind of cell you wish to create?

> 
> To get my stuff in, i have used the setuptools iter_entry_points method I am familiar with... and defined a new entry_point, ipnotebook. plugins provide their list of assets and where one might find their static paths. can anyone comment on how this is (potentially) different from The IPython Way? The extensions stuff in the doc didn't look like they would be useful, so i gave up on trying to use that.
> 
> the structure of working with cells in master is pretty rigid right now, with the enumerated list of cells type in the code explicitly in the front end in several places, js and html. my initial work (in a addition to inadvertently recreating some of ellisonbg's work on the tornado app) has been to refactor all of those cases of adding buttons or whatever into something that looks at a window.IPython-scope plugin registry.... this is defined in cell.js right now, but maybe should be someplace else.

I had a quick look, but it looks nice (and documented Javascript people !!! ).
I think it is already worth a PR agains IPython. 

> after i did discover ellisonbg's stuff, i saw the most recent work was on dependencies... during my stuff, i will look into frontend and backend asset managers: require.js

Yes, we were thinking of require.js, but none of us did get the time to really look at it. 
Also something we tend to forgot is that Fernando Explicitely asked that we found a way to test the javascript before starting any major change in javascript?
which could be really helpful to test for regression if we move this way.

> and webassets come to mind. this would have the pleasant benefit of clearing up some of the mathjax complications, and would make plugin development a lot more robust, if not simpler... also, if a user installed the kitchen sink of plugins, which started adding a lot of javascript assets, ideally a user should not see significant notebook slowdown until they were actually using new features.

For the rest I think Brian will be the best to answer on what are his project for that.

> looking forward to working on this over the holidays, and would love to have some fun things to show come 2013!

Sure, lot's of awesome things, already huge number of PR unreviewed that are waiting to be refined to be merged.


> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121214/af55fe21/attachment.html>

From damianavila at gmail.com  Fri Dec 14 09:00:51 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Fri, 14 Dec 2012 11:00:51 -0300
Subject: [IPython-dev] New IPEP 9: Slideshow from the notebook
Message-ID: <50CB3113.90206@gmail.com>

People,

I have written a little IPEP to centralize the discussion about 
slide-show capabilities for the IPython notebook.

You can see the wiki to get a resume: 
https://github.com/ipython/ipython/wiki/IPEP-9:-Slideshow-from-the-notebook
And the discussion started as an issue here: 
https://github.com/ipython/ipython/issues/2680

I hope you join us into the discussion.

Regards.

Dami?n.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121214/011e260f/attachment.html>

From ellisonbg at gmail.com  Fri Dec 14 13:32:32 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 14 Dec 2012 10:32:32 -0800
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
Message-ID: <CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>

Nicholas,

Hi!

On Fri, Dec 14, 2012 at 5:13 AM, Nicholas Bollweg
<nick.bollweg at gmail.com> wrote:
> hi everybody! longtime user, first time poster.
>
> i've been kicking around a lot of ideas of things to do with ipython
> notebook: while the bulk of stuff is clientside, to get there, i think i
> need to do custom cells, which is a little more than ellisonbg's
> "jsonhandlers" branch does. further, as the specific capabilities exposed
> will require some pretty big depndencies (pdf generation, numpy, etc.) I am
> thinking about doing everything as python modules that are installed into
> the environment, rather than as purely drop-in javascript assets.
>
> i've started the work on github here:
> https://github.com/bollwyvl/ipython/tree/custom_cells
>
> and a strawman cell provider here (does not contain working code, though it
> does load!):
> https://github.com/bollwyvl/ipython/tree/blockly
>
> any insight is welcome!

Can you talk more about the usage cases you have for new cell types?
Can you give examples?

The notebook document format is sort of our contract with the
universe.  We want to be able to have an extremely precise and
testable definition of what constitutes an IPython notebook, so that
everyone can open them, share them, convert them, run them etc.  If we
allow custom cell types, all of that goes out the door, so I don't see
how we can support custom cell types.  The notebook document format
would stop being the notebook document format.

But, I would like to learn more about your usage cases.  There might
be other ways of supporting what you want to do:

* Use an existing cell type with metadata.
* Provide a custom UI for an existing cell type but don't change the
notebook format.
* Actually add a new cell type to the notebook format officially if it
is something that a majority of our users would use often.

> to get my stuff in, i have used the setuptools iter_entry_points method I am
> familiar with... and defined a new entry_point, ipnotebook. plugins provide
> their list of assets and where one might find their static paths. can anyone
> comment on how this is (potentially) different from The IPython Way? The
> extensions stuff in the doc didn't look like they would be useful, so i gave
> up on trying to use that.

I am very interested in learning more about this.  Right now my
jsplugins/handlers branch uses a pretty simple way of installing
additional Js code = a user just drops it into a special dir in the
.ipython directory.  It would be nice if these Js plugins were regular
python modules that could be installed as usual.  But something to
keep in mind:

* We have to serve these files on the web.  I don't want to have our
web server walking all over a users system to pull files as that is a
huge security risk.  We need to isolate the location of these files.
* Some plugins don't have any python code so I am not sure it makes
sense to bundle things into a python package.
* We need to maintain the ability to install ipython without setuptool.

Do you have ideas on how we can improve the user experience of
installing and managing Js plugins?

> the structure of working with cells in master is pretty rigid right now,
> with the enumerated list of cells type in the code explicitly in the front
> end in several places, js and html. my initial work (in a addition to
> inadvertently recreating some of ellisonbg's work on the tornado app) has
> been to refactor all of those cases of adding buttons or whatever into
> something that looks at a window.IPython-scope plugin registry.... this is
> defined in cell.js right now, but maybe should be someplace else.

That rigidity is merely a reflection of the notebook format.

> after i did discover ellisonbg's stuff, i saw the most recent work was on
> dependencies... during my stuff, i will look into frontend and backend asset
> managers: require.js and webassets come to mind. this would have the
> pleasant benefit of clearing up some of the mathjax complications, and would
> make plugin development a lot more robust, if not simpler... also, if a user
> installed the kitchen sink of plugins, which started adding a lot of
> javascript assets, ideally a user should not see significant notebook
> slowdown until they were actually using new features.

Before we start actually coding on the require.js stuff or
reorganizing that stuff, we need to figure out what we want to do
exactly.  Here is an IPEP for us to work all of that out:

https://github.com/ipython/ipython/wiki/IPEP-5:-Javascript-structure-and-testing

This is very much a draft, but let's move this forward before we jump
in so we know where we are headed.

> looking forward to working on this over the holidays, and would love to have
> some fun things to show come 2013!

Great!

Cheers,

Brian

> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Fri Dec 14 13:52:28 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 14 Dec 2012 10:52:28 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAP-uhDfDn3HRZwckBMFQT3C8gK=tkSuy_3Q_yQ5CPbmejdOq3g@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
	<CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
	<CAP-uhDdKW7sNggOEV6x3ioWCu4LkSKmP_j93Z3UYZQuNahFLsQ@mail.gmail.com>
	<1FA84142-E068-4BF2-A012-9E0A6DDF0BD8@gmail.com>
	<CAP-uhDfDn3HRZwckBMFQT3C8gK=tkSuy_3Q_yQ5CPbmejdOq3g@mail.gmail.com>
Message-ID: <CAH4pYpTeEowOVD1YeiutY_3BnNnSbY6fceezZT3iuGXn2SsohA@mail.gmail.com>

I would like two things:

* To get input from others on this issue - esp fperez and minrk.  I
will see them next week and make sure we talk about this.
* To move forward with the measures that I proposed above.  We
currently have individuals and groups running public notebook servers
that have this vulnerability and it must be fixed.  People have not
started to exploit this, but we can't risk this any longer, it is
simply too dangerous.

On Fri, Dec 14, 2012 at 12:20 AM, Carl Smith <carl.input at gmail.com> wrote:
> Opps - sorry about that. It was really early.
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Fri Dec 14 16:12:08 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 14 Dec 2012 13:12:08 -0800
Subject: [IPython-dev] Some Thoughts on Notebook Security
In-Reply-To: <CAH4pYpTeEowOVD1YeiutY_3BnNnSbY6fceezZT3iuGXn2SsohA@mail.gmail.com>
References: <CAP-uhDc5Vensa2p_Ewe-JZOgsMQZ92U5EC9n2+u9AWuKxz4mOw@mail.gmail.com>
	<E9B53750-A7F7-4307-8EF9-D6BF973E4A45@gmail.com>
	<CAP-uhDcQyTE5NHVA2_LEtomRDiP76a2G2+N8tJF5cyPpcyVbew@mail.gmail.com>
	<7BF767A6-2203-4B4E-8CCA-53AEBC85EFE1@gmail.com>
	<CAH4pYpR_L=yoCdhcDi7nF93R4_jsjpUzcWg8_UGFuXOjGzR8Fg@mail.gmail.com>
	<40D15F76-8012-4350-9135-060C9F352A51@gmail.com>
	<CAH4pYpTcrXmhvDpbf9b-5=jJPFP4yAGGLcHSv+s-C2dxRKLt7w@mail.gmail.com>
	<CAP-uhDdKW7sNggOEV6x3ioWCu4LkSKmP_j93Z3UYZQuNahFLsQ@mail.gmail.com>
	<1FA84142-E068-4BF2-A012-9E0A6DDF0BD8@gmail.com>
	<CAP-uhDfDn3HRZwckBMFQT3C8gK=tkSuy_3Q_yQ5CPbmejdOq3g@mail.gmail.com>
	<CAH4pYpTeEowOVD1YeiutY_3BnNnSbY6fceezZT3iuGXn2SsohA@mail.gmail.com>
Message-ID: <CAHAreOpw9=jHB=X9okCWa=TUZ2Y__2fYYOOrcHwXCSdeW8fbLA@mail.gmail.com>

On Fri, Dec 14, 2012 at 10:52 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> * To get input from others on this issue - esp fperez and minrk.  I
> will see them next week and make sure we talk about this.

Been traveling/offline (writing this from a narrow window of time
in-between connections at LAX).  Let's put this high on our todo list
for me to catch up next week, once my travel marathon subsides and I
can plug back into everything.

Thanks everyone for the detailed feedback on this, thinking with
multiple heads about this is extremely useful to the project (esp.
given the core devs aren't traditionally web security people, we're
very happy to  have skills and input from others).

Cheers,

f


From nick.bollweg at gmail.com  Sat Dec 15 00:05:35 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 15 Dec 2012 00:05:35 -0500
Subject: [IPython-dev] custom web notebook cells
Message-ID: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>

sorry, my initial setup of digest makes it hard to get at the thread
structure directly, so hopefully this gets in the right place.

the feedback has been very good, impressive community (and product, of
course)
you have built here!

    > * We need to maintain the ability to install ipython without
setuptool.

whoa, very "there is no spoon." just went back and read some tickets that
mention setuptools/distutils. will respect the community's desires, here!

so i guess, if primarily-javascript packages *were* to be installed via
python,
the IPython Way could be to have some %load_ext cells at the
beginning of the notebook that would somehow change the static
lookup path at runtime. i'll revisit my investigation of the extensions!
once
there is one example like this, it should be easy to repeat the pattern.

    > Before we start actually coding on the require.js stuff or
    > reorganizing that stuff, we need to figure out what we want to do
    > exactly.  Here is an IPEP for us to work all of that out:

thanks for the pointer to the IPEP5, very good scope! while i am learning
to
love node, i might avoid bringing in the node dependency until
someone really has that itch to scratch... i could imagine the 0mq layer
being
perfect for interacting between python and node... under the correct
conditions.

ghost.py can handle the testing requirements, though it will crank up the
travis test runs by a pretty serious amount, due to the installation of qt
and
friends... ah! but The Other Notebook already uses qt, so maybe this isn't
as
bad of a hit. here's the ghost.py .travis.yml that is the recipe i have used
for getting it running:

    https://github.com/jeanphix/Ghost.py/blob/master/.travis.yml

having node around to do npm might not actually meet the needs of
the notebook plugin builder (see below)... webassets looks good, but
doesn't
handle the dependency tree. tough choices ahead.

    > * We have to serve these files on the web.  I don't want to have our
    >   web server walking all over a users system to pull files as that is
a
    >   huge security risk.  We need to isolate the location of these files.

yes, definitely a concern... need to wrap my head around this... what makes
the
most sense? options:

- copy stuff into the ipython location
  - seems bad
- have multiple static_paths
  - my current implementation, relies on runtime setuptools
- collect all static deps into a tmpdir which only contains the files needed
  - has potential...

    > * Some plugins don't have any python code so I am not sure it makes
    >   sense to bundle things into a python package.

indeed, i have fought with this demon a few times, most recently with
apache
allura's homerolled asset management system. putting my sysadmin hat on,
i'd have the fewest dependency management systems possible... apt-get/yum +
pip.
it's an immediate turnoff if i have to do much more than that. packaging as
python does have the whole pypi advantage, though.

of course, what could end up happening is that the plugins would have
versions
as well, and it could be that two different .ipynb's you wanted to look at
would use different versions. this suggests, perhaps, that the plugins
should
be installed into the notebook itself, which also chills me to the bone.

i guess we need to understand how a user would want to use these...
definitely
have to do more thinking.

    > But, I would like to learn more about your usage cases.  There might
    > be other ways of supporting what you want to do:
    > 1) Use an existing cell type with metadata.
    > 2) Provide a custom UI for an existing cell type but don't change the
    >    notebook format.
    > 3) Actually add a new cell type to the notebook format officially if
it
    >    is something that a majority of our users would use often.

my long-term vision is to build on what mathematica/jmp/tableau can do:
support
freeform exploration by a developer/researcher/student of a data-driven
problem, which gradually solidifies into a dashboard (for lack of a better
term) that allows an end user to play in a constrained data space, but
access
advanced features.

while getting there, i want to better be able to document what i am
thinking
about it one, context-switching-free space that versions/merges well.

i never thought of doing anything that would do (3): i am indeed
sensitive to the long-term viability of data formats, and want to do
whatever i
can to avoid contributing to "data death"... all the stuff i want to do is
very
web-focused (for both UI and output), and is therefore inherently
text-based...
it might just be Really Boring text if you are at the command line (hi,
SVG!).
i was thinking about it primarily from a UI point of view, the current
implementation of which centers around the IPython.Cell subclasses... so
basically what i want is what you describe in (2).

my use cases for custom cells are sort of summed up (mostly as
implementation ideas) in this ticket on my "write some d3 visualizations
with
blockly" project:

    https://github.com/bollwyvl/blockd3/issues/10

but here's some riffing on what i've learned in the last 24 hours :)

use cases:

- VisualCodeCell http://code.google.com/p/blockly/
    - motivation
        - i've helped students reach the the "a ha" moment with
          blockly's spiritual ancestor, scratch, but feel like the problems
          that python is able to tackle are much greater. i see the ipython
          notebook as a great home for the blockly python functionality, as
it
          can be learned by the absolute novice... when i was a python
novice,
          i think i had the "a ha" moment about five years ago when i
learned
          about tab completion and ? and ?? in the ipython console!

          i have been experimenting with blockly (which can generate
python) a
          good deal, but exclusively for javascript. presently blockly is
kind
          of lame for a python workflow.
    - implementation
        - CodeCell. blockly xml would fit nicely inside cell metadata,
which i have yet to explore, but need to!
    - challenges
        - the downside here is that any editing of the python outside of
the
          web environment (or if you didn't have the ipython-blockly plugin
i
          envision), the blockly environment can't reverse-engineer its xml
          from python. however, since blockly is more aimed at the novice,
this
          might not cause that much of an issue.
- DiagramCell http://code.google.com/p/svg-edit/
    - motivation
        - i love me some svg... it is deeply ingrained in my day-to-day
design
          and communication workflow with inkscape, and has become
increasingly
          important from my web application development tasks with the
meteoric
          rise of d3. svgedit is Almost As Good as inkscape, and for a lot
of
          things i want to do, it would be sufficient for a lot of quick
          vector-based documentation things that i'd want to attach.
    - implementation
        - MarkdownCell
    - challenges
        - none, really
- SpreadsheetCell https://github.com/bollwyvl/blockd3/issues/SlickGrid
    - motivation
        - some data really is tabular. i would love to be able to whip up a
          quick table (not using markdown!) and then pop that into a numpy
          calculation chain.

          also, being able to quickly walk around higher-order numpy arrays
          would be great, suggesting an excel-like sheet navigation for a
given
          array.
    - implementation
        - CodeCell. perhaps some metadata for formatting
    - challenges
        - not trying to implement Resolver One here, but a little bit of
          functions would be cool... i like some of the things that the
google
          spreadsheet has done to blow out a calculation into a bunch of
cell
          from input into one cell
- TangleCell http://worrydream.com/Tangle/
    - motivation
        - the tangle reactive document paradigm is awesome. i have done
some work with integrating this with markdown, but it never felt right,
          and took a lot of `exec` to work properly, which always gave me
the
          willies. writing constraints directly on the backend and then
just
          tying them to frontend in markdown seems like the high road.
    - implementation
        - MarkdownCell
- ControlCell
    - motivation
        - sometimes, i just want a button or a slider for changing a value.
    - implementation
        - CodeCell. metadata.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/379874d8/attachment.html>

From ellisonbg at gmail.com  Sat Dec 15 01:51:55 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 14 Dec 2012 22:51:55 -0800
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
Message-ID: <CAH4pYpQisX5FxHArLMhvJaKUFzfWYNBXf5Y7VL2kg3jyxWkCcA@mail.gmail.com>

Nicholas,

Awesome, I can tell the notebook has you thinking about lots of cool
things.  I love that.  I will write more soon, but here is the short
answer:

* Once we merge the Javascript plugins branch, we will be able to do
all of the types of "widgets" you mention (SVG, tangle, spreadsheet,
controls, etc) with our current code cells.  In fact, we even have a
huge grant from the Sloan foundation to work on this *exactly* for the
next two years:

http://ipython.org/sloan-grant.html

* I am almost positive that we will grow some sort of node dependency
over time - probably not for running the notebook, but most likely for
dev, testing etc.

* For testing, our best option so far is phantom.js and casper.js.

Cheers,

Brian

On Fri, Dec 14, 2012 at 9:05 PM, Nicholas Bollweg
<nick.bollweg at gmail.com> wrote:
> sorry, my initial setup of digest makes it hard to get at the thread
> structure directly, so hopefully this gets in the right place.
>
> the feedback has been very good, impressive community (and product, of
> course)
> you have built here!
>
>     > * We need to maintain the ability to install ipython without
> setuptool.
>
> whoa, very "there is no spoon." just went back and read some tickets that
> mention setuptools/distutils. will respect the community's desires, here!
>
> so i guess, if primarily-javascript packages *were* to be installed via
> python,
> the IPython Way could be to have some %load_ext cells at the
> beginning of the notebook that would somehow change the static
> lookup path at runtime. i'll revisit my investigation of the extensions!
> once
> there is one example like this, it should be easy to repeat the pattern.
>
>     > Before we start actually coding on the require.js stuff or
>     > reorganizing that stuff, we need to figure out what we want to do
>     > exactly.  Here is an IPEP for us to work all of that out:
>
> thanks for the pointer to the IPEP5, very good scope! while i am learning to
> love node, i might avoid bringing in the node dependency until
> someone really has that itch to scratch... i could imagine the 0mq layer
> being
> perfect for interacting between python and node... under the correct
> conditions.
>
> ghost.py can handle the testing requirements, though it will crank up the
> travis test runs by a pretty serious amount, due to the installation of qt
> and
> friends... ah! but The Other Notebook already uses qt, so maybe this isn't
> as
> bad of a hit. here's the ghost.py .travis.yml that is the recipe i have used
> for getting it running:
>
>     https://github.com/jeanphix/Ghost.py/blob/master/.travis.yml
>
> having node around to do npm might not actually meet the needs of
> the notebook plugin builder (see below)... webassets looks good, but doesn't
> handle the dependency tree. tough choices ahead.
>
>     > * We have to serve these files on the web.  I don't want to have our
>     >   web server walking all over a users system to pull files as that is
> a
>     >   huge security risk.  We need to isolate the location of these files.
>
> yes, definitely a concern... need to wrap my head around this... what makes
> the
> most sense? options:
>
> - copy stuff into the ipython location
>   - seems bad
> - have multiple static_paths
>   - my current implementation, relies on runtime setuptools
> - collect all static deps into a tmpdir which only contains the files needed
>   - has potential...
>
>     > * Some plugins don't have any python code so I am not sure it makes
>     >   sense to bundle things into a python package.
>
> indeed, i have fought with this demon a few times, most recently with apache
> allura's homerolled asset management system. putting my sysadmin hat on,
> i'd have the fewest dependency management systems possible... apt-get/yum +
> pip.
> it's an immediate turnoff if i have to do much more than that. packaging as
> python does have the whole pypi advantage, though.
>
> of course, what could end up happening is that the plugins would have
> versions
> as well, and it could be that two different .ipynb's you wanted to look at
> would use different versions. this suggests, perhaps, that the plugins
> should
> be installed into the notebook itself, which also chills me to the bone.
>
> i guess we need to understand how a user would want to use these...
> definitely
> have to do more thinking.
>
>     > But, I would like to learn more about your usage cases.  There might
>     > be other ways of supporting what you want to do:
>     > 1) Use an existing cell type with metadata.
>     > 2) Provide a custom UI for an existing cell type but don't change the
>     >    notebook format.
>     > 3) Actually add a new cell type to the notebook format officially if
> it
>     >    is something that a majority of our users would use often.
>
> my long-term vision is to build on what mathematica/jmp/tableau can do:
> support
> freeform exploration by a developer/researcher/student of a data-driven
> problem, which gradually solidifies into a dashboard (for lack of a better
> term) that allows an end user to play in a constrained data space, but
> access
> advanced features.
>
> while getting there, i want to better be able to document what i am thinking
> about it one, context-switching-free space that versions/merges well.
>
> i never thought of doing anything that would do (3): i am indeed
> sensitive to the long-term viability of data formats, and want to do
> whatever i
> can to avoid contributing to "data death"... all the stuff i want to do is
> very
> web-focused (for both UI and output), and is therefore inherently
> text-based...
> it might just be Really Boring text if you are at the command line (hi,
> SVG!).
> i was thinking about it primarily from a UI point of view, the current
> implementation of which centers around the IPython.Cell subclasses... so
> basically what i want is what you describe in (2).
>
> my use cases for custom cells are sort of summed up (mostly as
> implementation ideas) in this ticket on my "write some d3 visualizations
> with
> blockly" project:
>
>     https://github.com/bollwyvl/blockd3/issues/10
>
> but here's some riffing on what i've learned in the last 24 hours :)
>
> use cases:
>
> - VisualCodeCell http://code.google.com/p/blockly/
>     - motivation
>         - i've helped students reach the the "a ha" moment with
>           blockly's spiritual ancestor, scratch, but feel like the problems
>           that python is able to tackle are much greater. i see the ipython
>           notebook as a great home for the blockly python functionality, as
> it
>           can be learned by the absolute novice... when i was a python
> novice,
>           i think i had the "a ha" moment about five years ago when i
> learned
>           about tab completion and ? and ?? in the ipython console!
>
>           i have been experimenting with blockly (which can generate python)
> a
>           good deal, but exclusively for javascript. presently blockly is
> kind
>           of lame for a python workflow.
>     - implementation
>         - CodeCell. blockly xml would fit nicely inside cell metadata, which
> i have yet to explore, but need to!
>     - challenges
>         - the downside here is that any editing of the python outside of the
>           web environment (or if you didn't have the ipython-blockly plugin
> i
>           envision), the blockly environment can't reverse-engineer its xml
>           from python. however, since blockly is more aimed at the novice,
> this
>           might not cause that much of an issue.
> - DiagramCell http://code.google.com/p/svg-edit/
>     - motivation
>         - i love me some svg... it is deeply ingrained in my day-to-day
> design
>           and communication workflow with inkscape, and has become
> increasingly
>           important from my web application development tasks with the
> meteoric
>           rise of d3. svgedit is Almost As Good as inkscape, and for a lot
> of
>           things i want to do, it would be sufficient for a lot of quick
>           vector-based documentation things that i'd want to attach.
>     - implementation
>         - MarkdownCell
>     - challenges
>         - none, really
> - SpreadsheetCell https://github.com/bollwyvl/blockd3/issues/SlickGrid
>     - motivation
>         - some data really is tabular. i would love to be able to whip up a
>           quick table (not using markdown!) and then pop that into a numpy
>           calculation chain.
>
>           also, being able to quickly walk around higher-order numpy arrays
>           would be great, suggesting an excel-like sheet navigation for a
> given
>           array.
>     - implementation
>         - CodeCell. perhaps some metadata for formatting
>     - challenges
>         - not trying to implement Resolver One here, but a little bit of
>           functions would be cool... i like some of the things that the
> google
>           spreadsheet has done to blow out a calculation into a bunch of
> cell
>           from input into one cell
> - TangleCell http://worrydream.com/Tangle/
>     - motivation
>         - the tangle reactive document paradigm is awesome. i have done some
> work with integrating this with markdown, but it never felt right,
>           and took a lot of `exec` to work properly, which always gave me
> the
>           willies. writing constraints directly on the backend and then just
>           tying them to frontend in markdown seems like the high road.
>     - implementation
>         - MarkdownCell
> - ControlCell
>     - motivation
>         - sometimes, i just want a button or a slider for changing a value.
>     - implementation
>         - CodeCell. metadata.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Sat Dec 15 06:39:49 2012
From: carl.input at gmail.com (Carl Smith)
Date: Sat, 15 Dec 2012 11:39:49 +0000
Subject: [IPython-dev] What is "Print View" supposed to do?
In-Reply-To: <4B8905EA-2713-4E31-A034-5F6CC081A3E9@gmail.com>
References: <40C7B1BFC291ED4E9D10436D07736A33470B8F2F9E@EXMAIL7.haas.uc.berkeley.edu>
	<4B8905EA-2713-4E31-A034-5F6CC081A3E9@gmail.com>
Message-ID: <CAP-uhDdFTrPbjt3Fx_YoEKqLVoXELBr9W3t4BS1WYaL+ui6zyw@mail.gmail.com>

You can also use nbconvert to convert notebooks to other formats.


From bussonniermatthias at gmail.com  Sat Dec 15 06:47:16 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 12:47:16 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
	<CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>
Message-ID: <FFBA3218-A191-47FB-9438-58D9AB21694C@gmail.com>


Le 14 d?c. 2012 ? 19:32, Brian Granger a ?crit :

> Nicholas,
> 
> Hi!
> 
> On Fri, Dec 14, 2012 at 5:13 AM, Nicholas Bollweg
> <nick.bollweg at gmail.com> wrote:
>> hi everybody! longtime user, first time poster.
>> 
>> i've been kicking around a lot of ideas of things to do with ipython
>> notebook: while the bulk of stuff is clientside, to get there, i think i
>> need to do custom cells, which is a little more than ellisonbg's
>> "jsonhandlers" branch does. further, as the specific capabilities exposed
>> will require some pretty big depndencies (pdf generation, numpy, etc.) I am
>> thinking about doing everything as python modules that are installed into
>> the environment, rather than as purely drop-in javascript assets.
>> 
>> i've started the work on github here:
>> https://github.com/bollwyvl/ipython/tree/custom_cells
>> 
>> and a strawman cell provider here (does not contain working code, though it
>> does load!):
>> https://github.com/bollwyvl/ipython/tree/blockly
>> 
>> any insight is welcome!
> 
> Can you talk more about the usage cases you have for new cell types?
> Can you give examples?
> 
> The notebook document format is sort of our contract with the
> universe.  We want to be able to have an extremely precise and
> testable definition of what constitutes an IPython notebook, so that
> everyone can open them, share them, convert them, run them etc.  If we
> allow custom cell types, all of that goes out the door, so I don't see
> how we can support custom cell types.  The notebook document format
> would stop being the notebook document format.

I think I have to disagree with you, 
You could make the same remarks with custom joan handler. 
Actually if someone does not have the plugin they might not be able to see 
the json repr. 

And also we have the metadata for people to store almost any information they want.
Also you can imagine people can want custom cell even just to replace our ours for X
reason. 

I don't think they ask us to support those different cell but to allow easier hooks to add them.

But actually think the problem is what you call a **different cell**.
One could have 2 different type of code cell and you can choose which one to get 
with the metadata.

> But, I would like to learn more about your usage cases.  There might
> be other ways of supporting what you want to do:
> 
> * Use an existing cell type with metadata.
> * Provide a custom UI for an existing cell type but don't change the
> notebook format.
> * Actually add a new cell type to the notebook format officially if it
> is something that a majority of our users would use often.

I guess there is a confusion between cell and cell/type.
and looking a blocky it seem the desire cell here is a code-cell, but with 
visual programming where you move block. 


> 
>> to get my stuff in, i have used the setuptools iter_entry_points method I am
>> familiar with... and defined a new entry_point, ipnotebook. plugins provide
>> their list of assets and where one might find their static paths. can anyone
>> comment on how this is (potentially) different from The IPython Way? The
>> extensions stuff in the doc didn't look like they would be useful, so i gave
>> up on trying to use that.
> 
> I am very interested in learning more about this.  Right now my
> jsplugins/handlers branch uses a pretty simple way of installing
> additional Js code = a user just drops it into a special dir in the
> .ipython directory.  It would be nice if these Js plugins were regular
> python modules that could be installed as usual.  But something to
> keep in mind:
> 
> * We have to serve these files on the web.  I don't want to have our
> web server walking all over a users system to pull files as that is a
> huge security risk.  We need to isolate the location of these files.
> * Some plugins don't have any python code so I am not sure it makes
> sense to bundle things into a python package.
> * We need to maintain the ability to install ipython without setuptool.
> 
> Do you have ideas on how we can improve the user experience of
> installing and managing Js plugins?
> 
>> the structure of working with cells in master is pretty rigid right now,
>> with the enumerated list of cells type in the code explicitly in the front
>> end in several places, js and html. my initial work (in a addition to
>> inadvertently recreating some of ellisonbg's work on the tornado app) has
>> been to refactor all of those cases of adding buttons or whatever into
>> something that looks at a window.IPython-scope plugin registry.... this is
>> defined in cell.js right now, but maybe should be someplace else.
> 
> That rigidity is merely a reflection of the notebook format.
It is still pretty flexible as long as you respect the structure. 
there are already inconsistencies, 
But if you change code cell `code` field to `source` field then all the cell have almost perfectly the same structure. 
And this structure should be enough for almost any cell type. 

The subtleties going into metadata.
-- 
Matthias



From bussonniermatthias at gmail.com  Sat Dec 15 08:01:31 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 14:01:31 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CAH4pYpQisX5FxHArLMhvJaKUFzfWYNBXf5Y7VL2kg3jyxWkCcA@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpQisX5FxHArLMhvJaKUFzfWYNBXf5Y7VL2kg3jyxWkCcA@mail.gmail.com>
Message-ID: <34E6EB58-9660-44FD-B773-C526BE726C94@gmail.com>


Le 15 d?c. 2012 ? 07:51, Brian Granger a ?crit :

> Nicholas,
> 
> Awesome, I can tell the notebook has you thinking about lots of cool
> things.  I love that.  I will write more soon, but here is the short
> answer:
> 
> * Once we merge the Javascript plugins branch, we will be able to do
> all of the types of "widgets" you mention (SVG, tangle, spreadsheet,
> controls, etc) with our current code cells.  In fact, we even have a
> huge grant from the Sloan foundation to work on this *exactly* for the
> next two years:

Brian, 

The Javascript plugin branch does allow to have that in **output** right ? 
I think what is requested here is to directly have it as a cell, not something that goes through the kernel, 
so actually not needing to write some python code.

-- 
Matthias




From nick.bollweg at gmail.com  Sat Dec 15 12:26:48 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 15 Dec 2012 12:26:48 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <34E6EB58-9660-44FD-B773-C526BE726C94@gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpQisX5FxHArLMhvJaKUFzfWYNBXf5Y7VL2kg3jyxWkCcA@mail.gmail.com>
	<34E6EB58-9660-44FD-B773-C526BE726C94@gmail.com>
Message-ID: <CACejjWzQnEiEMk-SBGfvv_0p4xW45NKgxUV0Q2bSgf5ncEzDqQ@mail.gmail.com>

i have checked out the jshandlers branch and gotten all the stuff working
with the nx example. very cool! exactly the example i needed to get started.

i now see how the %load can be used, but still find the static asset
population cantankerous. my initial implementation copies the assets to the
current profile's static for you, but this still takes an app restart to
take effect. require can't come soon enough!

@matthias:
you hit it on the head re: jsonhandlers for input vs output. i would
ideally inject a widget into CodeCell.prototype.create_element to create my
blockly editor... right now doing a kernel-side Blockly class that
represents as a JSON object to trigger my handler, which then shows a
blockly editor, which, when updated the next code block and executes it...
Hack, sure. Want to see if it works!

a little more meta:
one of the best things I've seen for truly nice js/css plugins is
kickstrap... the "app store" feel is also very cool... desirable for
ipython? to my knowledge, however, it doesn't allow for addition of
apps/themes/extras at runtime, really. I do think bootstrap is worth a
look, though... There's already a theme on that looks like the current
theme.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/7731852d/attachment.html>

From takowl at gmail.com  Sat Dec 15 14:07:06 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 15 Dec 2012 19:07:06 +0000
Subject: [IPython-dev] Template cells
Message-ID: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>

Revisiting a discussion from a few days ago, to get more input.

At one point, we were considering ways to interpolate variables from the
Python namespace into Markdown cells, using custom syntax and transferring
simple variables using the user_expressions field of execution messages.

Prompted by another discussion, I felt that this approach was quite
limiting, and a better approach would be to have a 'template cell',
designed like this:

- To the user, it works like a Markdown cell; when they leave the cell, the
editor is replaced by the rendered version of the contents.
- Since we now depend on Jinja2 for the notebook, we use the template
syntax from that, rather than reinventing a restricted template syntax
ourselves. This gives us things like loops in the templates.
- To render the cell, it is sent to the kernel as an execution request.
- The rendering uses our display infrastructure, so e.g. the HTML
representation of an object is preferred. But the template can override
this to get the plain text repr.

Open questions:
- Should the templates be in Markdown (for consistency with other rich-text
cells), or in HTML (which is more common in templating)?
- If in Markdown, should they be an automatic feature of Markdown cells
(i.e. the Javascript looks for template syntax, and decides to send it to
the kernel)? Or is that too surprising?
- How can we ensure notebooks with template cells work with current
versions of IPython, which don't know about them?
- Should there be any way to tell template cells to automatically
re-render, e.g. on a timer, or after another cell is executed?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/824b924c/attachment.html>

From bussonniermatthias at gmail.com  Sat Dec 15 15:03:23 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 21:03:23 +0100
Subject: [IPython-dev] Template cells
In-Reply-To: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
Message-ID: <3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>


Le 15 d?c. 2012 ? 20:07, Thomas Kluyver a ?crit :

> Revisiting a discussion from a few days ago, to get more input.
> 
> At one point, we were considering ways to interpolate variables from the Python namespace into Markdown cells, using custom syntax and transferring simple variables using the user_expressions field of execution messages.
> 
> Prompted by another discussion, I felt that this approach was quite limiting, and a better approach would be to have a 'template cell', designed like this:
> 
I like the idea of template, I'm still wondering if it is not going too far.

> - To the user, it works like a Markdown cell; when they leave the cell, the editor is replaced by the rendered version of the contents.
> - Since we now depend on Jinja2 for the notebook, we use the template syntax from that, rather than reinventing a restricted template syntax ourselves. This gives us things like loops in the templates.

Jinja is python only, am I wrong ? And feel this would be a little too restrictive, moreover it will mix what the kernel does and what the server does. 
I'm not a big fan of having the server computing repr for the frontend... (even if we will have to go through this for multiuser) 

> - To render the cell, it is sent to the kernel as an execution request.
You stat to imply that the kernel does know it speaks to notebook...

> - The rendering uses our display infrastructure, so e.g. the HTML representation of an object is preferred. But the template can override this to get the plain text repr.
> 
> Open questions:
> - Should the templates be in Markdown (for consistency with other rich-text cells), or in HTML (which is more common in templating)?
> - If in Markdown, should they be an automatic feature of Markdown cells (i.e. the Javascript looks for template syntax, and decides to send it to the kernel)? Or is that too surprising?
> - How can we ensure notebooks with template cells work with current versions of IPython, which don't know about them?
This one is tough.

> - Should there be any way to tell template cells to automatically re-render, e.g. on a timer, or after another cell is executed?
It would lead to confusion unless the cell is explicitly and visually marked as being a live representation of the kernel. 

Why not a Jinja2 object in python that could return what you wish ? 

If you can show user_expression/variable in mdcell, you just have to reference this Jinja object... it has the same effect no ? 
And you could change the template engine if the kernel is not Python...

-- 
Matthias



From ellisonbg at gmail.com  Sat Dec 15 15:13:48 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 15 Dec 2012 12:13:48 -0800
Subject: [IPython-dev] Template cells
In-Reply-To: <3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
Message-ID: <CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>

On Sat, Dec 15, 2012 at 12:03 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 15 d?c. 2012 ? 20:07, Thomas Kluyver a ?crit :
>
>> Revisiting a discussion from a few days ago, to get more input.
>>
>> At one point, we were considering ways to interpolate variables from the Python namespace into Markdown cells, using custom syntax and transferring simple variables using the user_expressions field of execution messages.
>>
>> Prompted by another discussion, I felt that this approach was quite limiting, and a better approach would be to have a 'template cell', designed like this:
>>
> I like the idea of template, I'm still wondering if it is not going too far.

Hmm, there are somethings that i like about this for sure.  But I
worry that we are creating some nasty bastard child of Markdown +
LateX/MathJax + Templating that is going to haunt in in our later
years when we have to care for this beast.  This is a concern as other
people create frontends for the notebook - it will become difficult
the further we deviate away from plain Markdown

>> - To the user, it works like a Markdown cell; when they leave the cell, the editor is replaced by the rendered version of the contents.
>> - Since we now depend on Jinja2 for the notebook, we use the template syntax from that, rather than reinventing a restricted template syntax ourselves. This gives us things like loops in the templates.
>
> Jinja is python only, am I wrong ? And feel this would be a little too restrictive, moreover it will mix what the kernel does and what the server does.
> I'm not a big fan of having the server computing repr for the frontend... (even if we will have to go through this for multiuser)

In 10 years, the IPython Notebok will be largely language independent.
 Because of this, I don't think we want to pick a templating language
that is native to python.  It would need to be more general.

>> - To render the cell, it is sent to the kernel as an execution request.
> You stat to imply that the kernel does know it speaks to notebook...

I *really* want to avoid having to involve the server to render
Markdown cells.  Are there any nice JS templating engines?

>> - The rendering uses our display infrastructure, so e.g. the HTML representation of an object is preferred. But the template can override this to get the plain text repr.
>>
>> Open questions:
>> - Should the templates be in Markdown (for consistency with other rich-text cells), or in HTML (which is more common in templating)?
>> - If in Markdown, should they be an automatic feature of Markdown cells (i.e. the Javascript looks for template syntax, and decides to send it to the kernel)? Or is that too surprising?
>> - How can we ensure notebooks with template cells work with current versions of IPython, which don't know about them?
> This one is tough.
>
>> - Should there be any way to tell template cells to automatically re-render, e.g. on a timer, or after another cell is executed?
> It would lead to confusion unless the cell is explicitly and visually marked as being a live representation of the kernel.
>
> Why not a Jinja2 object in python that could return what you wish ?
>
> If you can show user_expression/variable in mdcell, you just have to reference this Jinja object... it has the same effect no ?
> And you could change the template engine if the kernel is not Python...

My gut feeling on this is that there are some nice ideas in this, but
that we need to sit on it for now and see how things develop with the
notebook and interactive widgets, which will provide an incredible
amount of flexibility in the notebook.  As a simpler target, I think
we should go back to Matthias's literate programming PR and think
carefully about the syntax of it.  I *love* the capability, but am
very hesitant to deviate from Markdown any further than we already
have.

Cheers,

Brian

> --
> Matthias
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Sat Dec 15 15:31:10 2012
From: carl.input at gmail.com (Carl Smith)
Date: Sat, 15 Dec 2012 20:31:10 +0000
Subject: [IPython-dev] Template cells
In-Reply-To: <CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
Message-ID: <CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>

I'm not sure if you guys have seen this, but there's a Python Markdown
project that might be useful here. It does mean a round trip for
rendering Markdown, but if you did decide to go in that direction
anyway, this offers an extensible Markdown renderer that we might be
able to build on. I love Markdown, but it would be nice to be able to
extend the language to include Notebook specific features.

http://freewisdom.org/projects/python-markdown/


From ellisonbg at gmail.com  Sat Dec 15 15:37:13 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 15 Dec 2012 12:37:13 -0800
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <FFBA3218-A191-47FB-9438-58D9AB21694C@gmail.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
	<CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>
	<FFBA3218-A191-47FB-9438-58D9AB21694C@gmail.com>
Message-ID: <CAH4pYpQ9bMotBYWJ0b0LXgzMwM=tzvuZta1CZhf_mVsD0jEy8w@mail.gmail.com>

OK, lots of great discussion here, going in many different directions,
I will try to keep up...

>> The notebook document format is sort of our contract with the
>> universe.  We want to be able to have an extremely precise and
>> testable definition of what constitutes an IPython notebook, so that
>> everyone can open them, share them, convert them, run them etc.  If we
>> allow custom cell types, all of that goes out the door, so I don't see
>> how we can support custom cell types.  The notebook document format
>> would stop being the notebook document format.
>
> I think I have to disagree with you,
> You could make the same remarks with custom joan handler.
> Actually if someone does not have the plugin they might not be able to see
> the json repr.

I view Javascript plugins in the same way as I view python packages:

* They have to be installed by the person who manages the notebook server.
* They are optional.
*  People who want to run notebooks that use various  plugins and
python packages will have to make sure they are installed.

These things are transient features of the notebook.  On the other
hand, there are certain things that will be entirely invariant over
all ipython notebook experiences in the universe:

* The notebook document format.
* The kernel messaging protocol (note that kernels can be different,
but they will all speak the same protocol).

> And also we have the metadata for people to store almost any information they want.
> Also you can imagine people can want custom cell even just to replace our ours for X
> reason.

I have started to become very concerned about the metadata.  Our
thinking so far has been: that metadata is complete unstructured.
IOW, users can write to the metadata section whatever they want.  When
nobody is doing that it works fine, but as more people start to use
the metadata, it is going to turn into the wild west.  There will be
conflicting metadata.  There will backwards incompatibilties.  We will
allow this for a while, but when users start to complain that there
notebooks don't work as expected, we will have to become extremely
conservative about the metadata.

Case in point: we are about to add a slideshow section to the cell
level metadata.  And the notebook UI will rely on this metadata being
in a very particular format.  If we *ever* change that metadata in the
future, we will have to increment the notebook document format version
to reflect it.  Some changes (adding a new attribute) won't be
backwards incompatible, but some might be and it will affect the
entire notebook document format.

> I don't think they ask us to support those different cell but to allow easier hooks to add them.
>
> But actually think the problem is what you call a **different cell**.
> One could have 2 different type of code cell and you can choose which one to get
> with the metadata.

I am open to using metadata to control how cells act.  We are already
doing this with the slide show and this may be a very powerful way of
moving forward.

>> But, I would like to learn more about your usage cases.  There might
>> be other ways of supporting what you want to do:
>>
>> * Use an existing cell type with metadata.
>> * Provide a custom UI for an existing cell type but don't change the
>> notebook format.
>> * Actually add a new cell type to the notebook format officially if it
>> is something that a majority of our users would use often.
>
> I guess there is a confusion between cell and cell/type.
> and looking a blocky it seem the desire cell here is a code-cell, but with
> visual programming where you move block.
>
>
>>
>>> to get my stuff in, i have used the setuptools iter_entry_points method I am
>>> familiar with... and defined a new entry_point, ipnotebook. plugins provide
>>> their list of assets and where one might find their static paths. can anyone
>>> comment on how this is (potentially) different from The IPython Way? The
>>> extensions stuff in the doc didn't look like they would be useful, so i gave
>>> up on trying to use that.
>>
>> I am very interested in learning more about this.  Right now my
>> jsplugins/handlers branch uses a pretty simple way of installing
>> additional Js code = a user just drops it into a special dir in the
>> .ipython directory.  It would be nice if these Js plugins were regular
>> python modules that could be installed as usual.  But something to
>> keep in mind:
>>
>> * We have to serve these files on the web.  I don't want to have our
>> web server walking all over a users system to pull files as that is a
>> huge security risk.  We need to isolate the location of these files.
>> * Some plugins don't have any python code so I am not sure it makes
>> sense to bundle things into a python package.
>> * We need to maintain the ability to install ipython without setuptool.
>>
>> Do you have ideas on how we can improve the user experience of
>> installing and managing Js plugins?
>>
>>> the structure of working with cells in master is pretty rigid right now,
>>> with the enumerated list of cells type in the code explicitly in the front
>>> end in several places, js and html. my initial work (in a addition to
>>> inadvertently recreating some of ellisonbg's work on the tornado app) has
>>> been to refactor all of those cases of adding buttons or whatever into
>>> something that looks at a window.IPython-scope plugin registry.... this is
>>> defined in cell.js right now, but maybe should be someplace else.
>>
>> That rigidity is merely a reflection of the notebook format.
> It is still pretty flexible as long as you respect the structure.
> there are already inconsistencies,
> But if you change code cell `code` field to `source` field then all the cell have almost perfectly the same structure.
> And this structure should be enough for almost any cell type.
>
> The subtleties going into metadata.
> --
> Matthias
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sat Dec 15 15:41:13 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 15 Dec 2012 12:41:13 -0800
Subject: [IPython-dev] Template cells
In-Reply-To: <CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
Message-ID: <CAH4pYpQ39ibx6WkXCJGPZ4ACPjTMun1wbSy2BQftUMxpwGi3Zw@mail.gmail.com>

I am aware of this implementation.  I just want to avoid the round
tripping for Markdown.  We would loose an incredible amount of
flexibility and responsiveness.  If JavaScript *couldn't* do things
like render Markdown, that would be a different story, but JavaScript
is more than capable of handling all of our need in this area.

On Sat, Dec 15, 2012 at 12:31 PM, Carl Smith <carl.input at gmail.com> wrote:
> I'm not sure if you guys have seen this, but there's a Python Markdown
> project that might be useful here. It does mean a round trip for
> rendering Markdown, but if you did decide to go in that direction
> anyway, this offers an extensible Markdown renderer that we might be
> able to build on. I love Markdown, but it would be nice to be able to
> extend the language to include Notebook specific features.
>
> http://freewisdom.org/projects/python-markdown/
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Sat Dec 15 15:52:28 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 21:52:28 +0100
Subject: [IPython-dev] Template cells
In-Reply-To: <CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
Message-ID: <E1F7B08A-89F7-4EEC-B33D-26BD393D5B05@gmail.com>


Le 15 d?c. 2012 ? 21:31, Carl Smith a ?crit :

> I'm not sure if you guys have seen this, but there's a Python Markdown
> project that might be useful here.

And this MD implementation is what nbconvert/nbviewer uses.

Thanks for pointing this though. 
-- 
Matthias

From nick.bollweg at gmail.com  Sat Dec 15 15:56:30 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 15 Dec 2012 15:56:30 -0500
Subject: [IPython-dev] Template cells
In-Reply-To: <E1F7B08A-89F7-4EEC-B33D-26BD393D5B05@gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
	<E1F7B08A-89F7-4EEC-B33D-26BD393D5B05@gmail.com>
Message-ID: <CACejjWwh8Oodr5SWJo5OOSedVON4+DsuvFMtTHzQOibSO9e+dA@mail.gmail.com>

JS templating:
Mustache.js is pretty darn popular, as are its clones. dust.js is worth a
look, as it is already ready for the kind of "zippy" stuff that 0mq can
do... there's a lot of good competition in the space right now, so people
are actually competing on performance, usability and features :)

A use case:
I had previously done some work with implementing a tangle.js (
http://worrydream.com/Tangle/) markdown extension (
https://github.com/bollwyvl/TangleDown). I ended up doing all the parsing
in py, which was not A Good Idea, as it turns out... whereas doing the
heavy duty constrain solving to get bidirectional stuff didn't work very
well in JS. Bad choice of tools, i guess.

I'd like to replace (to the extent possible) all of the javascript exec'd
stuff with bindings to kernel values, while keeping all of the constraint
logic in the backend: thus, I'd like user modifications to the front end to
be bound into execution requests to re-update the front-end without
rerendering the whole template.

This goes a little beyond what you are proposing, but there may be some
common ground.

A low-risk implementation:
Based on the stuff i am working on right now, it could look something like
this:
- store the templated markdown in a degenerate code cell:

    %load_ext tmplmd
    display(TemplatedMarkdown(""), context={})

- enable an input widget, something being discussed on the custom cells
thread... it could be hacked in already with an output handler... but i
digress. Anyhow, somehow show a UI that generates normal python code... in
this case, you'd update the arguments to TemplatedMarkdown, so it becomes:

    display(TemplatedMarkdown("""
    # {{title}}

    {{ for result in results }}
      - {{ result }}
    {{ endfor }}""",
    context=dict(title=foo, results=[bar, baz]))

- on the python side (where you use load_ipython_extension, see brian's
example in
https://github.com/ipython/jsplugins/blob/master/d3graph/d3graph.py)
implement a formatter that returns something like:
    {
        "template": <the template>,
        "context": {
            "title": foo,
            "results": ["foo", "bar"]
        },
        "handler": "tmplmd"
    }
- and then let the client take over to actually render it.

the ipynb just knows there is some code, and the text would stay mostly
human readable and editable for other consumers.


On Sat, Dec 15, 2012 at 3:52 PM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

>
> Le 15 d?c. 2012 ? 21:31, Carl Smith a ?crit :
>
> > I'm not sure if you guys have seen this, but there's a Python Markdown
> > project that might be useful here.
>
> And this MD implementation is what nbconvert/nbviewer uses.
>
> Thanks for pointing this though.
> --
> Matthias
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/9a7db74d/attachment.html>

From jason-sage at creativetrax.com  Sat Dec 15 15:59:55 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 13:59:55 -0700
Subject: [IPython-dev] Template cells
In-Reply-To: <CAH4pYpQ39ibx6WkXCJGPZ4ACPjTMun1wbSy2BQftUMxpwGi3Zw@mail.gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
	<CAH4pYpQ39ibx6WkXCJGPZ4ACPjTMun1wbSy2BQftUMxpwGi3Zw@mail.gmail.com>
Message-ID: <50CCE4CB.1030108@creativetrax.com>

On 12/15/12 1:41 PM, Brian Granger wrote:
> I am aware of this implementation.  I just want to avoid the round
> tripping for Markdown.  We would loose an incredible amount of
> flexibility and responsiveness.  If JavaScript *couldn't* do things
> like render Markdown, that would be a different story, but JavaScript
> is more than capable of handling all of our need in this area.

We could emphasize the utility of no round-tripping for markdown by 
having a live preview, which in general is incredibly convenient for 
double-checking the output is what you intend.

On another of Thomas's points:
 > - To render the cell, it is sent to the kernel as an execution request.

I'm not sure if you meant we should form a valid execute request, but I 
think we should definitely *not* construct a string which is an execute 
request (i.e., we should not be constructing python code in the 
browser).  If we need the server to execute some code in a special way, 
there should be a message type or metadata indicating this, with the 
string, rather than trying to construct a python execute request.  In 
other words, we should not be doing something like:

s = 'HTML(template.render('+escape(code)+')';
kernel.execute_request(s);


A "template cell" is really a special case of an interactive widget.  I 
agree with Brian that we should examine this and some other ideas about 
getting/setting variable values (e.g., interactive widgets).  For 
example, I can imagine a render of a template cell knowing a few code 
expressions that it needs, and shipping those independently to the 
notebook for evaluation, then plugging in the results itself.  In fact, 
it could easily be arranged to have the IPython kernel push updates to 
the cell if IPython noticed that one of the variables changed values, so 
you'd get realtime updates.

Thanks,

Jason



From ellisonbg at gmail.com  Sat Dec 15 16:00:33 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 15 Dec 2012 13:00:33 -0800
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
Message-ID: <CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>

On Fri, Dec 14, 2012 at 9:05 PM, Nicholas Bollweg
<nick.bollweg at gmail.com> wrote:
> sorry, my initial setup of digest makes it hard to get at the thread
> structure directly, so hopefully this gets in the right place.
>
> the feedback has been very good, impressive community (and product, of
> course)
> you have built here!
>
>     > * We need to maintain the ability to install ipython without
> setuptool.
>
> whoa, very "there is no spoon." just went back and read some tickets that
> mention setuptools/distutils. will respect the community's desires, here!

Haha, very funny way of putting it.  You should ask Fernando abou
setuptools some time...

> so i guess, if primarily-javascript packages *were* to be installed via
> python,
> the IPython Way could be to have some %load_ext cells at the
> beginning of the notebook that would somehow change the static
> lookup path at runtime. i'll revisit my investigation of the extensions!
> once
> there is one example like this, it should be easy to repeat the pattern.

We could have the notebook server copy the static assets into a
special directory upon starting.  That wouldn't be too difficult.

>     > Before we start actually coding on the require.js stuff or
>     > reorganizing that stuff, we need to figure out what we want to do
>     > exactly.  Here is an IPEP for us to work all of that out:
>
> thanks for the pointer to the IPEP5, very good scope! while i am learning to
> love node, i might avoid bringing in the node dependency until
> someone really has that itch to scratch... i could imagine the 0mq layer
> being
> perfect for interacting between python and node... under the correct
> conditions.

I haven't mentioned it before, but I am having very conflicted
thoughts about require.js.  On the one hand, we really need something
like it, but on the other I really don't like the constraints that it
places on a project.  I think we need to investigate this further (in
the context of IPEP 5).

> ghost.py can handle the testing requirements, though it will crank up the
> travis test runs by a pretty serious amount, due to the installation of qt
> and
> friends... ah! but The Other Notebook already uses qt, so maybe this isn't
> as
> bad of a hit. here's the ghost.py .travis.yml that is the recipe i have used
> for getting it running:
>
>     https://github.com/jeanphix/Ghost.py/blob/master/.travis.yml
>
> having node around to do npm might not actually meet the needs of
> the notebook plugin builder (see below)... webassets looks good, but doesn't
> handle the dependency tree. tough choices ahead.
>
>     > * We have to serve these files on the web.  I don't want to have our
>     >   web server walking all over a users system to pull files as that is
> a
>     >   huge security risk.  We need to isolate the location of these files.
>
> yes, definitely a concern... need to wrap my head around this... what makes
> the
> most sense? options:
>
> - copy stuff into the ipython location
>   - seems bad
> - have multiple static_paths
>   - my current implementation, relies on runtime setuptools
> - collect all static deps into a tmpdir which only contains the files needed
>   - has potential...

Yes, I am liking this third option.

>     > * Some plugins don't have any python code so I am not sure it makes
>     >   sense to bundle things into a python package.
>
> indeed, i have fought with this demon a few times, most recently with apache
> allura's homerolled asset management system. putting my sysadmin hat on,
> i'd have the fewest dependency management systems possible... apt-get/yum +
> pip.
> it's an immediate turnoff if i have to do much more than that. packaging as
> python does have the whole pypi advantage, though.

Yes, I really don't want to have to invent yet another package format
for all of this.

> of course, what could end up happening is that the plugins would have
> versions
> as well, and it could be that two different .ipynb's you wanted to look at
> would use different versions. this suggests, perhaps, that the plugins
> should
> be installed into the notebook itself, which also chills me to the bone.
>
> i guess we need to understand how a user would want to use these...
> definitely
> have to do more thinking.
>
>     > But, I would like to learn more about your usage cases.  There might
>     > be other ways of supporting what you want to do:
>     > 1) Use an existing cell type with metadata.
>     > 2) Provide a custom UI for an existing cell type but don't change the
>     >    notebook format.
>     > 3) Actually add a new cell type to the notebook format officially if
> it
>     >    is something that a majority of our users would use often.
>
> my long-term vision is to build on what mathematica/jmp/tableau can do:
> support
> freeform exploration by a developer/researcher/student of a data-driven
> problem, which gradually solidifies into a dashboard (for lack of a better
> term) that allows an end user to play in a constrained data space, but
> access
> advanced features.

This very much matches our overall vision.

> while getting there, i want to better be able to document what i am thinking
> about it one, context-switching-free space that versions/merges well.
>
> i never thought of doing anything that would do (3): i am indeed
> sensitive to the long-term viability of data formats, and want to do
> whatever i
> can to avoid contributing to "data death"... all the stuff i want to do is
> very
> web-focused (for both UI and output), and is therefore inherently
> text-based...
> it might just be Really Boring text if you are at the command line (hi,
> SVG!).
> i was thinking about it primarily from a UI point of view, the current
> implementation of which centers around the IPython.Cell subclasses... so
> basically what i want is what you describe in (2).
>
> my use cases for custom cells are sort of summed up (mostly as
> implementation ideas) in this ticket on my "write some d3 visualizations
> with
> blockly" project:
>
>     https://github.com/bollwyvl/blockd3/issues/10
>
> but here's some riffing on what i've learned in the last 24 hours :)

I think there is a big design pattern that is emerging in the
notebook.  Currently CodeCells have an input area and output area.
Obviously, these were originally created for code and the output of
the code.  But, with the Javascript plugins, the output area has
really become more than merely output.  In fact here is how I view it
now...

* The input area is used for python code that initializes a "widget".
That Python code is used to create the widget, configure it, provide
input to it, etc.
* The output area is used for the widget UI.  Widgets can be arbitrary
JS/HTML/CSS.  Widgets can make calls back to the kernel and receive
further JSON data back.  Widgets can even be nested.

Here is an example that I have started to work on.  The new Web Audio
API allows Javascript to access the raw audio data from a microphone.
Using this, it is possible to create a widget that records audio and
processes and visualizes the audio in JS+Python+matplotlib+d3+etc.
Essentially, these widgets become mini-applications that can leverage
the power of both Python and JavaScript.  I think the importance of
JavaScript in this vision cannot be overstated.  Libraries like d3 are
really transforming the landscape of what is possible, not just in the
web browser, but across all platforms.

I think there is a lot of design work to do to make it easy to write
these widgets and allow them to communicate with Python.  But this is
what we have funding for and will start to work on in Jan.

> use cases:
>
> - VisualCodeCell http://code.google.com/p/blockly/
>     - motivation
>         - i've helped students reach the the "a ha" moment with
>           blockly's spiritual ancestor, scratch, but feel like the problems
>           that python is able to tackle are much greater. i see the ipython
>           notebook as a great home for the blockly python functionality, as
> it
>           can be learned by the absolute novice... when i was a python
> novice,
>           i think i had the "a ha" moment about five years ago when i
> learned
>           about tab completion and ? and ?? in the ipython console!
>
>           i have been experimenting with blockly (which can generate python)
> a
>           good deal, but exclusively for javascript. presently blockly is
> kind
>           of lame for a python workflow.
>     - implementation
>         - CodeCell. blockly xml would fit nicely inside cell metadata, which
> i have yet to explore, but need to!
>     - challenges
>         - the downside here is that any editing of the python outside of the
>           web environment (or if you didn't have the ipython-blockly plugin
> i
>           envision), the blockly environment can't reverse-engineer its xml
>           from python. however, since blockly is more aimed at the novice,
> this
>           might not cause that much of an issue.

I don't see any reason this is not possible.

> - DiagramCell http://code.google.com/p/svg-edit/
>     - motivation
>         - i love me some svg... it is deeply ingrained in my day-to-day
> design
>           and communication workflow with inkscape, and has become
> increasingly
>           important from my web application development tasks with the
> meteoric
>           rise of d3. svgedit is Almost As Good as inkscape, and for a lot
> of
>           things i want to do, it would be sufficient for a lot of quick
>           vector-based documentation things that i'd want to attach.
>     - implementation
>         - MarkdownCell
>     - challenges
>         - none, really

This would be awesome and I think it too would fit within our vision.

> - SpreadsheetCell https://github.com/bollwyvl/blockd3/issues/SlickGrid
>     - motivation
>         - some data really is tabular. i would love to be able to whip up a
>           quick table (not using markdown!) and then pop that into a numpy
>           calculation chain.
>
>           also, being able to quickly walk around higher-order numpy arrays
>           would be great, suggesting an excel-like sheet navigation for a
> given
>           array.
>     - implementation
>         - CodeCell. perhaps some metadata for formatting
>     - challenges
>         - not trying to implement Resolver One here, but a little bit of
>           functions would be cool... i like some of the things that the
> google
>           spreadsheet has done to blow out a calculation into a bunch of
> cell
>           from input into one cell

While we didn't put this into our grant specifically, it is something
we have thought about.  I think it is mostly a question of designing
the base architecture well and them having someone tackle it..

> - TangleCell http://worrydream.com/Tangle/
>     - motivation
>         - the tangle reactive document paradigm is awesome. i have done some
> work with integrating this with markdown, but it never felt right,
>           and took a lot of `exec` to work properly, which always gave me
> the
>           willies. writing constraints directly on the backend and then just
>           tying them to frontend in markdown seems like the high road.
>     - implementation
>         - MarkdownCell

We need to think more about how to handle things like this.  It is
related to our discussions about literate programming and markdown
cell templates.

> - ControlCell
>     - motivation
>         - sometimes, i just want a button or a slider for changing a value.
>     - implementation
>         - CodeCell. metadata.

This one is definitely at the core of our grant and will be one of the
first things we do once we have the base architecture done.

Cheers,

Brian

> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From nick.bollweg at gmail.com  Sat Dec 15 16:09:27 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 15 Dec 2012 16:09:27 -0500
Subject: [IPython-dev] Template cells
In-Reply-To: <50CCE4CB.1030108@creativetrax.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
	<CAH4pYpQ39ibx6WkXCJGPZ4ACPjTMun1wbSy2BQftUMxpwGi3Zw@mail.gmail.com>
	<50CCE4CB.1030108@creativetrax.com>
Message-ID: <CACejjWx5oHPzGpD3AhB3xEjDkmCq+PX6U6nR=m7=AO3Cx5Kx5A@mail.gmail.com>

> I'm not sure if you meant we should form a valid execute request, but I
> think we should definitely *not* construct a string which is an execute
> request (i.e., we should not be constructing python code in the
> browser).

To clarify: is it a Bad Thing to be editing the contents of a CodeCell with
an interactive widget? Kinda planning on doing this with the Blockly
extension i am working on.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/f33e2beb/attachment.html>

From bussonniermatthias at gmail.com  Sat Dec 15 16:19:35 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 22:19:35 +0100
Subject: [IPython-dev] Template cells
In-Reply-To: <CACejjWx5oHPzGpD3AhB3xEjDkmCq+PX6U6nR=m7=AO3Cx5Kx5A@mail.gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
	<CAH4pYpQ39ibx6WkXCJGPZ4ACPjTMun1wbSy2BQftUMxpwGi3Zw@mail.gmail.com>
	<50CCE4CB.1030108@creativetrax.com>
	<CACejjWx5oHPzGpD3AhB3xEjDkmCq+PX6U6nR=m7=AO3Cx5Kx5A@mail.gmail.com>
Message-ID: <DF47D33C-472F-4B7E-9DE1-F2D9B0C20921@gmail.com>


Le 15 d?c. 2012 ? 22:09, Nicholas Bollweg a ?crit :

> 
> I'm not sure if you meant we should form a valid execute request, but I
> think we should definitely *not* construct a string which is an execute
> request (i.e., we should not be constructing python code in the
> browser).  
> To clarify: is it a Bad Thing to be editing the contents of a CodeCell with an interactive widget? Kinda planning on doing this with the Blockly extension i am working on.

I think there is a fundamental difference which is that here you want to write a widget for a language that is **by definition** a graphic language. 
So generating code from that is unavoidable. 

The question is could you just have a handler on kernel side that receives data and do what needs to be done with those data.

I'm not sure of what your end-goal is, but keep in mind that you can bind custom kernel to IPython notebook.
So if a blocky kernel make any sense... it might be more what you are looking for.
-- 
Matthias



> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From jason-sage at creativetrax.com  Sat Dec 15 16:20:34 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 14:20:34 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
Message-ID: <50CCE9A2.8000905@creativetrax.com>

On 12/15/12 2:00 PM, Brian Granger wrote:

> I think there is a big design pattern that is emerging in the
> notebook.  Currently CodeCells have an input area and output area.
> Obviously, these were originally created for code and the output of
> the code.  But, with the Javascript plugins, the output area has
> really become more than merely output.  In fact here is how I view it
> now...
>
> * The input area is used for python code that initializes a "widget".
> That Python code is used to create the widget, configure it, provide
> input to it, etc.
> * The output area is used for the widget UI.  Widgets can be arbitrary
> JS/HTML/CSS.  Widgets can make calls back to the kernel and receive
> further JSON data back.  Widgets can even be nested.

That's a nice way to think about it.  Can we extend it one level 
further?  One thing I've long dreamed for Sage is to have javascript 
widgets in the *input* area as well, and a way to flip back and forth 
between a text representation and other representations for the widgets. 
  So, for example, you could have:

a=<<numpy array>>

and the <<numpy array>> would be a small javascript spreadsheet control. 
  Hit a button or key, and the spreadsheet control converts to an 
equivalent text representation (which is what is actually sent over the 
wire to the kernel).  Or I'd like to have:

n=<<slider>>

where <<slider>> is a jquery slider, letting me pick the value for n.

Or have:

f=<<math expression>>

where <<math expression>> is a mathjax-enabled input widget that lets me 
type in normal mathematics.  Hit a button and it gets converted to the 
text equation representation.

Or:

Process(<<image>>)

and <<image>> is an actual picture (which can be flipped to a text 
representation, which might be ascii encoded, or might be a statement 
downloading an image, or something else).




>
> Here is an example that I have started to work on.  The new Web Audio
> API allows Javascript to access the raw audio data from a microphone.
> Using this, it is possible to create a widget that records audio and
> processes and visualizes the audio in JS+Python+matplotlib+d3+etc.

Wouldn't it be cool to have, in the *input*:

audio=<<widget>>
p=Process(audio)
p.save('newaudio.wav')



where widget has a record, playback, etc. buttons.  Hit the record 
button, and it records a data file and ships it to the server and sets 
the result to a.  Hit another button, and you could actually see a text 
representation of what `audio` is going to be set to, or even better, a 
waveform representing what `audio` will be set to.

These ideas also encapsulate a lot of the other cell types mentioned. 
Why have different types of cells, when you can just have different 
widgets inside the same cell?  What if I want to have a spreadsheet 
widget representing a numpy array and an audio widget representing an 
audio file in the same input cell?

Thanks,

Jason



From jason-sage at creativetrax.com  Sat Dec 15 16:25:19 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 14:25:19 -0700
Subject: [IPython-dev] Template cells
In-Reply-To: <DF47D33C-472F-4B7E-9DE1-F2D9B0C20921@gmail.com>
References: <CAOvn4qgYmO=Sz+s6SaT-cKLhTYy2Lszgg4ioT+YE0q0bUF8kSQ@mail.gmail.com>
	<3A312A47-9C45-47B1-861E-D8754795F56F@gmail.com>
	<CAH4pYpSsD0dRLAXkmM=aYVZESHyaYTb_RXtQsvtL7jGsVsXHig@mail.gmail.com>
	<CAP-uhDeS6BAOz5GZ4YTztwdTaQ4Vdq7bq1tz+9LQqeq4yv-OKQ@mail.gmail.com>
	<CAH4pYpQ39ibx6WkXCJGPZ4ACPjTMun1wbSy2BQftUMxpwGi3Zw@mail.gmail.com>
	<50CCE4CB.1030108@creativetrax.com>
	<CACejjWx5oHPzGpD3AhB3xEjDkmCq+PX6U6nR=m7=AO3Cx5Kx5A@mail.gmail.com>
	<DF47D33C-472F-4B7E-9DE1-F2D9B0C20921@gmail.com>
Message-ID: <50CCEABF.4010308@creativetrax.com>

On 12/15/12 2:19 PM, Matthias BUSSONNIER wrote:
>
> Le 15 d?c. 2012 ? 22:09, Nicholas Bollweg a ?crit :
>
>>
>> I'm not sure if you meant we should form a valid execute request, but I
>> think we should definitely *not* construct a string which is an execute
>> request (i.e., we should not be constructing python code in the
>> browser).
>> To clarify: is it a Bad Thing to be editing the contents of a CodeCell with an interactive widget? Kinda planning on doing this with the Blockly extension i am working on.
>
> I think there is a fundamental difference which is that here you want to write a widget for a language that is **by definition** a graphic language.
> So generating code from that is unavoidable.

Yes, I think there's a difference between having a graphical 
representation for code (see my other email on the custom web 
cells---I'd *love* to see graphical widgets that represent code) and 
manipulating and escaping a text string to be valid python code.


Hmmm...or am I contradicting myself?  Would it be better to send a JSON 
representation of the blockly cell to the kernel in a special message 
that means "here is a blockly description; make a Blockly object that 
has a .__call__ attribute and call it" and have the kernel generate the 
python code?

Thanks,

Jason



From jason-sage at creativetrax.com  Sat Dec 15 16:31:41 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 14:31:41 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CAH4pYpQ9bMotBYWJ0b0LXgzMwM=tzvuZta1CZhf_mVsD0jEy8w@mail.gmail.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
	<CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>
	<FFBA3218-A191-47FB-9438-58D9AB21694C@gmail.com>
	<CAH4pYpQ9bMotBYWJ0b0LXgzMwM=tzvuZta1CZhf_mVsD0jEy8w@mail.gmail.com>
Message-ID: <50CCEC3D.6060507@creativetrax.com>

On 12/15/12 1:37 PM, Brian Granger wrote:

> I have started to become very concerned about the metadata.  Our
> thinking so far has been: that metadata is complete unstructured.
> IOW, users can write to the metadata section whatever they want.  When
> nobody is doing that it works fine, but as more people start to use
> the metadata, it is going to turn into the wild west.  There will be
> conflicting metadata.  There will backwards incompatibilties.  We will
> allow this for a while, but when users start to complain that there
> notebooks don't work as expected, we will have to become extremely
> conservative about the metadata.
>
> Case in point: we are about to add a slideshow section to the cell
> level metadata.  And the notebook UI will rely on this metadata being
> in a very particular format.  If we *ever* change that metadata in the
> future, we will have to increment the notebook document format version
> to reflect it.  Some changes (adding a new attribute) won't be
> backwards incompatible, but some might be and it will affect the
> entire notebook document format.

Is the solution (to quote python zen): "Namespaces are one honking great 
idea -- let's do more of those!"?

Suppose the metadata dictionary is actually a key-value store where the 
key is the namespace for the metadata.  You reserve the "ipython" key, 
so that it is safe to make rigid rules and assumptions about anything in 
metadata['ipython'].  Allow others to make arbitrary other metadata 
namespaces and values.

Thanks,

Jason




From bussonniermatthias at gmail.com  Sat Dec 15 16:35:08 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 22:35:08 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CAH4pYpQ9bMotBYWJ0b0LXgzMwM=tzvuZta1CZhf_mVsD0jEy8w@mail.gmail.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
	<CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>
	<FFBA3218-A191-47FB-9438-58D9AB21694C@gmail.com>
	<CAH4pYpQ9bMotBYWJ0b0LXgzMwM=tzvuZta1CZhf_mVsD0jEy8w@mail.gmail.com>
Message-ID: <E3B12D27-8A07-4B5D-A7EF-1F969AD5A5EC@gmail.com>

Brian, 

> I have started to become very concerned about the metadata.  Our
> thinking so far has been: that metadata is complete unstructured.
> IOW, users can write to the metadata section whatever they want.  When
> nobody is doing that it works fine, but as more people start to use
> the metadata, it is going to turn into the wild west.  There will be
> conflicting metadata.  There will backwards incompatibilties.  We will
> allow this for a while, but when users start to complain that there
> notebooks don't work as expected, we will have to become extremely
> conservative about the metadata.
> 
> Case in point: we are about to add a slideshow section to the cell
> level metadata.  And the notebook UI will rely on this metadata being
> in a very particular format.  If we *ever* change that metadata in the
> future, we will have to increment the notebook document format version
> to reflect it.  Some changes (adding a new attribute) won't be
> backwards incompatible, but some might be and it will affect the
> entire notebook document format.
> 
>> [...]
> 
> I am open to using metadata to control how cells act.  We are already
> doing this with the slide show and this may be a very powerful way of
> moving forward.


First, 
I think that everything that uses metadata should make the assumption
that the metadata field may not exist. 

This would highly help because you can then "clear" the metadata in case of problem. 

Also, I do think that the slideshow mode should live as an extension. 
Is it convenient to think that because we are in the core-dev we can put our
piece of code in the core.

This has always been my goal at the beginning, and I strongly believe it should be.

But having it as a separate extension will force us to comply to the rules we impose on 
other to build extension.

Also, I still believe that having smaller separate project could allow to have smaller team
maybe of person not in the core to work on that.

If a specific feature that use metadata seem to become really important, then we should consider
moving it to the notebook format.
-- 
Matthias









From bussonniermatthias at gmail.com  Sat Dec 15 16:42:56 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 22:42:56 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50CCEC3D.6060507@creativetrax.com>
References: <CACejjWzb=8ZAoro99f2bDjo03kuoJzDhsP_EPNgeKnTTdx4L-Q@mail.gmail.com>
	<CAH4pYpTJmHZxAXe+9r=764vKcFCUhMQctCVNS3dJhd8WEHXQ9A@mail.gmail.com>
	<FFBA3218-A191-47FB-9438-58D9AB21694C@gmail.com>
	<CAH4pYpQ9bMotBYWJ0b0LXgzMwM=tzvuZta1CZhf_mVsD0jEy8w@mail.gmail.com>
	<50CCEC3D.6060507@creativetrax.com>
Message-ID: <8A5771BD-77A3-463E-90D7-F18BE5AE6FEE@gmail.com>


Le 15 d?c. 2012 ? 22:31, Jason Grout a ?crit :

> On 12/15/12 1:37 PM, Brian Granger wrote:
> 
>> I have started to become very concerned about the metadata.  Our
>> thinking so far has been: that metadata is complete unstructured.
>> IOW, users can write to the metadata section whatever they want.  When
>> nobody is doing that it works fine, but as more people start to use
>> the metadata, it is going to turn into the wild west.  There will be
>> conflicting metadata.  There will backwards incompatibilties.  We will
>> allow this for a while, but when users start to complain that there
>> notebooks don't work as expected, we will have to become extremely
>> conservative about the metadata.
>> 
>> Case in point: we are about to add a slideshow section to the cell
>> level metadata.  And the notebook UI will rely on this metadata being
>> in a very particular format.  If we *ever* change that metadata in the
>> future, we will have to increment the notebook document format version
>> to reflect it.  Some changes (adding a new attribute) won't be
>> backwards incompatible, but some might be and it will affect the
>> entire notebook document format.
> 
> Is the solution (to quote python zen): "Namespaces are one honking great 
> idea -- let's do more of those!"?
> 
> Suppose the metadata dictionary is actually a key-value store where the 
> key is the namespace for the metadata.  You reserve the "ipython" key, 
> so that it is safe to make rigid rules and assumptions about anything in 
> metadata['ipython'].  Allow others to make arbitrary other metadata 
> namespaces and values.

I suppose we could also do that, but the difference is, we don't yet expose official way for plugins to manipulate 
what is not in metadata, whereas this IPython namespace would be accessible, so will became a de-facto standard 
link to the notebook format version number, and people will try to mess with it.

Also we are currently writing the doc and we already strongly encourage people to use namespace, 
still we do not enforce it as we are actually building 2 project that would be using the same metadata structure. 
-- 
Matthias


> Thanks,
> 
> Jason
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From bussonniermatthias at gmail.com  Sat Dec 15 17:02:26 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 23:02:26 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50CCE9A2.8000905@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
Message-ID: <51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>

>  Or I'd like to have:
> 
> n=<<slider>>

Do you know 
http://livecoding.io/3419309
Javascript tab, click on a number and press Alt
You can do the same with #xxxxxx.

The editor is CodeMirror, the one we use in IPython notebook....

-- 
Matthias


From bussonniermatthias at gmail.com  Sat Dec 15 17:05:22 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 15 Dec 2012 23:05:22 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
Message-ID: <F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>


Le 15 d?c. 2012 ? 23:02, Matthias BUSSONNIER a ?crit :

>> Or I'd like to have:
>> 
>> n=<<slider>>
> 
> Do you know 
> http://livecoding.io/3419309
> Javascript tab, click on a number and press Alt
> You can do the same with #xxxxxx.
> 
> The editor is CodeMirror, the one we use in IPython notebook....

And by the way for those who have never seen 

Inventing on Principle from Brett Victor. 
http://vimeo.com/36579366
-- 
M

From jason-sage at creativetrax.com  Sat Dec 15 17:20:53 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 15:20:53 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
Message-ID: <50CCF7C5.6090400@creativetrax.com>

On 12/15/12 3:02 PM, Matthias BUSSONNIER wrote:
>>   Or I'd like to have:
>>
>> n=<<slider>>
>
> Do you know
> http://livecoding.io/3419309
> Javascript tab, click on a number and press Alt
> You can do the same with #xxxxxx.

Yes, those are exactly the sorts of things that I've been playing with 
over the last year.  And I second the reference to Brett Victor's talk 
(actually, he gave some other very enlightening talks and blog posts 
that further develop some of those ideas, which I also recommend highly).

Here are some other implementations of Brett's ideas that I posted about 
in May: 
http://lists.ipython.scipy.org/pipermail/ipython-dev/2012-May/009239.html

The khan academy code editor (written by John Resig himself) is also 
another implementation of these ideas.  Light Table is another 
implementation of some of these ideas: http://www.lighttable.com/

Anyways, the field is wide open for exploring these things.  It's an 
exciting time to live, isn't it?

Jason



From jason-sage at creativetrax.com  Sat Dec 15 17:22:01 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 15:22:01 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50CCE9A2.8000905@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
Message-ID: <50CCF809.4090808@creativetrax.com>

On 12/15/12 2:20 PM, Jason Grout wrote:
>    So, for example, you could have:

Here's another example to contribute to the brainstorm:

def f(n=1, d=2):
     <<<blockly widget>>>

where the blockly widget is a graphical description of the function body.

Thanks,

Jason



From jason-sage at creativetrax.com  Sat Dec 15 17:47:26 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 15 Dec 2012 15:47:26 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
	<F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>
Message-ID: <50CCFDFE.2040406@creativetrax.com>

On 12/15/12 3:05 PM, Matthias BUSSONNIER wrote:
>
> Le 15 d?c. 2012 ? 23:02, Matthias BUSSONNIER a ?crit :
>
>>> Or I'd like to have:
>>>
>>> n=<<slider>>
>>
>> Do you know
>> http://livecoding.io/3419309
>> Javascript tab, click on a number and press Alt
>> You can do the same with #xxxxxx.
>>
>> The editor is CodeMirror, the one we use in IPython notebook....
>
> And by the way for those who have never seen
>
> Inventing on Principle from Brett Victor.
> http://vimeo.com/36579366


I'll just emphasize one more Brett Victor piece I thought was great:

http://worrydream.com/#!/LearnableProgramming

Thanks,

Jason




From arnaud at oscopy.org  Sat Dec 15 18:31:01 2012
From: arnaud at oscopy.org (Arnaud Gardelein)
Date: Sun, 16 Dec 2012 00:31:01 +0100
Subject: [IPython-dev] [ANNOUNCE] oscopy v0.71-rc1: electrical simulation
	results viewer
Message-ID: <1355614261.6904.45.camel@bomberx.somewhere-in-the-space.org>

I'm very pleased to announce the first release candidate for Oscopy
v0.71. For those just tuning in, Oscopy is a plotting program based on
IPython for viewing and post-processing results of electrical
simulations (more details on oscopy website [1]). Documentation can be
found here [2].

Main user-visible change of this version:
* Support for new input data format: spice2, spice3, touchstone, ...
* New zooming functions: mouse-wheel support, x10 mode, span
* Figure windows reworked: new operation bar, scrollbars for panning
* Command line options restored: batch mode, interactive, quiet
* Duplicate Signals are renamed on file reading
* Update of User Manual
* Now installation is needed to run oscopy.

Main developer-visible changes:
* Separation between back-end (oscopy) and front-end (ioscopy)

And of course a lot of bug fixes.

Significant improvements were performed on the look and feel of the
graphical interface and input file formats. Before releasing the v0.71 I
eagerly need a feedback on those changes to make sure I'm going in the
right direction, this is why I announce a RC here.

I would really appreciate if you could give a try to this version: check
out this RC, verify it installs correctly, and test it. I kindly ask you
to report any bugs on oscopy-dev mailing list
(mailto:oscopy-dev at oscopy.org) in CC of this message.

The source code is hosted at repo.or.cz: http://repo.or.cz/w/oscopy.git

My apologies for the multiples posts.

Arnaud.

[1] http://oscopy.org
[2] http://oscopy.org/wiki/_media/oscopy/ioscopy-manual-071.pdf



From nick.bollweg at gmail.com  Sat Dec 15 21:52:47 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sat, 15 Dec 2012 21:52:47 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50CCFDFE.2040406@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
	<F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>
	<50CCFDFE.2040406@creativetrax.com>
Message-ID: <CACejjWyP9F5T2ze00OpTZp-zsVAbsxFAOMAO321W0MfcBSrsFQ@mail.gmail.com>

Good discussions today, folks. Impressive breadth and depth of focus.

+1 on watching Brett Victor's stuff: he makes really nice things (like
tangle!) and makes you feel good about being in this technology game that
sometimes just makes me sad.

I did get to a 0.0.1 release of the ipython-blockly stuff here:

https://github.com/bollwyvl/ipython/tree/blockly

The setup is a little fiddly, but I got one cell on the page, generating
code which could then be run. Whew! It uses the `%load_ext`, and a new
metadata namespace (blockly). Right now, it just uses a sub-key `xml`: I
don't know what else could show up there, eventually.

By the by: is there a JSON schema for the notebook versions? I did some
work on that for cdnjs: i found them annoying to build, but useful from a
testing perspective (we were looking at `package.json`s for all the stuff
in the cdn). Perhaps we can use them as part of the "contract" for writing
to notebook on the JS side? or the whatever side, i guess: there are
implementations in most environments.

I look forward to spending some more time on it soon, but am going to be
Not Coding tomorrow :)

Cheers!
nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121215/4eddab11/attachment.html>

From bussonniermatthias at gmail.com  Sun Dec 16 03:55:08 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sun, 16 Dec 2012 09:55:08 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWyP9F5T2ze00OpTZp-zsVAbsxFAOMAO321W0MfcBSrsFQ@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
	<F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>
	<50CCFDFE.2040406@creativetrax.com>
	<CACejjWyP9F5T2ze00OpTZp-zsVAbsxFAOMAO321W0MfcBSrsFQ@mail.gmail.com>
Message-ID: <CANJQusXaKcoEdi1meyBGFdAFg_YhVZnn+fqEi85cZo_PtwCEWA@mail.gmail.com>

Short frim my phone.

Yes there is a schema draft 3. Looks to "json schema validator" pr.
Le 16 d?c. 2012 03:52, "Nicholas Bollweg" <nick.bollweg at gmail.com> a ?crit :

> Good discussions today, folks. Impressive breadth and depth of focus.
>
> +1 on watching Brett Victor's stuff: he makes really nice things (like
> tangle!) and makes you feel good about being in this technology game that
> sometimes just makes me sad.
>
> I did get to a 0.0.1 release of the ipython-blockly stuff here:
>
> https://github.com/bollwyvl/ipython/tree/blockly
>
> The setup is a little fiddly, but I got one cell on the page, generating
> code which could then be run. Whew! It uses the `%load_ext`, and a new
> metadata namespace (blockly). Right now, it just uses a sub-key `xml`: I
> don't know what else could show up there, eventually.
>
> By the by: is there a JSON schema for the notebook versions? I did some
> work on that for cdnjs: i found them annoying to build, but useful from a
> testing perspective (we were looking at `package.json`s for all the stuff
> in the cdn). Perhaps we can use them as part of the "contract" for writing
> to notebook on the JS side? or the whatever side, i guess: there are
> implementations in most environments.
>
> I look forward to spending some more time on it soon, but am going to be
> Not Coding tomorrow :)
>
> Cheers!
> nick
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121216/bc82d318/attachment.html>

From nick.bollweg at gmail.com  Sun Dec 16 15:51:22 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sun, 16 Dec 2012 15:51:22 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CANJQusXaKcoEdi1meyBGFdAFg_YhVZnn+fqEi85cZo_PtwCEWA@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
	<F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>
	<50CCFDFE.2040406@creativetrax.com>
	<CACejjWyP9F5T2ze00OpTZp-zsVAbsxFAOMAO321W0MfcBSrsFQ@mail.gmail.com>
	<CANJQusXaKcoEdi1meyBGFdAFg_YhVZnn+fqEi85cZo_PtwCEWA@mail.gmail.com>
Message-ID: <CACejjWxRy4z+gMWc3JSDx-EUdx6HWux5p94Xf14nrSSrH5BRDw@mail.gmail.com>

I'll check out the json schema: will it be used on backend and frontend(s)?

Also pushed new rev of ipython-blockly: now supports multiple blockly cells
in the same notebook. had to use iframes to do it after i got a wontfix
from the upstream, but it's cool... though none of the keyboard shortcuts
work when the focus is in the iframe, and there are probably other
surprises.

also made a little movie of my fumbling with the UI (sorry for the swf):
http://megaswf.com/s/2532010

cheers
nick


On Sun, Dec 16, 2012 at 3:55 AM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> Short frim my phone.
>
> Yes there is a schema draft 3. Looks to "json schema validator" pr.
> Le 16 d?c. 2012 03:52, "Nicholas Bollweg" <nick.bollweg at gmail.com> a
> ?crit :
>
>> Good discussions today, folks. Impressive breadth and depth of focus.
>>
>> +1 on watching Brett Victor's stuff: he makes really nice things (like
>> tangle!) and makes you feel good about being in this technology game that
>> sometimes just makes me sad.
>>
>> I did get to a 0.0.1 release of the ipython-blockly stuff here:
>>
>> https://github.com/bollwyvl/ipython/tree/blockly
>>
>> The setup is a little fiddly, but I got one cell on the page, generating
>> code which could then be run. Whew! It uses the `%load_ext`, and a new
>> metadata namespace (blockly). Right now, it just uses a sub-key `xml`: I
>> don't know what else could show up there, eventually.
>>
>> By the by: is there a JSON schema for the notebook versions? I did some
>> work on that for cdnjs: i found them annoying to build, but useful from a
>> testing perspective (we were looking at `package.json`s for all the stuff
>> in the cdn). Perhaps we can use them as part of the "contract" for writing
>> to notebook on the JS side? or the whatever side, i guess: there are
>> implementations in most environments.
>>
>> I look forward to spending some more time on it soon, but am going to be
>> Not Coding tomorrow :)
>>
>> Cheers!
>> nick
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121216/467de3c0/attachment.html>

From bussonniermatthias at gmail.com  Sun Dec 16 16:51:28 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 16 Dec 2012 22:51:28 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWxRy4z+gMWc3JSDx-EUdx6HWux5p94Xf14nrSSrH5BRDw@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<51ED60F9-9B5C-4303-891C-991E65DAF89D@gmail.com>
	<F098D171-189E-460B-9F3B-380E156EDACD@gmail.com>
	<50CCFDFE.2040406@creativetrax.com>
	<CACejjWyP9F5T2ze00OpTZp-zsVAbsxFAOMAO321W0MfcBSrsFQ@mail.gmail.com>
	<CANJQusXaKcoEdi1meyBGFdAFg_YhVZnn+fqEi85cZo_PtwCEWA@mail.gmail.com>
	<CACejjWxRy4z+gMWc3JSDx-EUdx6HWux5p94Xf14nrSSrH5BRDw@mail.gmail.com>
Message-ID: <47CB7727-E515-4CE0-958F-BE49675110BE@gmail.com>


Le 16 d?c. 2012 ? 21:51, Nicholas Bollweg a ?crit :

> I'll check out the json schema: will it be used on backend and frontend(s)?

Probably on the backend first to raise only warning as even example notebook don't conform to the schema in some places.
Also to prevent some regression or test other tools that generate ipynbs.
But the schema only validate the structure, not the content.
Tools to check the content still have to be written.

> Also pushed new rev of ipython-blockly: now supports multiple blockly cells in the same notebook. had to use iframes to do it after i got a wontfix from the upstream, but it's cool... though none of the keyboard shortcuts work when the focus is in the iframe, and there are probably other surprises.
> 
> also made a little movie of my fumbling with the UI (sorry for the swf):
> http://megaswf.com/s/2532010

Looks quite cool !

> cheers
> nick
-- 
Matthias

From bussonniermatthias at gmail.com  Mon Dec 17 12:48:49 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 17 Dec 2012 18:48:49 +0100
Subject: [IPython-dev] Should I still contribute to IPython ?
Message-ID: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>

Dear User and Dev, 

This morning I received an personal email from one of our user with the
following inside:

> I was wondering whether my involvement in the development [?] make too much
> sense now, that IPython has received a grant for the next two years, [?]  it
> seems that there are going to be a number of people on the team, who are very
> proficient, [?] and I wouldn't be able to contribute too much, anyway.

I realized, that even we got a lot a congratulation mails after the announce of the grant,
we didn't get any expression of concern or questions on what will happened, this was 
a little less the case on hacker-news.

What's follow is my personal opinion, how I feel about the development of
IPython over the past Year.  Because I feel it is important to tell that
being founded will not change how we work, will not change the people.

So as you may know, IPython did receive a Sloan Foundation Grant for the next
two years, that will help the development of IPython, But this does not mean we
will ever cut ourselves from our user bases. 

First of all, being in the core development team does not necessarily mean we
will get any money from the grant.  Some of the current core team dev will be
"hired" to work part or full time on IPython. Personally I won't, even if I were
paid for that it wouldn't change what I would be doing (just maybe my schedule).

This will not prevent me from fixing bug and bring improvement to IPython.  Nor
will I assume that Devs that are paid from this grant have more obligation to
fix bugs or answer to question than I am.

Being a core dev does not mean that we are more capable than any other
user of the list. There is no official way to become a core dev, but
empirically, I would say that when you submit (or review) too many good pull 
request we give you the responsibility  to merge others'.

One of the things that make IPython so great is that we develop things that we
use, I do make patches for IPython because I feel the use for it in my work,
the diversity of people that contribute and use it make IPython what it is now :
If we look a the opened PR, we see that more than the half are opened by people
that are not in the core team, and *big* PR are in this big half.

You might be concern to not being able to keep up with something some
could do in minutes, and I guess this is the main concern most of you have. I
reassure you we will alway take time to guide you to make a PR. 

I first contributed 1 Year ago, from an ~10 lines change that took me a week to
write, I had kind words from fperez, epatters, and minrk to explain me how to
do things they could have done in 10 minutes tops.

Now I do the same for every pull request from users. Even if fixing a bug would
take me less time than writing the explanation for an user, I still write the
explanation and help do the right thing even for one line. I do strongly
believe that this is worth doing in the long run. And I know that I am not the
only one that will keep up doing so as long as he can. Often the most difficult 
thing is to know the code base, not actually writing the patch, and nothing but 
time helps that.

Heck, the Sloan-Foundation almost exclusively give to project linked to
education, if we can't be good teacher, how could build a tool to help other to
teach ?

So yes I will continue to contribute, on my free time, I hope I'll find some
way to get more time to be involve. Yes the development will probably go
faster, and there are things I won't be able to follow, but even now, I already
discover that some stuff had been added to IPython during the week that I
wasn't aware of, and I'm convinced there are still room for all kind of
contribution, from QTconsole UI to big AST hook to do amazing stuff I don't
even dream of.

Hopping to have convince you that we will not change, and that you will continue to 
bring us big surprises with awesome idea and PRs.

Thanks to all of you, also and especially to Fernando without whom IPython 
would not exist today. 

Sorry for the length, 
-- 
Matthias



[Grant announce on HN] http://news.ycombinator.com/item?id=4909070
And a small link, also from hacker news to conclude :
[Dear Open Source Project Leader: Quit Being A Jerk] http://news.ycombinator.com/item?id=4921152 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121217/eb529980/attachment.html>

From gael.varoquaux at normalesup.org  Mon Dec 17 13:22:38 2012
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 17 Dec 2012 19:22:38 +0100
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
Message-ID: <20121217182238.GA4996@phare.normalesup.org>

It's a natural question that comes to mind: 

    Some people have more time, paid time, than me to contribute to a
    project. They naturally know the code base better and are more efficient
    when working on the project. Why should I contribute?

The reason that I will keep contributing to projects even if I am not
core dev and the core devs are doing an excellent job is because the sum
of those 'minor' contributions are what makes an open source project
fantastic. The unique thing about open source is that it is powered by
users for their own needs. One works on the minor feature that makes his
life easier. Those pile up, and the project ends up with a different
feeling than a project developed by professionals.

This is why over and over open source software answers problems better
than software built by software giants. By giving up your option to
contribute, you would be giving up what makes open source software rock.

My 2 cents,

Ga?l

PS: this is also why, as a researcher drowning under all kind of work, I
fight to keep some time to do software.


From ellisonbg at gmail.com  Mon Dec 17 15:00:31 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 17 Dec 2012 12:00:31 -0800
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
Message-ID: <CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>

Matthias,

Thanks so much for bringing this up to the list...you bring up issues
that I have been thinking a lot about recently.  I am sure Fernando
has thoughts too, but he is enjoying some snow up at Mammoth right
now.  I should mention that I will also be visiting Fernando later
this week at Berkeley.  I had planned on talking with him about these
issues so it is a good time to start the discussion...

> This morning I received an personal email from one of our user with the
> following inside:
>
>> I was wondering whether my involvement in the development [?] make too
>> much
>> sense now, that IPython has received a grant for the next two years, [?]
>> it
>> seems that there are going to be a number of people on the team, who are
>> very
>> proficient, [?] and I wouldn't be able to contribute too much, anyway.

Here is my short answer: absolutely, we *really* want and need the
contributions of the larger community.

And my long answer:

A few years back, IPython development was moving along very slowly.
There were only a few core developers and we didn't get many
contributions from the community.  Then we moved the github and
developed the qtconsole, notebook and zmq based parallel capabilities
- development has *exploded*.  Our core team of developers has grown
dramatically and the number of contributions from the larger community
has grown by leaps and bounds.  What stands out to me about this
growth is how it happens.  As Matthias describes so well, it starts by
new people contributing a single small fix, feature, etc.  They they
make another contribution, and then another and another until they are
making massive contributions to the project and helping to steer its
direction.  Why do people do this?  I am convinced that it is *deeply
rewarding and enjoyable* for them.  Why else would they put in so much
effort on something that pulls them away from their day job, friends
and family.  So what criteria should someone use to determine if they
will contribute to the project?

Do you find contributing enjoyable and rewarding?  If you do, then you
should contribute no matter how many of us are paid to work on the
project.

But I think there is another question lurking here: will we continue
to welcome or value the contributions of the larger community, or will
people be excluded if they are not part of the Sloan funded team.
This is something that Fernando and I have talked about extensively
and the answer is that the community is and will always be the
foundation of the project and we will do everything we can to enable
people to participate in the project.  This is extremely important and
something that the Sloan foundation is paying very close attention to.
 In fact, I would say that the Sloan foundation would never have even
talked to us if it were not for the life, excitement and health of our
community.

> I realized, that even we got a lot a congratulation mails after the announce
> of the grant,
> we didn't get any expression of concern or questions on what will happened,
> this was
> a little less the case on hacker-news.
>
> What's follow is my personal opinion, how I feel about the development of
> IPython over the past Year.  Because I feel it is important to tell that
> being founded will not change how we work, will not change the people.
>
> So as you may know, IPython did receive a Sloan Foundation Grant for the
> next
> two years, that will help the development of IPython, But this does not mean
> we
> will ever cut ourselves from our user bases.

Absolutely!

> First of all, being in the core development team does not necessarily mean
> we
> will get any money from the grant.  Some of the current core team dev will
> be
> "hired" to work part or full time on IPython. Personally I won't, even if I
> were
> paid for that it wouldn't change what I would be doing (just maybe my
> schedule).

Yes, this is a very important point: not all of the core devs will be
funded by the Sloan foundation.  That means that our Sloan funded work
*has to* be integrated with the larger project and community.

> This will not prevent me from fixing bug and bring improvement to IPython.
> Nor
> will I assume that Devs that are paid from this grant have more obligation
> to
> fix bugs or answer to question than I am.

Absolutely, the devotion of volunteer developers is in no way inferior
to that of people who are paid to work on the project.

> Being a core dev does not mean that we are more capable than any other
> user of the list. There is no official way to become a core dev, but
> empirically, I would say that when you submit (or review) too many good pull
> request we give you the responsibility  to merge others'.
>
> One of the things that make IPython so great is that we develop things that
> we
> use, I do make patches for IPython because I feel the use for it in my work,
> the diversity of people that contribute and use it make IPython what it is
> now :
> If we look a the opened PR, we see that more than the half are opened by
> people
> that are not in the core team, and *big* PR are in this big half.

Yep, at this point, we receive a significant number of PRs from
non-core developers.

> You might be concern to not being able to keep up with something some
> could do in minutes, and I guess this is the main concern most of you have.
> I
> reassure you we will alway take time to guide you to make a PR.
>
> I first contributed 1 Year ago, from an ~10 lines change that took me a week
> to
> write, I had kind words from fperez, epatters, and minrk to explain me how
> to
> do things they could have done in 10 minutes tops.
>
> Now I do the same for every pull request from users. Even if fixing a bug
> would
> take me less time than writing the explanation for an user, I still write
> the
> explanation and help do the right thing even for one line. I do strongly
> believe that this is worth doing in the long run. And I know that I am not
> the
> only one that will keep up doing so as long as he can. Often the most
> difficult
> thing is to know the code base, not actually writing the patch, and nothing
> but
> time helps that.
>
> Heck, the Sloan-Foundation almost exclusively give to project linked to
> education, if we can't be good teacher, how could build a tool to help other
> to
> teach ?

Absolutely...

> So yes I will continue to contribute, on my free time, I hope I'll find some
> way to get more time to be involve. Yes the development will probably go
> faster, and there are things I won't be able to follow, but even now, I
> already
> discover that some stuff had been added to IPython during the week that I
> wasn't aware of, and I'm convinced there are still room for all kind of
> contribution, from QTconsole UI to big AST hook to do amazing stuff I don't
> even dream of.
>
> Hopping to have convince you that we will not change, and that you will
> continue to
> bring us big surprises with awesome idea and PRs.
>
> Thanks to all of you, also and especially to Fernando without whom IPython
> would not exist today.

Absolutely, thank you to *everyone* who participates in the project
and community - and to Fernando for blowing off his thesis..

So I think there are some important issues that the Sloan foundation
grant does bring up, that we will have to work though.  Right now we
are already flooded by the number of pull requests and contributions
that we are getting.  With 5 of us working full time, the level of
activity is going to go through the roof.  This is exciting, but will
definitely change some aspects of the project.  Some of the questions
that we are thinking about right now:

* How do we coordinate all of this activity?  Already, our github
issues are becoming unusable because of the sheer number of them.  We
have to figure out a better way of using github issues, wiki pages,
etc. to coordinate the increased activity the grant will bring.
* Currently, we don't really have any long/medium term planning for
the project.  Our current model works great for attracting the types
of contributions we are getting right now, but it makes it very
difficult to tackle larger projects that require a coordinated and
sustained effort by multiple people.  I don't know what it will look
like, but we are going to need to do this type of planning for the
project.
* How do we manage communication?  Verbal communication is much more
efficient than emails or even IRC.  The 4 people at Berkeley will have
an incredible advantage in being able to talk daily.  We don't want to
cripple or remove that advantage, but we need to figure out how to
include other core devs and people from the community.  This is
particularly relevant to myself as I am the only person involved in
the Sloan work that is not at Berkeley.

Cheers,

Brian


> Sorry for the length,
> --
> Matthias
>
>
>
> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
> And a small link, also from hacker news to conclude :
> [Dear Open Source Project Leader: Quit Being A Jerk]
> http://news.ycombinator.com/item?id=4921152
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From asmeurer at gmail.com  Mon Dec 17 17:36:47 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 17 Dec 2012 15:36:47 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
Message-ID: <CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>

I remember reading about a study that said that open source projects
that have funded developers actually get less contributions (or at
least fewer contributors) because there is a mindset of, "why should I
contribute when there is someone who is paid to do it? Surely that
person/persons will get around to fixing the issue themselves."  I
think the email Matthias received is indicative of this mindset.
Clearly it is wrong (obviously, more contributions are still more
contributions), but I think you should really think about how to make
this permeate through your community, so that you minimize the number
of people who end up thinking this way.

One good thing to do, as Matthias said, is to always encourage people
to write their own patches.  It is an investment to spend a day
helping someone write up a pull request for a fix when you could have
done the whole thing yourself in under half an hour, but by guiding
the new contributor, you potentially gain a new developer.  Ond?ej
?ert?k wrote a blog post a while back about how he managed to make
SymPy such a successful community
http://ondrejcertik.blogspot.com/2009/05/my-experience-with-running-opensource.html.
 The gist of it is what I just said.

For your situation, I think this additionally sends the message that
it is not only OK, but encouraged to send your own patches, rather
than wait for those people who are being paid to fix the bug for you.

On Mon, Dec 17, 2012 at 1:00 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Matthias,
>
> Thanks so much for bringing this up to the list...you bring up issues
> that I have been thinking a lot about recently.  I am sure Fernando
> has thoughts too, but he is enjoying some snow up at Mammoth right
> now.  I should mention that I will also be visiting Fernando later
> this week at Berkeley.  I had planned on talking with him about these
> issues so it is a good time to start the discussion...
>
>> This morning I received an personal email from one of our user with the
>> following inside:
>>
>>> I was wondering whether my involvement in the development [?] make too
>>> much
>>> sense now, that IPython has received a grant for the next two years, [?]
>>> it
>>> seems that there are going to be a number of people on the team, who are
>>> very
>>> proficient, [?] and I wouldn't be able to contribute too much, anyway.
>
> Here is my short answer: absolutely, we *really* want and need the
> contributions of the larger community.
>
> And my long answer:
>
> A few years back, IPython development was moving along very slowly.
> There were only a few core developers and we didn't get many
> contributions from the community.  Then we moved the github and
> developed the qtconsole, notebook and zmq based parallel capabilities
> - development has *exploded*.  Our core team of developers has grown
> dramatically and the number of contributions from the larger community
> has grown by leaps and bounds.  What stands out to me about this
> growth is how it happens.  As Matthias describes so well, it starts by
> new people contributing a single small fix, feature, etc.  They they
> make another contribution, and then another and another until they are
> making massive contributions to the project and helping to steer its
> direction.  Why do people do this?  I am convinced that it is *deeply
> rewarding and enjoyable* for them.  Why else would they put in so much
> effort on something that pulls them away from their day job, friends
> and family.  So what criteria should someone use to determine if they
> will contribute to the project?
>
> Do you find contributing enjoyable and rewarding?  If you do, then you
> should contribute no matter how many of us are paid to work on the
> project.
>
> But I think there is another question lurking here: will we continue
> to welcome or value the contributions of the larger community, or will
> people be excluded if they are not part of the Sloan funded team.
> This is something that Fernando and I have talked about extensively
> and the answer is that the community is and will always be the
> foundation of the project and we will do everything we can to enable
> people to participate in the project.  This is extremely important and
> something that the Sloan foundation is paying very close attention to.
>  In fact, I would say that the Sloan foundation would never have even
> talked to us if it were not for the life, excitement and health of our
> community.
>
>> I realized, that even we got a lot a congratulation mails after the announce
>> of the grant,
>> we didn't get any expression of concern or questions on what will happened,
>> this was
>> a little less the case on hacker-news.
>>
>> What's follow is my personal opinion, how I feel about the development of
>> IPython over the past Year.  Because I feel it is important to tell that
>> being founded will not change how we work, will not change the people.
>>
>> So as you may know, IPython did receive a Sloan Foundation Grant for the
>> next
>> two years, that will help the development of IPython, But this does not mean
>> we
>> will ever cut ourselves from our user bases.
>
> Absolutely!
>
>> First of all, being in the core development team does not necessarily mean
>> we
>> will get any money from the grant.  Some of the current core team dev will
>> be
>> "hired" to work part or full time on IPython. Personally I won't, even if I
>> were
>> paid for that it wouldn't change what I would be doing (just maybe my
>> schedule).
>
> Yes, this is a very important point: not all of the core devs will be
> funded by the Sloan foundation.  That means that our Sloan funded work
> *has to* be integrated with the larger project and community.
>
>> This will not prevent me from fixing bug and bring improvement to IPython.
>> Nor
>> will I assume that Devs that are paid from this grant have more obligation
>> to
>> fix bugs or answer to question than I am.
>
> Absolutely, the devotion of volunteer developers is in no way inferior
> to that of people who are paid to work on the project.
>
>> Being a core dev does not mean that we are more capable than any other
>> user of the list. There is no official way to become a core dev, but
>> empirically, I would say that when you submit (or review) too many good pull
>> request we give you the responsibility  to merge others'.
>>
>> One of the things that make IPython so great is that we develop things that
>> we
>> use, I do make patches for IPython because I feel the use for it in my work,
>> the diversity of people that contribute and use it make IPython what it is
>> now :
>> If we look a the opened PR, we see that more than the half are opened by
>> people
>> that are not in the core team, and *big* PR are in this big half.
>
> Yep, at this point, we receive a significant number of PRs from
> non-core developers.
>
>> You might be concern to not being able to keep up with something some
>> could do in minutes, and I guess this is the main concern most of you have.
>> I
>> reassure you we will alway take time to guide you to make a PR.
>>
>> I first contributed 1 Year ago, from an ~10 lines change that took me a week
>> to
>> write, I had kind words from fperez, epatters, and minrk to explain me how
>> to
>> do things they could have done in 10 minutes tops.
>>
>> Now I do the same for every pull request from users. Even if fixing a bug
>> would
>> take me less time than writing the explanation for an user, I still write
>> the
>> explanation and help do the right thing even for one line. I do strongly
>> believe that this is worth doing in the long run. And I know that I am not
>> the
>> only one that will keep up doing so as long as he can. Often the most
>> difficult
>> thing is to know the code base, not actually writing the patch, and nothing
>> but
>> time helps that.
>>
>> Heck, the Sloan-Foundation almost exclusively give to project linked to
>> education, if we can't be good teacher, how could build a tool to help other
>> to
>> teach ?
>
> Absolutely...
>
>> So yes I will continue to contribute, on my free time, I hope I'll find some
>> way to get more time to be involve. Yes the development will probably go
>> faster, and there are things I won't be able to follow, but even now, I
>> already
>> discover that some stuff had been added to IPython during the week that I
>> wasn't aware of, and I'm convinced there are still room for all kind of
>> contribution, from QTconsole UI to big AST hook to do amazing stuff I don't
>> even dream of.
>>
>> Hopping to have convince you that we will not change, and that you will
>> continue to
>> bring us big surprises with awesome idea and PRs.
>>
>> Thanks to all of you, also and especially to Fernando without whom IPython
>> would not exist today.
>
> Absolutely, thank you to *everyone* who participates in the project
> and community - and to Fernando for blowing off his thesis..
>
> So I think there are some important issues that the Sloan foundation
> grant does bring up, that we will have to work though.  Right now we
> are already flooded by the number of pull requests and contributions
> that we are getting.  With 5 of us working full time, the level of
> activity is going to go through the roof.  This is exciting, but will
> definitely change some aspects of the project.  Some of the questions
> that we are thinking about right now:
>
> * How do we coordinate all of this activity?  Already, our github
> issues are becoming unusable because of the sheer number of them.  We
> have to figure out a better way of using github issues, wiki pages,
> etc. to coordinate the increased activity the grant will bring.

I'm curious how you end up solving this problem.  I've found the
GitHub issues to be minimalistic, which makes them easy to use, but
they also lack in some key features that make managing thousands of
issues bearable.

> * Currently, we don't really have any long/medium term planning for
> the project.  Our current model works great for attracting the types
> of contributions we are getting right now, but it makes it very
> difficult to tackle larger projects that require a coordinated and
> sustained effort by multiple people.  I don't know what it will look
> like, but we are going to need to do this type of planning for the
> project.
> * How do we manage communication?  Verbal communication is much more
> efficient than emails or even IRC.  The 4 people at Berkeley will have
> an incredible advantage in being able to talk daily.  We don't want to
> cripple or remove that advantage, but we need to figure out how to
> include other core devs and people from the community.  This is
> particularly relevant to myself as I am the only person involved in
> the Sloan work that is not at Berkeley.

An idea floated around SymPy at some point to use Google+ hangouts
(basically, multi-way video chats).  We haven't tried it yet, but it
seems like it might work well.

Aaron Meurer

>
> Cheers,
>
> Brian
>
>
>> Sorry for the length,
>> --
>> Matthias
>>
>>
>>
>> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
>> And a small link, also from hacker news to conclude :
>> [Dear Open Source Project Leader: Quit Being A Jerk]
>> http://news.ycombinator.com/item?id=4921152
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From jason-sage at creativetrax.com  Mon Dec 17 17:52:09 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Mon, 17 Dec 2012 15:52:09 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50CCF809.4090808@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
Message-ID: <50CFA219.4020703@creativetrax.com>

On 12/15/12 3:22 PM, Jason Grout wrote:
> On 12/15/12 2:20 PM, Jason Grout wrote:
>>     So, for example, you could have:
>
> Here's another example to contribute to the brainstorm:
>
> def f(n=1, d=2):
>       <<<blockly widget>>>
>
> where the blockly widget is a graphical description of the function body.

I see that CodeMirror 3 is out now, which allows you to mark some text 
and replace it with a DOM node [1].  I think you see this in the code 
folding demo:

http://codemirror.net/demo/folding.html

It seems that our editor now supports this sort of functionality!  I've 
been waiting for this capability in CodeMirror for years; I'm really 
excited about it!

Thanks,

Jason

[1] See the markText function: 
http://codemirror.net/doc/manual.html#markText, and then replaceWith option.


From zvoros at gmail.com  Mon Dec 17 18:04:57 2012
From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=)
Date: Tue, 18 Dec 2012 00:04:57 +0100
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
Message-ID: <50CFA519.5090504@gmail.com>


On 12/17/2012 11:36 PM, Aaron Meurer wrote:
> there is a mindset of, "why should I
> contribute when there is someone who is paid to do it? Surely that
> person/persons will get around to fixing the issue themselves."  I
> think the email Matthias received is indicative of this mindset.
I am not so sure that this conclusion is correct. One of the problems is 
that many projects (and IPython certainly falls into this category) have 
grown to such a complexity, that it becomes exceedingly difficult to 
understand how the code is organised. In the case of IPython, it is 
exacerbated by the fact that three programming paradigms (python, 
javascript, and the Qt toolkit) have to work together in a consistent 
way, and one cannot make a meaningful contribution without understanding 
something about how these are interconnected. On the other hand, many of 
us (namely, those who are not paid for developing IPython) do something 
else for a living, and I believe, in the question that was quoted by 
Matthias, the concern was that "outsiders" won't be able to keep up with 
the pace of the development. I really don't think that the mean mindset 
that you are trying to imply was at the core of that e-mail. It wasn't 
about money, it was about the time that core/paid developers and the 
rest of us can spend on coding.

And that brings us to your first statement:

> I remember reading about a study that said that open source projects
> that have funded developers actually get less contributions (or at
> least fewer contributors)
While this might be true, the question really is, whether the 
development of funded projects slows down (I doubt, for in that case, 
funding agencies and foundations would stop funding after a while, 
unless their purpose is to kill otherwise viable projects), or it speeds 
up, but with fewer contributors. If the second option is true, then I 
would say that the concern in the quoted question is probably not 
entirely baseless... And it would also mean that the money wasn't 
unwisely spent.

Cheers,
Zolt?n


From ellisonbg at gmail.com  Mon Dec 17 19:02:40 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 17 Dec 2012 16:02:40 -0800
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
Message-ID: <CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>

On Mon, Dec 17, 2012 at 2:36 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
> I remember reading about a study that said that open source projects
> that have funded developers actually get less contributions (or at
> least fewer contributors) because there is a mindset of, "why should I
> contribute when there is someone who is paid to do it? Surely that
> person/persons will get around to fixing the issue themselves."  I
> think the email Matthias received is indicative of this mindset.
> Clearly it is wrong (obviously, more contributions are still more
> contributions), but I think you should really think about how to make
> this permeate through your community, so that you minimize the number
> of people who end up thinking this way.
>
> One good thing to do, as Matthias said, is to always encourage people
> to write their own patches.  It is an investment to spend a day
> helping someone write up a pull request for a fix when you could have
> done the whole thing yourself in under half an hour, but by guiding
> the new contributor, you potentially gain a new developer.  Ond?ej
> ?ert?k wrote a blog post a while back about how he managed to make
> SymPy such a successful community
> http://ondrejcertik.blogspot.com/2009/05/my-experience-with-running-opensource.html.
>  The gist of it is what I just said.
>
> For your situation, I think this additionally sends the message that
> it is not only OK, but encouraged to send your own patches, rather
> than wait for those people who are being paid to fix the bug for you.

Yes, I really like how Ondrej has always encouraged people to send
patches and I think we need to encourage people to do that.  Part of
the reality is that a good portion of our Sloan funded time on the
project will be reviewing pull requests - and that is a good thing.

>> * How do we coordinate all of this activity?  Already, our github
>> issues are becoming unusable because of the sheer number of them.  We
>> have to figure out a better way of using github issues, wiki pages,
>> etc. to coordinate the increased activity the grant will bring.
>
> I'm curious how you end up solving this problem.  I've found the
> GitHub issues to be minimalistic, which makes them easy to use, but
> they also lack in some key features that make managing thousands of
> issues bearable.

I know it probably sounds heretical, but I think that part of the
solution has to be to reduce the number of issues.  I don't mean by
closing issues by working harder and coding more, I mean simply
*closing* them outright to reflect the reality that we are not going
to work on them anytime soon.  There is no universe where it is useful
to have thousands of issues.  How many can each developer
realistically keep track of?  A few dozen?  When there are more than
that we simply ignore the issues and work on the things we want to
work on...I was recently talking to the lead developer of a large open
source project that has this problem, and he said "oh it's horrible
and I completely ignore our issues on github"  Personally, when I am
working heavily on the notebook, I maintain my own "issues list" in a
markdown document on my desktop.  We have to figure out a better
way...

Cheers,

Brian

>> * Currently, we don't really have any long/medium term planning for
>> the project.  Our current model works great for attracting the types
>> of contributions we are getting right now, but it makes it very
>> difficult to tackle larger projects that require a coordinated and
>> sustained effort by multiple people.  I don't know what it will look
>> like, but we are going to need to do this type of planning for the
>> project.
>> * How do we manage communication?  Verbal communication is much more
>> efficient than emails or even IRC.  The 4 people at Berkeley will have
>> an incredible advantage in being able to talk daily.  We don't want to
>> cripple or remove that advantage, but we need to figure out how to
>> include other core devs and people from the community.  This is
>> particularly relevant to myself as I am the only person involved in
>> the Sloan work that is not at Berkeley.
>
> An idea floated around SymPy at some point to use Google+ hangouts
> (basically, multi-way video chats).  We haven't tried it yet, but it
> seems like it might work well.
>
> Aaron Meurer
>
>>
>> Cheers,
>>
>> Brian
>>
>>
>>> Sorry for the length,
>>> --
>>> Matthias
>>>
>>>
>>>
>>> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
>>> And a small link, also from hacker news to conclude :
>>> [Dear Open Source Project Leader: Quit Being A Jerk]
>>> http://news.ycombinator.com/item?id=4921152
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From asmeurer at gmail.com  Mon Dec 17 19:03:00 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 17 Dec 2012 17:03:00 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <50CFA519.5090504@gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<50CFA519.5090504@gmail.com>
Message-ID: <CAKgW=6LyAZYWCR+pr-EaTUKDaJv46pDHMy-YbOpApX8UG-K1Vw@mail.gmail.com>

On Mon, Dec 17, 2012 at 4:04 PM, Zolt?n V?r?s <zvoros at gmail.com> wrote:
>
> On 12/17/2012 11:36 PM, Aaron Meurer wrote:
>>
>> there is a mindset of, "why should I
>> contribute when there is someone who is paid to do it? Surely that
>> person/persons will get around to fixing the issue themselves."  I
>> think the email Matthias received is indicative of this mindset.
>
> I am not so sure that this conclusion is correct. One of the problems is
> that many projects (and IPython certainly falls into this category) have
> grown to such a complexity, that it becomes exceedingly difficult to
> understand how the code is organised. In the case of IPython, it is
> exacerbated by the fact that three programming paradigms (python,
> javascript, and the Qt toolkit) have to work together in a consistent way,
> and one cannot make a meaningful contribution without understanding
> something about how these are interconnected. On the other hand, many of us
> (namely, those who are not paid for developing IPython) do something else
> for a living, and I believe, in the question that was quoted by Matthias,
> the concern was that "outsiders" won't be able to keep up with the pace of
> the development. I really don't think that the mean mindset that you are
> trying to imply was at the core of that e-mail. It wasn't about money, it
> was about the time that core/paid developers and the rest of us can spend on
> coding.
>
> And that brings us to your first statement:
>
>
>> I remember reading about a study that said that open source projects
>> that have funded developers actually get less contributions (or at
>> least fewer contributors)
>
> While this might be true, the question really is, whether the development of
> funded projects slows down (I doubt, for in that case, funding agencies and
> foundations would stop funding after a while, unless their purpose is to
> kill otherwise viable projects), or it speeds up, but with fewer
> contributors. If the second option is true, then I would say that the
> concern in the quoted question is probably not entirely baseless... And it
> would also mean that the money wasn't unwisely spent.

I've no doubt that it would speed up the development on net balance
(unless the money really was poorly spent).  But there are more issues
at stake here, such as the heath of the community, which affects
things like the degree to which the community's wishes are reflected
in what the developers spend their time doing.

And if it's not too much work to get both paid development *and* high
levels of volunteer development, then surely the latter would be
preferred.  I believe that maintaining a high level of volunteer
development is just a matter of maintaining the community ideal of
encouraging volunteer contributions, so that people get the feeling
that contributing is the correct answer more often than they get the
feeling that it isn't.

Aaron Meurer

>
> Cheers,
> Zolt?n


From asmeurer at gmail.com  Mon Dec 17 19:18:53 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 17 Dec 2012 17:18:53 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
Message-ID: <CAKgW=6LChOPOQhsz7XDjxJpqhddf0d-KC93DwKQH3_-jkx+YJA@mail.gmail.com>

On Mon, Dec 17, 2012 at 5:02 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> On Mon, Dec 17, 2012 at 2:36 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>> I remember reading about a study that said that open source projects
>> that have funded developers actually get less contributions (or at
>> least fewer contributors) because there is a mindset of, "why should I
>> contribute when there is someone who is paid to do it? Surely that
>> person/persons will get around to fixing the issue themselves."  I
>> think the email Matthias received is indicative of this mindset.
>> Clearly it is wrong (obviously, more contributions are still more
>> contributions), but I think you should really think about how to make
>> this permeate through your community, so that you minimize the number
>> of people who end up thinking this way.
>>
>> One good thing to do, as Matthias said, is to always encourage people
>> to write their own patches.  It is an investment to spend a day
>> helping someone write up a pull request for a fix when you could have
>> done the whole thing yourself in under half an hour, but by guiding
>> the new contributor, you potentially gain a new developer.  Ond?ej
>> ?ert?k wrote a blog post a while back about how he managed to make
>> SymPy such a successful community
>> http://ondrejcertik.blogspot.com/2009/05/my-experience-with-running-opensource.html.
>>  The gist of it is what I just said.
>>
>> For your situation, I think this additionally sends the message that
>> it is not only OK, but encouraged to send your own patches, rather
>> than wait for those people who are being paid to fix the bug for you.
>
> Yes, I really like how Ondrej has always encouraged people to send
> patches and I think we need to encourage people to do that.  Part of
> the reality is that a good portion of our Sloan funded time on the
> project will be reviewing pull requests - and that is a good thing.

It's really amazing actually.  90% of time, all you have to do is say
"can you send a pull request fixing that?" (especially if it's clear
that the person knows how to fix the issue), and they will do it. Even
if you would have guessed that the person would not be able to do it,
because they seem just like a user, and not someone who knows how to
find bugs in the code base or use git, they surprise you more often
than not.  I think just the simple act of asking someone to contribute
makes them feel responsible for doing it, which they did not feel
before.

>
>>> * How do we coordinate all of this activity?  Already, our github
>>> issues are becoming unusable because of the sheer number of them.  We
>>> have to figure out a better way of using github issues, wiki pages,
>>> etc. to coordinate the increased activity the grant will bring.
>>
>> I'm curious how you end up solving this problem.  I've found the
>> GitHub issues to be minimalistic, which makes them easy to use, but
>> they also lack in some key features that make managing thousands of
>> issues bearable.
>
> I know it probably sounds heretical, but I think that part of the
> solution has to be to reduce the number of issues.  I don't mean by
> closing issues by working harder and coding more, I mean simply
> *closing* them outright to reflect the reality that we are not going
> to work on them anytime soon.  There is no universe where it is useful
> to have thousands of issues.  How many can each developer
> realistically keep track of?  A few dozen?  When there are more than
> that we simply ignore the issues and work on the things we want to
> work on...I was recently talking to the lead developer of a large open
> source project that has this problem, and he said "oh it's horrible
> and I completely ignore our issues on github"  Personally, when I am
> working heavily on the notebook, I maintain my own "issues list" in a
> markdown document on my desktop.  We have to figure out a better
> way...

This does sound heretical to me, but I have a personality that
dislikes deleting anything.  I personally think of issues as ways of
organizing discussions about problems/enhancement requests so that
when the issue comes up again, it is easy to see what was already
discussed.

SymPy has over 1000 open issues in our issue tracker (of over 3000
total).  Admittedly, quite a few of the older of these are outdated
and should be closed, but we don't take the time to go through them.
But I personally feel that that many issues is manageable because we
have extensive labeling, and Google Code has great search
functionality (which GitHub lacks), so that I feel that I can almost
always find the issue I want if I look for it.

I'm actually reminded of the one time that I posted a message to the
git mailing list.  I asked them if they had an issue tracker, and they
told me that they don't, but rather they *just* use the mailing list.
To quote Junio Hamano, the lead developer of git:

"What we do is to take advantage of the fact that issues people do care
about are important ones, and others that nobody cares about are not worth
pursuing.

"In a sense, 'people forgetting' is a lot more important than 'people
remembering' to filter unimportant issues (issues that are so unimportant
that even the original complainer does not bother to come back and
re-raise it)."

(By the way, this isn't the only strange thing about the git
community. Their mailing list requires no registration of any kind to
post, meaning that about half the messages on the list are spam).

Aaron Meurer

>
> Cheers,
>
> Brian
>
>>> * Currently, we don't really have any long/medium term planning for
>>> the project.  Our current model works great for attracting the types
>>> of contributions we are getting right now, but it makes it very
>>> difficult to tackle larger projects that require a coordinated and
>>> sustained effort by multiple people.  I don't know what it will look
>>> like, but we are going to need to do this type of planning for the
>>> project.
>>> * How do we manage communication?  Verbal communication is much more
>>> efficient than emails or even IRC.  The 4 people at Berkeley will have
>>> an incredible advantage in being able to talk daily.  We don't want to
>>> cripple or remove that advantage, but we need to figure out how to
>>> include other core devs and people from the community.  This is
>>> particularly relevant to myself as I am the only person involved in
>>> the Sloan work that is not at Berkeley.
>>
>> An idea floated around SymPy at some point to use Google+ hangouts
>> (basically, multi-way video chats).  We haven't tried it yet, but it
>> seems like it might work well.
>>
>> Aaron Meurer
>>
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>>
>>>> Sorry for the length,
>>>> --
>>>> Matthias
>>>>
>>>>
>>>>
>>>> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
>>>> And a small link, also from hacker news to conclude :
>>>> [Dear Open Source Project Leader: Quit Being A Jerk]
>>>> http://news.ycombinator.com/item?id=4921152
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>>
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From rmcgibbo at gmail.com  Mon Dec 17 19:52:55 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Mon, 17 Dec 2012 16:52:55 -0800
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAKgW=6LChOPOQhsz7XDjxJpqhddf0d-KC93DwKQH3_-jkx+YJA@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
	<CAKgW=6LChOPOQhsz7XDjxJpqhddf0d-KC93DwKQH3_-jkx+YJA@mail.gmail.com>
Message-ID: <22A89AB5-67B9-4F55-922C-B3E99C6259D7@gmail.com>

As someone who just started to contribute before the news of the Sloan funding came out,
I'm (1) happy to see this thread, because I was (privately) having some of the same questions
and (2) happy to hear that you guys are still committed to encouraging community involvement.

Also glad to hear that there will be more time for reviewing PRs!

I understand the desire/need to have a more full featured issue tracking solution, but I think for 
attracting and retaining new contributors, the minimal interface of github issues is really an asset.
(Obviously this needs to be balanced against the needs of the core team who have to filter through
the issues.)

But from personal experience, I probably would not be contributing if my first impression of the 
processes had been that contributing was some elaborate and "highly managed" process. At that
very first stage of involvement, friction matters.

-Robert McGibbon
Stanford University

On Dec 17, 2012, at 4:18 PM, Aaron Meurer wrote:

> On Mon, Dec 17, 2012 at 5:02 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> On Mon, Dec 17, 2012 at 2:36 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
>>> I remember reading about a study that said that open source projects
>>> that have funded developers actually get less contributions (or at
>>> least fewer contributors) because there is a mindset of, "why should I
>>> contribute when there is someone who is paid to do it? Surely that
>>> person/persons will get around to fixing the issue themselves."  I
>>> think the email Matthias received is indicative of this mindset.
>>> Clearly it is wrong (obviously, more contributions are still more
>>> contributions), but I think you should really think about how to make
>>> this permeate through your community, so that you minimize the number
>>> of people who end up thinking this way.
>>> 
>>> One good thing to do, as Matthias said, is to always encourage people
>>> to write their own patches.  It is an investment to spend a day
>>> helping someone write up a pull request for a fix when you could have
>>> done the whole thing yourself in under half an hour, but by guiding
>>> the new contributor, you potentially gain a new developer.  Ond?ej
>>> ?ert?k wrote a blog post a while back about how he managed to make
>>> SymPy such a successful community
>>> http://ondrejcertik.blogspot.com/2009/05/my-experience-with-running-opensource.html.
>>> The gist of it is what I just said.
>>> 
>>> For your situation, I think this additionally sends the message that
>>> it is not only OK, but encouraged to send your own patches, rather
>>> than wait for those people who are being paid to fix the bug for you.
>> 
>> Yes, I really like how Ondrej has always encouraged people to send
>> patches and I think we need to encourage people to do that.  Part of
>> the reality is that a good portion of our Sloan funded time on the
>> project will be reviewing pull requests - and that is a good thing.
> 
> It's really amazing actually.  90% of time, all you have to do is say
> "can you send a pull request fixing that?" (especially if it's clear
> that the person knows how to fix the issue), and they will do it. Even
> if you would have guessed that the person would not be able to do it,
> because they seem just like a user, and not someone who knows how to
> find bugs in the code base or use git, they surprise you more often
> than not.  I think just the simple act of asking someone to contribute
> makes them feel responsible for doing it, which they did not feel
> before.
> 
>> 
>>>> * How do we coordinate all of this activity?  Already, our github
>>>> issues are becoming unusable because of the sheer number of them.  We
>>>> have to figure out a better way of using github issues, wiki pages,
>>>> etc. to coordinate the increased activity the grant will bring.
>>> 
>>> I'm curious how you end up solving this problem.  I've found the
>>> GitHub issues to be minimalistic, which makes them easy to use, but
>>> they also lack in some key features that make managing thousands of
>>> issues bearable.
>> 
>> I know it probably sounds heretical, but I think that part of the
>> solution has to be to reduce the number of issues.  I don't mean by
>> closing issues by working harder and coding more, I mean simply
>> *closing* them outright to reflect the reality that we are not going
>> to work on them anytime soon.  There is no universe where it is useful
>> to have thousands of issues.  How many can each developer
>> realistically keep track of?  A few dozen?  When there are more than
>> that we simply ignore the issues and work on the things we want to
>> work on...I was recently talking to the lead developer of a large open
>> source project that has this problem, and he said "oh it's horrible
>> and I completely ignore our issues on github"  Personally, when I am
>> working heavily on the notebook, I maintain my own "issues list" in a
>> markdown document on my desktop.  We have to figure out a better
>> way...
> 
> This does sound heretical to me, but I have a personality that
> dislikes deleting anything.  I personally think of issues as ways of
> organizing discussions about problems/enhancement requests so that
> when the issue comes up again, it is easy to see what was already
> discussed.
> 
> SymPy has over 1000 open issues in our issue tracker (of over 3000
> total).  Admittedly, quite a few of the older of these are outdated
> and should be closed, but we don't take the time to go through them.
> But I personally feel that that many issues is manageable because we
> have extensive labeling, and Google Code has great search
> functionality (which GitHub lacks), so that I feel that I can almost
> always find the issue I want if I look for it.
> 
> I'm actually reminded of the one time that I posted a message to the
> git mailing list.  I asked them if they had an issue tracker, and they
> told me that they don't, but rather they *just* use the mailing list.
> To quote Junio Hamano, the lead developer of git:
> 
> "What we do is to take advantage of the fact that issues people do care
> about are important ones, and others that nobody cares about are not worth
> pursuing.
> 
> "In a sense, 'people forgetting' is a lot more important than 'people
> remembering' to filter unimportant issues (issues that are so unimportant
> that even the original complainer does not bother to come back and
> re-raise it)."
> 
> (By the way, this isn't the only strange thing about the git
> community. Their mailing list requires no registration of any kind to
> post, meaning that about half the messages on the list are spam).
> 
> Aaron Meurer
> 
>> 
>> Cheers,
>> 
>> Brian
>> 
>>>> * Currently, we don't really have any long/medium term planning for
>>>> the project.  Our current model works great for attracting the types
>>>> of contributions we are getting right now, but it makes it very
>>>> difficult to tackle larger projects that require a coordinated and
>>>> sustained effort by multiple people.  I don't know what it will look
>>>> like, but we are going to need to do this type of planning for the
>>>> project.
>>>> * How do we manage communication?  Verbal communication is much more
>>>> efficient than emails or even IRC.  The 4 people at Berkeley will have
>>>> an incredible advantage in being able to talk daily.  We don't want to
>>>> cripple or remove that advantage, but we need to figure out how to
>>>> include other core devs and people from the community.  This is
>>>> particularly relevant to myself as I am the only person involved in
>>>> the Sloan work that is not at Berkeley.
>>> 
>>> An idea floated around SymPy at some point to use Google+ hangouts
>>> (basically, multi-way video chats).  We haven't tried it yet, but it
>>> seems like it might work well.
>>> 
>>> Aaron Meurer
>>> 
>>>> 
>>>> Cheers,
>>>> 
>>>> Brian
>>>> 
>>>> 
>>>>> Sorry for the length,
>>>>> --
>>>>> Matthias
>>>>> 
>>>>> 
>>>>> 
>>>>> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
>>>>> And a small link, also from hacker news to conclude :
>>>>> [Dear Open Source Project Leader: Quit Being A Jerk]
>>>>> http://news.ycombinator.com/item?id=4921152
>>>>> 
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Brian E. Granger
>>>> Cal Poly State University, San Luis Obispo
>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> 
>> 
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From nick.bollweg at gmail.com  Mon Dec 17 19:58:04 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Mon, 17 Dec 2012 19:58:04 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50CFA219.4020703@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
Message-ID: <CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>

Hm... that's pretty cool... how would we know what to work on? I dislike
putting non-python placeholders in the code itself (if that's what you
meant), as, as usual, we'd like the thing to work outside of the web
notebook... even be able to edit it after the fact (and lose the blockly
editability). I was already planning to do something hash-based to make
sure you don't inadvertently destroy stuff.

    # blockly foo
    def foo(bar):
        baz = None
        return baz
    # endblockly foo

A quick thing to remember here is that part of the goodness of blockly is
the type checking; crucial for visual feedback so that you don't try to add
a string to an integer, even though python would happily oblige you! So
this would yield:

    # blockly foo(int bar): str baz
    def foo(bar):
        baz = None
        return baz
    # endblockly foo

Doing comment-based stuff would be rough, as we'd have to write some
tab-completion that knew how to work on fancy comments. Using a de facto
standard like javadoc/rst would be an option, I suppose.

Keeping that in mind, I am thinking to a) keep tab completion and b) keep
the blockly goodness, we'll need to help blockly out. Maybe something like:

    @blockly("foo", args={"bar": int}, _return={"baz": str})
    def foo(bar):
        return baz

Disadvantage here is that the @blockly, which does nothing outside of the
notebook, would be embedded in the code, which is lame.

Additionally, the output cell element gets obliterated every time, so the
decorator approach might be weird... although, as long as you are ONLY
editing a function (and the code doesn't do anything else), it could work.
Actually, I don't know if the display would work correctly.

Also, doing OO with Blockly is not really supported right now: it doesn't
really support the name-based accessing of an object's members, so the leap
could be hard here (assuming the decorator) for an instance method:

    @blockly("__init__", args={"bar": int}, _self={"bar": int})
    class Foo(object):
        def __init__(self, bar):
            self.bar = bar

Which would create a variable Block named, literally, "self.bar" that you
could set stuff to and access. However, at present, it would be hard to
have more than one blockly editor per CodeCell.

Rambling now: need to write code. My next plan was to just try to get some
variables into scope, as the definition of functions, etc. might be "Day 2"
of "Learning Python Visually," but I'll keep some of these things in mind!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121217/3485e1e1/attachment.html>

From benjaminrk at gmail.com  Mon Dec 17 20:23:55 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 17 Dec 2012 17:23:55 -0800
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <20121217182238.GA4996@phare.normalesup.org>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<20121217182238.GA4996@phare.normalesup.org>
Message-ID: <CAHNn8BWR8Wc6uOgnZR5K-X6ZzDMAqnEKGyMFZ8DCfhqNsLoWDQ@mail.gmail.com>

We always want more contributions!

I think the biggest and simplest reason to continue contributions is that
one of the major limiting factors for new contributions is core devs'
available time for reviewing PRs,
and helping contributors out with their code.  One of the major benefits
of the funding is that we should have much more time for that starting in
January (up from effectively zero for some of us).

If anything, the experience of new contributors should be *better* during
this time,
because they will get more core dev response and attention.

-MinRK


On Mon, Dec 17, 2012 at 10:22 AM, Gael Varoquaux <
gael.varoquaux at normalesup.org> wrote:

> It's a natural question that comes to mind:
>
>     Some people have more time, paid time, than me to contribute to a
>     project. They naturally know the code base better and are more
> efficient
>     when working on the project. Why should I contribute?
>
> The reason that I will keep contributing to projects even if I am not
> core dev and the core devs are doing an excellent job is because the sum
> of those 'minor' contributions are what makes an open source project
> fantastic. The unique thing about open source is that it is powered by
> users for their own needs. One works on the minor feature that makes his
> life easier. Those pile up, and the project ends up with a different
> feeling than a project developed by professionals.
>
> This is why over and over open source software answers problems better
> than software built by software giants. By giving up your option to
> contribute, you would be giving up what makes open source software rock.
>
> My 2 cents,
>
> Ga?l
>
> PS: this is also why, as a researcher drowning under all kind of work, I
> fight to keep some time to do software.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121217/26f09029/attachment.html>

From jhowison at ischool.utexas.edu  Mon Dec 17 20:29:50 2012
From: jhowison at ischool.utexas.edu (James Howison)
Date: Mon, 17 Dec 2012 19:29:50 -0600
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
Message-ID: <94415C3A935344BB96181538D6E611C6@ischool.utexas.edu>

FWIW, the only study I know that has looked at the role of full-time paid developers and volunteers on projects found that there were *more* volunteers working on modules that had full-time paid developers working on them (with an exception of situations where the commercial contribution was very focused). Of course the iPython situation is different than Eclipse, where this study was done, but perhaps this finding can help combat problematic perceptions.

<quote>  
To address whether commercial participation increased or decreased volunteer participation in Open Source communities we conducted a combination qualitative/quantitative study with interviews of Open Source developers and found that in most cases participation by commercial developers attracts volunteers but that this effect is strongly moderated by the firm?s overall strategy in relation to the project and community. We found that firms offering support services or contributing across the entire community typically increase the number of volunteer participants, while firms focused on a niche component within the community, typically have statistically insignificant effects. These findings suggest that commercial firms that seek to participate in an Open Source project should choose to participate and contribute across an entire community rather than a single niche component.  
</quote>

Wagstrom, Herbsleb, Kraut, and Mockus. The Impact of Commercial Organizations on Volunteer Participation in an Online Community. 2010 Academy of Management Annual Meeting. Montreal, Canada, August 2010.  
http://academic.patrick.wagstrom.net/publications/Wagstrom_2010_TheImpactOfCommercialOrganizationsOnVolunteerParticipationInAnOnlineCommunity.pdf?attredirects=0&d=1


--  
James Howison
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Monday, December 17, 2012 at 16:36 CST, Aaron Meurer wrote:

> I remember reading about a study that said that open source projects
> that have funded developers actually get less contributions (or at
> least fewer contributors) because there is a mindset of, "why should I
> contribute when there is someone who is paid to do it? Surely that
> person/persons will get around to fixing the issue themselves." I
> think the email Matthias received is indicative of this mindset.
> Clearly it is wrong (obviously, more contributions are still more
> contributions), but I think you should really think about how to make
> this permeate through your community, so that you minimize the number
> of people who end up thinking this way.
>  
> One good thing to do, as Matthias said, is to always encourage people
> to write their own patches. It is an investment to spend a day
> helping someone write up a pull request for a fix when you could have
> done the whole thing yourself in under half an hour, but by guiding
> the new contributor, you potentially gain a new developer. Ond?ej
> ?ert?k wrote a blog post a while back about how he managed to make
> SymPy such a successful community
> http://ondrejcertik.blogspot.com/2009/05/my-experience-with-running-opensource.html.
> The gist of it is what I just said.
>  
> For your situation, I think this additionally sends the message that
> it is not only OK, but encouraged to send your own patches, rather
> than wait for those people who are being paid to fix the bug for you.
>  
> On Mon, Dec 17, 2012 at 1:00 PM, Brian Granger <ellisonbg at gmail.com (mailto:ellisonbg at gmail.com)> wrote:
> > Matthias,
> >  
> > Thanks so much for bringing this up to the list...you bring up issues
> > that I have been thinking a lot about recently. I am sure Fernando
> > has thoughts too, but he is enjoying some snow up at Mammoth right
> > now. I should mention that I will also be visiting Fernando later
> > this week at Berkeley. I had planned on talking with him about these
> > issues so it is a good time to start the discussion...
> >  
> > > This morning I received an personal email from one of our user with the
> > > following inside:
> > >  
> > > > I was wondering whether my involvement in the development [?] make too
> > > > much
> > > > sense now, that IPython has received a grant for the next two years, [?]
> > > > it
> > > > seems that there are going to be a number of people on the team, who are
> > > > very
> > > > proficient, [?] and I wouldn't be able to contribute too much, anyway.
> > > >  
> > >  
> > >  
> >  
> >  
> >  
> > Here is my short answer: absolutely, we *really* want and need the
> > contributions of the larger community.
> >  
> > And my long answer:
> >  
> > A few years back, IPython development was moving along very slowly.
> > There were only a few core developers and we didn't get many
> > contributions from the community. Then we moved the github and
> > developed the qtconsole, notebook and zmq based parallel capabilities
> > - development has *exploded*. Our core team of developers has grown
> > dramatically and the number of contributions from the larger community
> > has grown by leaps and bounds. What stands out to me about this
> > growth is how it happens. As Matthias describes so well, it starts by
> > new people contributing a single small fix, feature, etc. They they
> > make another contribution, and then another and another until they are
> > making massive contributions to the project and helping to steer its
> > direction. Why do people do this? I am convinced that it is *deeply
> > rewarding and enjoyable* for them. Why else would they put in so much
> > effort on something that pulls them away from their day job, friends
> > and family. So what criteria should someone use to determine if they
> > will contribute to the project?
> >  
> > Do you find contributing enjoyable and rewarding? If you do, then you
> > should contribute no matter how many of us are paid to work on the
> > project.
> >  
> > But I think there is another question lurking here: will we continue
> > to welcome or value the contributions of the larger community, or will
> > people be excluded if they are not part of the Sloan funded team.
> > This is something that Fernando and I have talked about extensively
> > and the answer is that the community is and will always be the
> > foundation of the project and we will do everything we can to enable
> > people to participate in the project. This is extremely important and
> > something that the Sloan foundation is paying very close attention to.
> > In fact, I would say that the Sloan foundation would never have even
> > talked to us if it were not for the life, excitement and health of our
> > community.
> >  
> > > I realized, that even we got a lot a congratulation mails after the announce
> > > of the grant,
> > > we didn't get any expression of concern or questions on what will happened,
> > > this was
> > > a little less the case on hacker-news.
> > >  
> > > What's follow is my personal opinion, how I feel about the development of
> > > IPython over the past Year. Because I feel it is important to tell that
> > > being founded will not change how we work, will not change the people.
> > >  
> > > So as you may know, IPython did receive a Sloan Foundation Grant for the
> > > next
> > > two years, that will help the development of IPython, But this does not mean
> > > we
> > > will ever cut ourselves from our user bases.
> > >  
> >  
> >  
> >  
> > Absolutely!
> >  
> > > First of all, being in the core development team does not necessarily mean
> > > we
> > > will get any money from the grant. Some of the current core team dev will
> > > be
> > > "hired" to work part or full time on IPython. Personally I won't, even if I
> > > were
> > > paid for that it wouldn't change what I would be doing (just maybe my
> > > schedule).
> > >  
> >  
> >  
> >  
> > Yes, this is a very important point: not all of the core devs will be
> > funded by the Sloan foundation. That means that our Sloan funded work
> > *has to* be integrated with the larger project and community.
> >  
> > > This will not prevent me from fixing bug and bring improvement to IPython.
> > > Nor
> > > will I assume that Devs that are paid from this grant have more obligation
> > > to
> > > fix bugs or answer to question than I am.
> > >  
> >  
> >  
> >  
> > Absolutely, the devotion of volunteer developers is in no way inferior
> > to that of people who are paid to work on the project.
> >  
> > > Being a core dev does not mean that we are more capable than any other
> > > user of the list. There is no official way to become a core dev, but
> > > empirically, I would say that when you submit (or review) too many good pull
> > > request we give you the responsibility to merge others'.
> > >  
> > > One of the things that make IPython so great is that we develop things that
> > > we
> > > use, I do make patches for IPython because I feel the use for it in my work,
> > > the diversity of people that contribute and use it make IPython what it is
> > > now :
> > > If we look a the opened PR, we see that more than the half are opened by
> > > people
> > > that are not in the core team, and *big* PR are in this big half.
> > >  
> >  
> >  
> >  
> > Yep, at this point, we receive a significant number of PRs from
> > non-core developers.
> >  
> > > You might be concern to not being able to keep up with something some
> > > could do in minutes, and I guess this is the main concern most of you have.
> > > I
> > > reassure you we will alway take time to guide you to make a PR.
> > >  
> > > I first contributed 1 Year ago, from an ~10 lines change that took me a week
> > > to
> > > write, I had kind words from fperez, epatters, and minrk to explain me how
> > > to
> > > do things they could have done in 10 minutes tops.
> > >  
> > > Now I do the same for every pull request from users. Even if fixing a bug
> > > would
> > > take me less time than writing the explanation for an user, I still write
> > > the
> > > explanation and help do the right thing even for one line. I do strongly
> > > believe that this is worth doing in the long run. And I know that I am not
> > > the
> > > only one that will keep up doing so as long as he can. Often the most
> > > difficult
> > > thing is to know the code base, not actually writing the patch, and nothing
> > > but
> > > time helps that.
> > >  
> > > Heck, the Sloan-Foundation almost exclusively give to project linked to
> > > education, if we can't be good teacher, how could build a tool to help other
> > > to
> > > teach ?
> > >  
> >  
> >  
> >  
> > Absolutely...
> >  
> > > So yes I will continue to contribute, on my free time, I hope I'll find some
> > > way to get more time to be involve. Yes the development will probably go
> > > faster, and there are things I won't be able to follow, but even now, I
> > > already
> > > discover that some stuff had been added to IPython during the week that I
> > > wasn't aware of, and I'm convinced there are still room for all kind of
> > > contribution, from QTconsole UI to big AST hook to do amazing stuff I don't
> > > even dream of.
> > >  
> > > Hopping to have convince you that we will not change, and that you will
> > > continue to
> > > bring us big surprises with awesome idea and PRs.
> > >  
> > > Thanks to all of you, also and especially to Fernando without whom IPython
> > > would not exist today.
> > >  
> >  
> >  
> >  
> > Absolutely, thank you to *everyone* who participates in the project
> > and community - and to Fernando for blowing off his thesis..
> >  
> > So I think there are some important issues that the Sloan foundation
> > grant does bring up, that we will have to work though. Right now we
> > are already flooded by the number of pull requests and contributions
> > that we are getting. With 5 of us working full time, the level of
> > activity is going to go through the roof. This is exciting, but will
> > definitely change some aspects of the project. Some of the questions
> > that we are thinking about right now:
> >  
> > * How do we coordinate all of this activity? Already, our github
> > issues are becoming unusable because of the sheer number of them. We
> > have to figure out a better way of using github issues, wiki pages,
> > etc. to coordinate the increased activity the grant will bring.
> >  
>  
>  
>  
> I'm curious how you end up solving this problem. I've found the
> GitHub issues to be minimalistic, which makes them easy to use, but
> they also lack in some key features that make managing thousands of
> issues bearable.
>  
> > * Currently, we don't really have any long/medium term planning for
> > the project. Our current model works great for attracting the types
> > of contributions we are getting right now, but it makes it very
> > difficult to tackle larger projects that require a coordinated and
> > sustained effort by multiple people. I don't know what it will look
> > like, but we are going to need to do this type of planning for the
> > project.
> > * How do we manage communication? Verbal communication is much more
> > efficient than emails or even IRC. The 4 people at Berkeley will have
> > an incredible advantage in being able to talk daily. We don't want to
> > cripple or remove that advantage, but we need to figure out how to
> > include other core devs and people from the community. This is
> > particularly relevant to myself as I am the only person involved in
> > the Sloan work that is not at Berkeley.
> >  
>  
>  
>  
> An idea floated around SymPy at some point to use Google+ hangouts
> (basically, multi-way video chats). We haven't tried it yet, but it
> seems like it might work well.
>  
> Aaron Meurer
>  
> >  
> > Cheers,
> >  
> > Brian
> >  
> >  
> > > Sorry for the length,
> > > --
> > > Matthias
> > >  
> > >  
> > >  
> > > [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
> > > And a small link, also from hacker news to conclude :
> > > [Dear Open Source Project Leader: Quit Being A Jerk]
> > > http://news.ycombinator.com/item?id=4921152
> > >  
> > > _______________________________________________
> > > IPython-dev mailing list
> > > IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/ipython-dev
> > >  
> >  
> >  
> >  
> >  
> >  
> > --
> > Brian E. Granger
> > Cal Poly State University, San Luis Obispo
> > bgranger at calpoly.edu (mailto:bgranger at calpoly.edu) and ellisonbg at gmail.com (mailto:ellisonbg at gmail.com)
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >  
>  
>  
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org (mailto:IPython-dev at scipy.org)
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>  





From taavi.burns at gmail.com  Tue Dec 18 00:58:57 2012
From: taavi.burns at gmail.com (Taavi Burns)
Date: Tue, 18 Dec 2012 00:58:57 -0500
Subject: [IPython-dev] IPython nosetests extension
In-Reply-To: <CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>
References: <CAKrk_d_081PjEcCqiFTUQPM=uLQ+f3aOF=GWQhPdczScnmhTGw@mail.gmail.com>
	<CAH4pYpQRBXCNuyndEcZosmrG-4jgXUK4bU+MFK0apNEV_2uhug@mail.gmail.com>
	<CAKrk_d8wL93isdsZQg1ENfatESncmZina2=wEn1b=xbdVXgyYw@mail.gmail.com>
	<CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>
Message-ID: <CAKrk_d9+x=Y6_ODeV+xxqAXTAG-0dazV2jkaqubdmmy6SX9ESg@mail.gmail.com>

On Fri, Dec 7, 2012 at 1:27 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> > I've enhanced the test failure stack traces such that the function's
> > "filename" (like <ipython-input-DD-HHHHHHHHHHHH>) links to the cell that
> > defines that code. That makes it easy to click close to the definition
> of a
> > failing test or function. Even better would be linking directly to the
> line
> > of code in question? :)  Unfortunately the current implementation has a
> > visual bug when linking to an anchor: the top IPython bar shifts up about
> > 0.3em (it shifts back if you hit the empty fragment, "#").
>
> Ahh, this is a nice feature.  Can you describe a bit more about how
> you track this information as it flows though the code.  Do the cell
> ids get sent to the kernel?  What does the kernet do with them?
>

When each cell gets compiled, ipython generates a
(IPython.core.compilerop.code_name, I think) and uses that the
(pseudo-)filename associated with the code object. I don't have to track it
myself:
1. The traceback lists the (pseudo-)filename in a pretty recognizeable
way:  re.search(r'ipython-input-(?P<cell_eval_order>\d+)-[0-9a-f]{12}', ?)
2. Each cell has the cell_eval_order in square brackets beside it.

All I have to do is link the two. :) The kernel itself doesn't need to know
anything, I'm just taking advantage of the predictability of the HTML
output. One could as easily do all the linkifying in JS instead of
beforehand in the plugin's Python code.


> > I do have a few questions up-front:
> > 1. Is there a better way than peeking at sys.displayhook to determine
> what
> > kind of output to produce?
>
> I am not quite sure what you mean and how you "peek" and
> sys.displayhook for this.
>

For now, I stole a page from
https://github.com/catherinedevlin/ipython_doctester with respect to
displaying text/plain when using ipython console vs displaying
text/html+application/javascript in the notebook. But it feels like a
REALLY ugly hack (and requires pyzmq to be installed just for an import to
see that we're NOT using the notebook?).

The particular code is
https://github.com/taavi/ipython_nose/blob/master/ipython_nose.py#L293 .


> > 2. I'd really like to have cell anchor generation happen always. Seems
> like
> > it would be useful to provide intra-notebook links to cells. I'm not sure
> > how one would manage those links as the execution order evolves, though.
> For
> > this extension, that migration is actually a good thing, as the anchor
> > points to the code that was actually run even if it's no longer visible.
>
> I am trying to understand what the usage cases are.  Would these links
> be typed in by hand by a user?  If so, we would want the ids to have a
> human readable form.  If they are always generated by code they could
> be ugly.
>

Nope, these are just the automatically-generated links already mentioned
above. They're there just so the user can click on a psuedo-filename and
(hopefully) have the browser scroll to the cell that contained the test
code itself.


> The other thing we have to keep in mind is that we can't ever couple
> the kernel to the notebook frontend.  That is, the kernel can't ever
> know about these cell ids in any formal way.
>

That makes sense. I'm more interested in the notebook frontend being able
to take advantage of the kernel's existing data, which is sort of the
opposite thing (I think).

> 3. I'd love to have a consistent interface to stream something like stdout
> > character-wise instead of line-wise. Right now it's hacked to hook stdout
> > directly on the console, and with even more horrible jQuery machinations
> on
> > the notebook side.
>
> Can't you call flush to do this?  Can you describe this a bit more.
> Again, not following how all of this is used.  It sounds messy...
>

Nope, somehow whenever I try to write a single char for each test, it comes
out on its own line (wrapped in its own pre tag).

Huh. But it looks like if I do this in an ipython notebook cell:
import sys
import time
sys.stdout.write('a')
sys.stdout.flush()
time.sleep(5)
sys.stdout.write('b')

it works! So there must be something else going on causing the extra tags.
Thanks for pointing out that it OUGHT to work!
https://github.com/taavi/ipython_nose/issues/4

-- 
taa
/*eof*/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/6326195b/attachment.html>

From rmcgibbo at gmail.com  Tue Dec 18 02:27:35 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Mon, 17 Dec 2012 23:27:35 -0800
Subject: [IPython-dev] function specific tab completion via annotations
Message-ID: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>

Hey,

I just posted a new PR for annotation based custom tab completion. https://github.com/ipython/ipython/pull/2701

The code is fairly complex (~1000 new lines), so I assume that there are bugs. For that reason (and because its fun!), I encourage people
to try taking it out for a spin.


It lets you do things like this:
In [1]: from IPython.extensions.completion import tab_complete, globs_to,  literal

In [2]: @tab_complete
  ...: def baz(x) -> str:
  ...:     pass
  ...:

In [3]: baz(notevaluated).<TAB>
.capitalize  .find        .isspace     .partition   .rstrip      .translate
.center      .format      .istitle     .replace     .split       .upper
.count       .index       .isupper     .rfind       .splitlines  .zfill
.decode      .isalnum     .join        .rindex      .startswith
.encode      .isalpha     .ljust       .rjust       .strip
.endswith    .isdigit     .lower       .rpartition  .swapcase
.expandtabs  .islower     .lstrip      .rsplit      .title

And this:
In [4]: @tab_complete
   ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
   ...:     pass
   ...:

In [5]: foo(<TAB>
'COPYING.txt'        'dist/'              'setupext/'
'IPython/'           'docs/'              'tools/'
'__pycache__/'       'ipython.egg-info/'
'build/'             'scripts/'

In[6]: foo('COPYING.txt', <TAB>
'r', 'w'


-Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121217/31e1e3b3/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 18 02:34:21 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 08:34:21 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
Message-ID: <5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>

Le 18 d?c. 2012 ? 01:58, Nicholas Bollweg a ?crit :



>  I was already planning to do something hash-based to make sure you don't inadvertently destroy stuff.
> 
>     # blockly foo
>     def foo(bar):
>         baz = None
>         return baz
>     # endblockly foo


It's fine to use # if the notebook is bind to python, 
but don't forget that it might not always be the only case, so we probably won't hardcode something like that in the javascript. 

Also we have to achieve to bring code mirror 3.0 into the notebook. 
I don't know how much different the API is, but we already broke some User habits when upgrading from 2.1 to 2.3 I think.
-- 
Matthias



From ellisonbg at gmail.com  Tue Dec 18 02:49:39 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 17 Dec 2012 23:49:39 -0800
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAKgW=6LChOPOQhsz7XDjxJpqhddf0d-KC93DwKQH3_-jkx+YJA@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
	<CAKgW=6LChOPOQhsz7XDjxJpqhddf0d-KC93DwKQH3_-jkx+YJA@mail.gmail.com>
Message-ID: <CAH4pYpQgSuuC4wDa22p=sObL6nb_OzY+mTFdAbRa65Fwizwskw@mail.gmail.com>

>> I know it probably sounds heretical, but I think that part of the
>> solution has to be to reduce the number of issues.  I don't mean by
>> closing issues by working harder and coding more, I mean simply
>> *closing* them outright to reflect the reality that we are not going
>> to work on them anytime soon.  There is no universe where it is useful
>> to have thousands of issues.  How many can each developer
>> realistically keep track of?  A few dozen?  When there are more than
>> that we simply ignore the issues and work on the things we want to
>> work on...I was recently talking to the lead developer of a large open
>> source project that has this problem, and he said "oh it's horrible
>> and I completely ignore our issues on github"  Personally, when I am
>> working heavily on the notebook, I maintain my own "issues list" in a
>> markdown document on my desktop.  We have to figure out a better
>> way...
>
> This does sound heretical to me, but I have a personality that
> dislikes deleting anything.  I personally think of issues as ways of
> organizing discussions about problems/enhancement requests so that
> when the issue comes up again, it is easy to see what was already
> discussed.

Keep in mind, I am not advocating *deleting* issues, just closing
them.  Closed issues are still tagged and searchable and preserve the
record of any work or discussion on the topic.

> SymPy has over 1000 open issues in our issue tracker (of over 3000
> total).  Admittedly, quite a few of the older of these are outdated
> and should be closed, but we don't take the time to go through them.
> But I personally feel that that many issues is manageable because we
> have extensive labeling, and Google Code has great search
> functionality (which GitHub lacks), so that I feel that I can almost
> always find the issue I want if I look for it.

Even with github issues now I can find an issue if I know it exists.
Where it is less useful is as a tool for deciding what we/I should be
working on next and tracking our current work.   Sure we have
priorities in our labeling scheme, but I think we mostly ignore those
when it comes time for each of us to pick things to work on.  When
issues fall off our email radar, they quickly disappear into the
abyss.

I think that issues serve a *very important role* in preserving
discussions, even for issues that don't lead to code.  I do think the
our current approach to issue does work very well for that.

> I'm actually reminded of the one time that I posted a message to the
> git mailing list.  I asked them if they had an issue tracker, and they
> told me that they don't, but rather they *just* use the mailing list.
> To quote Junio Hamano, the lead developer of git:
>
> "What we do is to take advantage of the fact that issues people do care
> about are important ones, and others that nobody cares about are not worth
> pursuing.
>
> "In a sense, 'people forgetting' is a lot more important than 'people
> remembering' to filter unimportant issues (issues that are so unimportant
> that even the original complainer does not bother to come back and
> re-raise it)."
>
> (By the way, this isn't the only strange thing about the git
> community. Their mailing list requires no registration of any kind to
> post, meaning that about half the messages on the list are spam).
>
> Aaron Meurer
>
>>
>> Cheers,
>>
>> Brian
>>
>>>> * Currently, we don't really have any long/medium term planning for
>>>> the project.  Our current model works great for attracting the types
>>>> of contributions we are getting right now, but it makes it very
>>>> difficult to tackle larger projects that require a coordinated and
>>>> sustained effort by multiple people.  I don't know what it will look
>>>> like, but we are going to need to do this type of planning for the
>>>> project.
>>>> * How do we manage communication?  Verbal communication is much more
>>>> efficient than emails or even IRC.  The 4 people at Berkeley will have
>>>> an incredible advantage in being able to talk daily.  We don't want to
>>>> cripple or remove that advantage, but we need to figure out how to
>>>> include other core devs and people from the community.  This is
>>>> particularly relevant to myself as I am the only person involved in
>>>> the Sloan work that is not at Berkeley.
>>>
>>> An idea floated around SymPy at some point to use Google+ hangouts
>>> (basically, multi-way video chats).  We haven't tried it yet, but it
>>> seems like it might work well.
>>>
>>> Aaron Meurer
>>>
>>>>
>>>> Cheers,
>>>>
>>>> Brian
>>>>
>>>>
>>>>> Sorry for the length,
>>>>> --
>>>>> Matthias
>>>>>
>>>>>
>>>>>
>>>>> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
>>>>> And a small link, also from hacker news to conclude :
>>>>> [Dear Open Source Project Leader: Quit Being A Jerk]
>>>>> http://news.ycombinator.com/item?id=4921152
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Brian E. Granger
>>>> Cal Poly State University, San Luis Obispo
>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Dec 18 02:51:29 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 17 Dec 2012 23:51:29 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
Message-ID: <CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>

This looks really great.  Have you tested it with the notebook yet?  I
would imagine that it should already work there as well.

On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
> Hey,
>
> I just posted a new PR for annotation based custom tab completion.
> https://github.com/ipython/ipython/pull/2701
>
> The code is fairly complex (~1000 new lines), so I assume that there are
> bugs. For that reason (and because its fun!), I encourage people
> to try taking it out for a spin.
>
>
> It lets you do things like this:
>
> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
> literal
>
> In [2]: @tab_complete
>   ...: def baz(x) -> str:
>   ...:     pass
>   ...:
>
> In [3]: baz(notevaluated).<TAB>
> .capitalize  .find        .isspace     .partition   .rstrip      .translate
> .center      .format      .istitle     .replace     .split       .upper
> .count       .index       .isupper     .rfind       .splitlines  .zfill
> .decode      .isalnum     .join        .rindex      .startswith
> .encode      .isalpha     .ljust       .rjust       .strip
> .endswith    .isdigit     .lower       .rpartition  .swapcase
> .expandtabs  .islower     .lstrip      .rsplit      .title
>
>
> And this:
>
> In [4]: @tab_complete
>    ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>    ...:     pass
>    ...:
>
> In [5]: foo(<TAB>
> 'COPYING.txt'        'dist/'              'setupext/'
> 'IPython/'           'docs/'              'tools/'
> '__pycache__/'       'ipython.egg-info/'
> 'build/'             'scripts/'
>
> In[6]: foo('COPYING.txt', <TAB>
> 'r', 'w'
>
>
>
> -Robert
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From rmcgibbo at gmail.com  Tue Dec 18 03:03:33 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Tue, 18 Dec 2012 00:03:33 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
Message-ID: <880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>

It's not working in the notebook with the argument specific completions, but the return value completion is working. There must be some issue with a difference in the line splitting with readline and without. I'll track it down.

-Robert

On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:

> This looks really great.  Have you tested it with the notebook yet?  I
> would imagine that it should already work there as well.
> 
> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>> Hey,
>> 
>> I just posted a new PR for annotation based custom tab completion.
>> https://github.com/ipython/ipython/pull/2701
>> 
>> The code is fairly complex (~1000 new lines), so I assume that there are
>> bugs. For that reason (and because its fun!), I encourage people
>> to try taking it out for a spin.
>> 
>> 
>> It lets you do things like this:
>> 
>> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>> literal
>> 
>> In [2]: @tab_complete
>>  ...: def baz(x) -> str:
>>  ...:     pass
>>  ...:
>> 
>> In [3]: baz(notevaluated).<TAB>
>> .capitalize  .find        .isspace     .partition   .rstrip      .translate
>> .center      .format      .istitle     .replace     .split       .upper
>> .count       .index       .isupper     .rfind       .splitlines  .zfill
>> .decode      .isalnum     .join        .rindex      .startswith
>> .encode      .isalpha     .ljust       .rjust       .strip
>> .endswith    .isdigit     .lower       .rpartition  .swapcase
>> .expandtabs  .islower     .lstrip      .rsplit      .title
>> 
>> 
>> And this:
>> 
>> In [4]: @tab_complete
>>   ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>>   ...:     pass
>>   ...:
>> 
>> In [5]: foo(<TAB>
>> 'COPYING.txt'        'dist/'              'setupext/'
>> 'IPython/'           'docs/'              'tools/'
>> '__pycache__/'       'ipython.egg-info/'
>> 'build/'             'scripts/'
>> 
>> In[6]: foo('COPYING.txt', <TAB>
>> 'r', 'w'
>> 
>> 
>> 
>> -Robert
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From bussonniermatthias at gmail.com  Tue Dec 18 03:13:10 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 09:13:10 +0100
Subject: [IPython-dev] IPython nosetests extension
In-Reply-To: <CAKrk_d9+x=Y6_ODeV+xxqAXTAG-0dazV2jkaqubdmmy6SX9ESg@mail.gmail.com>
References: <CAKrk_d_081PjEcCqiFTUQPM=uLQ+f3aOF=GWQhPdczScnmhTGw@mail.gmail.com>
	<CAH4pYpQRBXCNuyndEcZosmrG-4jgXUK4bU+MFK0apNEV_2uhug@mail.gmail.com>
	<CAKrk_d8wL93isdsZQg1ENfatESncmZina2=wEn1b=xbdVXgyYw@mail.gmail.com>
	<CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>
	<CAKrk_d9+x=Y6_ODeV+xxqAXTAG-0dazV2jkaqubdmmy6SX9ESg@mail.gmail.com>
Message-ID: <79F9959B-4ACC-46F9-B48A-E7781DEE669B@gmail.com>


> 
> I am not quite sure what you mean and how you "peek" and
> sys.displayhook for this.
> 
> For now, I stole a page from https://github.com/catherinedevlin/ipython_doctester with respect to displaying text/plain when using ipython console vs displaying text/html+application/javascript in the notebook. But it feels like a REALLY ugly hack (and requires pyzmq to be installed just for an import to see that we're NOT using the notebook?).
> 
> The particular code is https://github.com/taavi/ipython_nose/blob/master/ipython_nose.py#L293 .

Haven't read the all thread, but, why not having a object that have 
both an _repr_ and and _repr_html_ (supported by qtconsole) 


class Bunch(object):
    
    def __init__(self,value):
        self.value = value
    
    
    def _repr_javascript_(self):
        """only notebook can do
        
        will result in output in JS console **not in notebook itself**
        """
        return 'console.log("'+self.value+'")'
    
    def _repr_html_(self):
        """qtconsole can do too"""
        return 'html:'+self.value
    
    def __repr__(self):
        """ the rest """
        return 'plain:'+self.value
   
from IPython.display import display

def displayer(string):
    display(Bunch(string))

...
displayer('foobar')

-> what you expect depending on the frontend.

My 2 cents.
-- 
Matthias

From bussonniermatthias at gmail.com  Tue Dec 18 03:22:41 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 09:22:41 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
Message-ID: <B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>


Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :

> It's not working in the notebook with the argument specific completions, but the return value completion is working. There must be some issue with a difference in the line splitting with readline and without. I'll track it down.

That's probably the js that removes everything between bracket by nothing
(IIRC, first completer was doing that.)

baz(notevaluated).<tab>
to 
baz.<tab>

before completion. (or is it tooltip... that does it) 

Or a security about 'as you type' 

Actually every key press recall the completer once it has been invoked and it decide to 'kill itself' 
on certain things (like closing bracket, spaces...) 

That would be 

IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
-- 
Matthias



> 
> -Robert
> 
> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
> 
>> This looks really great.  Have you tested it with the notebook yet?  I
>> would imagine that it should already work there as well.
>> 
>> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>>> Hey,
>>> 
>>> I just posted a new PR for annotation based custom tab completion.
>>> https://github.com/ipython/ipython/pull/2701
>>> 
>>> The code is fairly complex (~1000 new lines), so I assume that there are
>>> bugs. For that reason (and because its fun!), I encourage people
>>> to try taking it out for a spin.
>>> 
>>> 
>>> It lets you do things like this:
>>> 
>>> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>>> literal
>>> 
>>> In [2]: @tab_complete
>>> ...: def baz(x) -> str:
>>> ...:     pass
>>> ...:
>>> 
>>> In [3]: baz(notevaluated).<TAB>
>>> .capitalize  .find        .isspace     .partition   .rstrip      .translate
>>> .center      .format      .istitle     .replace     .split       .upper
>>> .count       .index       .isupper     .rfind       .splitlines  .zfill
>>> .decode      .isalnum     .join        .rindex      .startswith
>>> .encode      .isalpha     .ljust       .rjust       .strip
>>> .endswith    .isdigit     .lower       .rpartition  .swapcase
>>> .expandtabs  .islower     .lstrip      .rsplit      .title
>>> 
>>> 
>>> And this:
>>> 
>>> In [4]: @tab_complete
>>>  ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>>>  ...:     pass
>>>  ...:
>>> 
>>> In [5]: foo(<TAB>
>>> 'COPYING.txt'        'dist/'              'setupext/'
>>> 'IPython/'           'docs/'              'tools/'
>>> '__pycache__/'       'ipython.egg-info/'
>>> 'build/'             'scripts/'
>>> 
>>> In[6]: foo('COPYING.txt', <TAB>
>>> 'r', 'w'
>>> 
>>> 
>>> 
>>> -Robert
>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>> 
>> 
>> 
>> -- 
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From rmcgibbo at gmail.com  Tue Dec 18 03:28:16 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Tue, 18 Dec 2012 00:28:16 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
Message-ID: <93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>

Marrhias

the baz(notevaluated).<TAB> is working in the notebook.

it's the foo(<TAB> that's not working... yet.

my fridge is not out of beer, so i'm still working :)

-Robert

On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:

> 
> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
> 
>> It's not working in the notebook with the argument specific completions, but the return value completion is working. There must be some issue with a difference in the line splitting with readline and without. I'll track it down.
> 
> That's probably the js that removes everything between bracket by nothing
> (IIRC, first completer was doing that.)
> 
> baz(notevaluated).<tab>
> to 
> baz.<tab>
> 
> before completion. (or is it tooltip... that does it) 
> 
> Or a security about 'as you type' 
> 
> Actually every key press recall the completer once it has been invoked and it decide to 'kill itself' 
> on certain things (like closing bracket, spaces...) 
> 
> That would be 
> 
> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
> -- 
> Matthias
> 
> 
> 
>> 
>> -Robert
>> 
>> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
>> 
>>> This looks really great.  Have you tested it with the notebook yet?  I
>>> would imagine that it should already work there as well.
>>> 
>>> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>>>> Hey,
>>>> 
>>>> I just posted a new PR for annotation based custom tab completion.
>>>> https://github.com/ipython/ipython/pull/2701
>>>> 
>>>> The code is fairly complex (~1000 new lines), so I assume that there are
>>>> bugs. For that reason (and because its fun!), I encourage people
>>>> to try taking it out for a spin.
>>>> 
>>>> 
>>>> It lets you do things like this:
>>>> 
>>>> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>>>> literal
>>>> 
>>>> In [2]: @tab_complete
>>>> ...: def baz(x) -> str:
>>>> ...:     pass
>>>> ...:
>>>> 
>>>> In [3]: baz(notevaluated).<TAB>
>>>> .capitalize  .find        .isspace     .partition   .rstrip      .translate
>>>> .center      .format      .istitle     .replace     .split       .upper
>>>> .count       .index       .isupper     .rfind       .splitlines  .zfill
>>>> .decode      .isalnum     .join        .rindex      .startswith
>>>> .encode      .isalpha     .ljust       .rjust       .strip
>>>> .endswith    .isdigit     .lower       .rpartition  .swapcase
>>>> .expandtabs  .islower     .lstrip      .rsplit      .title
>>>> 
>>>> 
>>>> And this:
>>>> 
>>>> In [4]: @tab_complete
>>>> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>>>> ...:     pass
>>>> ...:
>>>> 
>>>> In [5]: foo(<TAB>
>>>> 'COPYING.txt'        'dist/'              'setupext/'
>>>> 'IPython/'           'docs/'              'tools/'
>>>> '__pycache__/'       'ipython.egg-info/'
>>>> 'build/'             'scripts/'
>>>> 
>>>> In[6]: foo('COPYING.txt', <TAB>
>>>> 'r', 'w'
>>>> 
>>>> 
>>>> 
>>>> -Robert
>>>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/2b0aa477/attachment.html>

From jason-sage at creativetrax.com  Tue Dec 18 03:30:02 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 01:30:02 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAH4pYpQgSuuC4wDa22p=sObL6nb_OzY+mTFdAbRa65Fwizwskw@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
	<CAKgW=6LChOPOQhsz7XDjxJpqhddf0d-KC93DwKQH3_-jkx+YJA@mail.gmail.com>
	<CAH4pYpQgSuuC4wDa22p=sObL6nb_OzY+mTFdAbRa65Fwizwskw@mail.gmail.com>
Message-ID: <50D0298A.4060502@creativetrax.com>

On 12/18/12 12:49 AM, Brian Granger wrote:
> Keep in mind, I am not advocating*deleting*  issues, just closing
> them.  Closed issues are still tagged and searchable and preserve the
> record of any work or discussion on the topic.

Too bad you can't specify the resolution reason (fixed, wontfix, 
duplicate, etc.) when you close (like every other self-respecting bug 
tracker I know :).  I guess you can add yet another tag, but that's 
suboptimal compared to having the structured information of "resolution 
reason".

Thanks,

Jason



From bussonniermatthias at gmail.com  Tue Dec 18 04:17:24 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 18 Dec 2012 10:17:24 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
Message-ID: <CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>

Ah... Then it's probably the tooltip code that take over.

Same folder, codecell.js look for TAB or pre_cursor.

Completion is not triggerd if char in front of cursor is space or opening
bracket.

Short from my phone.
Le 18 d?c. 2012 09:28, "Robert McGibbon" <rmcgibbo at gmail.com> a ?crit :

> Marrhias
>
> the baz(notevaluated).<TAB> *is* working in the notebook.
>
> it's the foo(<TAB> that's not working... yet.
>
> my fridge is not out of beer, so i'm still working :)
>
> -Robert
>
> On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:
>
>
> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
>
> It's not working in the notebook with the argument specific completions,
> but the return value completion is working. There must be some issue with a
> difference in the line splitting with readline and without. I'll track it
> down.
>
>
> That's probably the js that removes everything between bracket by nothing
> (IIRC, first completer was doing that.)
>
> baz(notevaluated).<tab>
> to
> baz.<tab>
>
> before completion. (or is it tooltip... that does it)
>
> Or a security about 'as you type'
>
> Actually every key press recall the completer once it has been invoked and
> it decide to 'kill itself'
> on certain things (like closing bracket, spaces...)
>
> That would be
>
> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
> --
> Matthias
>
>
>
>
> -Robert
>
>
> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
>
>
> This looks really great.  Have you tested it with the notebook yet?  I
>
> would imagine that it should already work there as well.
>
>
> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com>
> wrote:
>
> Hey,
>
>
> I just posted a new PR for annotation based custom tab completion.
>
> https://github.com/ipython/ipython/pull/2701
>
>
> The code is fairly complex (~1000 new lines), so I assume that there are
>
> bugs. For that reason (and because its fun!), I encourage people
>
> to try taking it out for a spin.
>
>
>
> It lets you do things like this:
>
>
> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>
> literal
>
>
> In [2]: @tab_complete
>
> ...: def baz(x) -> str:
>
> ...:     pass
>
> ...:
>
>
> In [3]: baz(notevaluated).<TAB>
>
> .capitalize  .find        .isspace     .partition   .rstrip      .translate
>
> .center      .format      .istitle     .replace     .split       .upper
>
> .count       .index       .isupper     .rfind       .splitlines  .zfill
>
> .decode      .isalnum     .join        .rindex      .startswith
>
> .encode      .isalpha     .ljust       .rjust       .strip
>
> .endswith    .isdigit     .lower       .rpartition  .swapcase
>
> .expandtabs  .islower     .lstrip      .rsplit      .title
>
>
>
> And this:
>
>
> In [4]: @tab_complete
>
> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>
> ...:     pass
>
> ...:
>
>
> In [5]: foo(<TAB>
>
> 'COPYING.txt'        'dist/'              'setupext/'
>
> 'IPython/'           'docs/'              'tools/'
>
> '__pycache__/'       'ipython.egg-info/'
>
> 'build/'             'scripts/'
>
>
> In[6]: foo('COPYING.txt', <TAB>
>
> 'r', 'w'
>
>
>
>
> -Robert
>
>
> _______________________________________________
>
> IPython-dev mailing list
>
> IPython-dev at scipy.org
>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
>
> --
>
> Brian E. Granger
>
> Cal Poly State University, San Luis Obispo
>
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
>
> IPython-dev mailing list
>
> IPython-dev at scipy.org
>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
>
> IPython-dev mailing list
>
> IPython-dev at scipy.org
>
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/496fe686/attachment.html>

From rmcgibbo at gmail.com  Tue Dec 18 04:52:20 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Tue, 18 Dec 2012 01:52:20 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
Message-ID: <9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>

Matthias,

Good call. That's it exactly. Unfortunately, I think dealing with this issue is a little above my expertise/experience.

When I apply this patch (pasted below) to the PR, the tab completions show in the notebook for foo(<TAB>.

The problem is that behavior kind of conflicts with the tooltip feature, which is not a feature of the terminal console.

-Robert

p.s.

I think I'm finally starting to see the wisdom of the comment on line 1046 of IPython/core/complete.py,

>         # FIXME: we should extend our api to return a dict with completions for
>         # different types of objects.  The rlcomplete() method could then
>         # simply collapse the dict into a list for readline, but we'd have
>         # richer completion semantics in other evironments.


$ git diff
diff --git a/IPython/frontend/html/notebook/static/js/codecell.js b/IPython/frontend/html/notebook/static/js/codecell.js
index ea34719..000734b 100644
--- a/IPython/frontend/html/notebook/static/js/codecell.js
+++ b/IPython/frontend/html/notebook/static/js/codecell.js
@@ -155,6 +155,8 @@ var IPython = (function (IPython) {
                 IPython.tooltip.request(that);
                 // Prevent the event from bubbling up.
                 event.stop();
+                console.log('RTM DEBUG');
+                this.completer.startCompletion();
                 // Prevent CodeMirror from handling the tab.
                 return true;
             } else {
diff --git a/IPython/frontend/html/notebook/static/js/completer.js b/IPython/frontend/html/notebook/static/js/completer.js
index ac2d95c..94aff4b 100644
--- a/IPython/frontend/html/notebook/static/js/completer.js
+++ b/IPython/frontend/html/notebook/static/js/completer.js
@@ -105,8 +105,9 @@ var IPython = (function (IPython) {
         // we need to check that we are still on a word boundary
         // because while typing the completer is still reinvoking itself
         if (!/[0-9a-z._/\\:~-]/i.test(pre_cursor)) {
-            this.close();
-            return;
+            //this.close();
+            //return;
+            console.log('RTM DEBUG2');
         }
 
         this.autopick = false;


On Dec 18, 2012, at 1:17 AM, Matthias Bussonnier wrote:

> Ah... Then it's probably the tooltip code that take over.
> 
> Same folder, codecell.js look for TAB or pre_cursor.
> 
> Completion is not triggerd if char in front of cursor is space or opening bracket.
> 
> Short from my phone.
> 
> Le 18 d?c. 2012 09:28, "Robert McGibbon" <rmcgibbo at gmail.com> a ?crit :
> Marrhias
> 
> the baz(notevaluated).<TAB> is working in the notebook.
> 
> it's the foo(<TAB> that's not working... yet.
> 
> my fridge is not out of beer, so i'm still working :)
> 
> -Robert
> 
> On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:
> 
>> 
>> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
>> 
>>> It's not working in the notebook with the argument specific completions, but the return value completion is working. There must be some issue with a difference in the line splitting with readline and without. I'll track it down.
>> 
>> That's probably the js that removes everything between bracket by nothing
>> (IIRC, first completer was doing that.)
>> 
>> baz(notevaluated).<tab>
>> to 
>> baz.<tab>
>> 
>> before completion. (or is it tooltip... that does it) 
>> 
>> Or a security about 'as you type' 
>> 
>> Actually every key press recall the completer once it has been invoked and it decide to 'kill itself' 
>> on certain things (like closing bracket, spaces...) 
>> 
>> That would be 
>> 
>> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
>> -- 
>> Matthias
>> 
>> 
>> 
>>> 
>>> -Robert
>>> 
>>> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
>>> 
>>>> This looks really great.  Have you tested it with the notebook yet?  I
>>>> would imagine that it should already work there as well.
>>>> 
>>>> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>>>>> Hey,
>>>>> 
>>>>> I just posted a new PR for annotation based custom tab completion.
>>>>> https://github.com/ipython/ipython/pull/2701
>>>>> 
>>>>> The code is fairly complex (~1000 new lines), so I assume that there are
>>>>> bugs. For that reason (and because its fun!), I encourage people
>>>>> to try taking it out for a spin.
>>>>> 
>>>>> 
>>>>> It lets you do things like this:
>>>>> 
>>>>> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>>>>> literal
>>>>> 
>>>>> In [2]: @tab_complete
>>>>> ...: def baz(x) -> str:
>>>>> ...:     pass
>>>>> ...:
>>>>> 
>>>>> In [3]: baz(notevaluated).<TAB>
>>>>> .capitalize  .find        .isspace     .partition   .rstrip      .translate
>>>>> .center      .format      .istitle     .replace     .split       .upper
>>>>> .count       .index       .isupper     .rfind       .splitlines  .zfill
>>>>> .decode      .isalnum     .join        .rindex      .startswith
>>>>> .encode      .isalpha     .ljust       .rjust       .strip
>>>>> .endswith    .isdigit     .lower       .rpartition  .swapcase
>>>>> .expandtabs  .islower     .lstrip      .rsplit      .title
>>>>> 
>>>>> 
>>>>> And this:
>>>>> 
>>>>> In [4]: @tab_complete
>>>>> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>>>>> ...:     pass
>>>>> ...:
>>>>> 
>>>>> In [5]: foo(<TAB>
>>>>> 'COPYING.txt'        'dist/'              'setupext/'
>>>>> 'IPython/'           'docs/'              'tools/'
>>>>> '__pycache__/'       'ipython.egg-info/'
>>>>> 'build/'             'scripts/'
>>>>> 
>>>>> In[6]: foo('COPYING.txt', <TAB>
>>>>> 'r', 'w'
>>>>> 
>>>>> 
>>>>> 
>>>>> -Robert
>>>>> 
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> Brian E. Granger
>>>> Cal Poly State University, San Luis Obispo
>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/187c49b2/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 18 05:03:53 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 11:03:53 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
Message-ID: <7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>


Le 18 d?c. 2012 ? 10:52, Robert McGibbon a ?crit :

> Matthias,
> 
> Good call. That's it exactly. Unfortunately, I think dealing with this issue is a little above my expertise/experience.
> 
> When I apply this patch (pasted below) to the PR, the tab completions show in the notebook for foo(<TAB>.
> 
> The problem is that behavior kind of conflicts with the tooltip feature, which is not a feature of the terminal console.

without your patch, 
You could set         
this.tooltip_on_tab = true;
L25 in codecell.js to false. 
It will trigger the tooltip only if you stay after a opening bracket for a certain amount of time.
The the completer should work as expected. 

It should be made configurable. 

we could try to tweak the behavior so that (<tab> trigger tooltip
([space]<tab> trigger completion.

But I guess that's beyond the extend of your PR.
-- 
Matthias


> 
> -Robert
> 
> p.s.
> 
> I think I'm finally starting to see the wisdom of the comment on line 1046 of IPython/core/complete.py,
> 
>>         # FIXME: we should extend our api to return a dict with completions for
>>         # different types of objects.  The rlcomplete() method could then
>>         # simply collapse the dict into a list for readline, but we'd have
>>         # richer completion semantics in other evironments.
> 
> 
> $ git diff
> diff --git a/IPython/frontend/html/notebook/static/js/codecell.js b/IPython/frontend/html/notebook/static/js/codecell.js
> index ea34719..000734b 100644
> --- a/IPython/frontend/html/notebook/static/js/codecell.js
> +++ b/IPython/frontend/html/notebook/static/js/codecell.js
> @@ -155,6 +155,8 @@ var IPython = (function (IPython) {
>                  IPython.tooltip.request(that);
>                  // Prevent the event from bubbling up.
>                  event.stop();
> +                console.log('RTM DEBUG');
> +                this.completer.startCompletion();
>                  // Prevent CodeMirror from handling the tab.
>                  return true;
>              } else {
> diff --git a/IPython/frontend/html/notebook/static/js/completer.js b/IPython/frontend/html/notebook/static/js/completer.js
> index ac2d95c..94aff4b 100644
> --- a/IPython/frontend/html/notebook/static/js/completer.js
> +++ b/IPython/frontend/html/notebook/static/js/completer.js
> @@ -105,8 +105,9 @@ var IPython = (function (IPython) {
>          // we need to check that we are still on a word boundary
>          // because while typing the completer is still reinvoking itself
>          if (!/[0-9a-z._/\\:~-]/i.test(pre_cursor)) {
> -            this.close();
> -            return;
> +            //this.close();
> +            //return;
> +            console.log('RTM DEBUG2');
>          }
>  
>          this.autopick = false;
> 
> 
> On Dec 18, 2012, at 1:17 AM, Matthias Bussonnier wrote:
> 
>> Ah... Then it's probably the tooltip code that take over.
>> 
>> Same folder, codecell.js look for TAB or pre_cursor.
>> 
>> Completion is not triggerd if char in front of cursor is space or opening bracket.
>> 
>> Short from my phone.
>> 
>> Le 18 d?c. 2012 09:28, "Robert McGibbon" <rmcgibbo at gmail.com> a ?crit :
>> Marrhias
>> 
>> the baz(notevaluated).<TAB> is working in the notebook.
>> 
>> it's the foo(<TAB> that's not working... yet.
>> 
>> my fridge is not out of beer, so i'm still working :)
>> 
>> -Robert
>> 
>> On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:
>> 
>>> 
>>> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
>>> 
>>>> It's not working in the notebook with the argument specific completions, but the return value completion is working. There must be some issue with a difference in the line splitting with readline and without. I'll track it down.
>>> 
>>> That's probably the js that removes everything between bracket by nothing
>>> (IIRC, first completer was doing that.)
>>> 
>>> baz(notevaluated).<tab>
>>> to 
>>> baz.<tab>
>>> 
>>> before completion. (or is it tooltip... that does it) 
>>> 
>>> Or a security about 'as you type' 
>>> 
>>> Actually every key press recall the completer once it has been invoked and it decide to 'kill itself' 
>>> on certain things (like closing bracket, spaces...) 
>>> 
>>> That would be 
>>> 
>>> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
>>> -- 
>>> Matthias
>>> 
>>> 
>>> 
>>>> 
>>>> -Robert
>>>> 
>>>> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
>>>> 
>>>>> This looks really great.  Have you tested it with the notebook yet?  I
>>>>> would imagine that it should already work there as well.
>>>>> 
>>>>> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>>>>>> Hey,
>>>>>> 
>>>>>> I just posted a new PR for annotation based custom tab completion.
>>>>>> https://github.com/ipython/ipython/pull/2701
>>>>>> 
>>>>>> The code is fairly complex (~1000 new lines), so I assume that there are
>>>>>> bugs. For that reason (and because its fun!), I encourage people
>>>>>> to try taking it out for a spin.
>>>>>> 
>>>>>> 
>>>>>> It lets you do things like this:
>>>>>> 
>>>>>> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>>>>>> literal
>>>>>> 
>>>>>> In [2]: @tab_complete
>>>>>> ...: def baz(x) -> str:
>>>>>> ...:     pass
>>>>>> ...:
>>>>>> 
>>>>>> In [3]: baz(notevaluated).<TAB>
>>>>>> .capitalize  .find        .isspace     .partition   .rstrip      .translate
>>>>>> .center      .format      .istitle     .replace     .split       .upper
>>>>>> .count       .index       .isupper     .rfind       .splitlines  .zfill
>>>>>> .decode      .isalnum     .join        .rindex      .startswith
>>>>>> .encode      .isalpha     .ljust       .rjust       .strip
>>>>>> .endswith    .isdigit     .lower       .rpartition  .swapcase
>>>>>> .expandtabs  .islower     .lstrip      .rsplit      .title
>>>>>> 
>>>>>> 
>>>>>> And this:
>>>>>> 
>>>>>> In [4]: @tab_complete
>>>>>> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>>>>>> ...:     pass
>>>>>> ...:
>>>>>> 
>>>>>> In [5]: foo(<TAB>
>>>>>> 'COPYING.txt'        'dist/'              'setupext/'
>>>>>> 'IPython/'           'docs/'              'tools/'
>>>>>> '__pycache__/'       'ipython.egg-info/'
>>>>>> 'build/'             'scripts/'
>>>>>> 
>>>>>> In[6]: foo('COPYING.txt', <TAB>
>>>>>> 'r', 'w'
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -Robert
>>>>>> 
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> -- 
>>>>> Brian E. Granger
>>>>> Cal Poly State University, San Luis Obispo
>>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>> 
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/7a60fa25/attachment.html>

From nick.bollweg at gmail.com  Tue Dec 18 07:41:46 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Tue, 18 Dec 2012 07:41:46 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
Message-ID: <CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>

Hmm... I think with code generators, i have an inkling that a notation
should be used that itself is not in the target language to tell the
generator how to generate it... cog uses comments, so I kinda went there...
Decorators are a nice pun in python, as they aren't exactly "normal"... And
don't let you know when you are Done (but whitespace does), though I am not
sure what to use in JS... ideally, I'd rather that the resulting code
*didn't* have a hard dependency on the blockly generator extension (in
whatever language), but obviously we need some "metadata" (not
cell.metadata) in the code to remain robust to users doing interesting
things.

a pure JS solution is desirable, but right now I am bound by the
jsonhandler API, which means I need to emit a type that I control.... This
is why I've gone down the 2-cell path initially... also, until I get much
smarter (or see a really good example) there is stuff i don't know I CAN do
on the frontend (I.e. Get the current local names and types in scope...)
that I can do trivially on the backend. I can't even think of the
ASTQuery(tm) I would need to ask things about the current state of the code
to get what I need.

should have some refactored code up soon! The variables are tricky!

+1 on CM3... They are doing great work over there.
 On Dec 18, 2012 2:34 AM, "Matthias BUSSONNIER" <
bussonniermatthias at gmail.com> wrote:

> Le 18 d?c. 2012 ? 01:58, Nicholas Bollweg a ?crit :
>
>
>
> >  I was already planning to do something hash-based to make sure you
> don't inadvertently destroy stuff.
> >
> >     # blockly foo
> >     def foo(bar):
> >         baz = None
> >         return baz
> >     # endblockly foo
>
>
> It's fine to use # if the notebook is bind to python,
> but don't forget that it might not always be the only case, so we probably
> won't hardcode something like that in the javascript.
>
> Also we have to achieve to bring code mirror 3.0 into the notebook.
> I don't know how much different the API is, but we already broke some User
> habits when upgrading from 2.1 to 2.3 I think.
> --
> Matthias
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/f9435826/attachment.html>

From jason-sage at creativetrax.com  Tue Dec 18 07:52:35 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 05:52:35 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
	<CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
Message-ID: <50D06713.2090500@creativetrax.com>

On 12/18/12 5:41 AM, Nicholas Bollweg wrote:
> but obviously we need some "metadata" (not cell.metadata) in the code to
> remain robust to users doing interesting things.

What if you stored the limits of interesting character ranges in the 
cell metadata when the cell is stored?  When the cell is instantiated, 
you could mark the relevant ranges with the CodeMirror 3 markText 
function so that the range is continually updated.  When the cell is 
saved, you can update the cell metadata with the new ranges.

Thanks,

Jason



From nissimk at gmail.com  Tue Dec 18 08:02:44 2012
From: nissimk at gmail.com (Nissim Karpenstein)
Date: Tue, 18 Dec 2012 08:02:44 -0500
Subject: [IPython-dev] Notebook CodeCell for editing and executing javascript
Message-ID: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>

Hi,

I'm new here and I wanted to run this by the group before I start working
on it.  I'm interested in building a facility to use the notebook to edit
and execute javascript code for presenting data from the python kernel.  I
know there is a magic function and there is some sort of way to have python
code render js in <script> tags, and you can put javascript in the Markdown
cells, but I was envisioning something more like a standard codecell, but
execution is handled by the browser rather than the kernel.

Let's say you've loaded some data and run some computations in your python
kernel and now you want to display results in the notebook using some
javascript library like d3, datatables or processing.  If you want to do
this by writing the javascript code in a CodeCell and evaluating that code
in the browser you need a couple of things:

1) ability to change a code cell language to javascipt in the notebook
frontend.  I saw that there is a language field in the notebook file format
for CodeCell but it is always = python.

2) facility to have the browser javascript engine evaluate the contents of
the javascript CodeCell when it is executed by the user

3) http REST api that will return json representations of Python objects
that exist in the kernel.  (this may already exist, I don't really
understand the kernel communication protocol yet).

There are probably some more things like the ability to add CSS, HTML, js
and image resources to the server that it can then serve back to the
client.  I see that there's been a lot of discussion related to this in the
plugins thread, but maybe there can also be a method for accessing
resources from the network and then telling the notebook server to serve
them from some path.

I'd like to implement this and I had a couple of questions:

1) Does this fit in with your ideas for the notebook or is editing non
python code in CodeCell outside of the vision?

2) Should there be another Dropdown for the language of the CodeCell or
should there be a javascript option added to the existing select box.

Thanks

   -Nissim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/feb32149/attachment.html>

From nick.bollweg at gmail.com  Tue Dec 18 08:16:28 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Tue, 18 Dec 2012 08:16:28 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50D06713.2090500@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
	<CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
	<50D06713.2090500@creativetrax.com>
Message-ID: <CACejjWzHuVfRB4vuWXGTfJKs3UD5_LuptHOhM2=c4_JB6Obavw@mail.gmail.com>

Good brainstorming. I think as I*nb becomes more mainstream, we'll see more
code generators (or is it the other way?), so these are good things to hash
out now.

Offsets are tricky... this would make it such that editing the cell without
%load_ext blockly is almost guaranteed to break it even if you don't edit
the blockly region's content. delimiter-based stuff would at least be
robust in that fashion. Also, in the case of strongly-whitespaced (lolz)
languages, the delimiters would let you know at what col you need to be
spitting out code... I would hate to be unable to refactor some "naked"
blockly-managed code(eek! Sounds like .net) into a function and lose the
blocks.

actually, would a %blockly magic be appropriate? That might be at the right
level of "different from source", and presumably magics will eventually be
cross-language, right? Still would have the hard deep, though...
On Dec 18, 2012 7:52 AM, "Jason Grout" <jason-sage at creativetrax.com> wrote:

> On 12/18/12 5:41 AM, Nicholas Bollweg wrote:
> > but obviously we need some "metadata" (not cell.metadata) in the code to
> > remain robust to users doing interesting things.
>
> What if you stored the limits of interesting character ranges in the
> cell metadata when the cell is stored?  When the cell is instantiated,
> you could mark the relevant ranges with the CodeMirror 3 markText
> function so that the range is continually updated.  When the cell is
> saved, you can update the cell metadata with the new ranges.
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/9ddba49e/attachment.html>

From jason-sage at creativetrax.com  Tue Dec 18 08:26:40 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 06:26:40 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWzHuVfRB4vuWXGTfJKs3UD5_LuptHOhM2=c4_JB6Obavw@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
	<CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
	<50D06713.2090500@creativetrax.com>
	<CACejjWzHuVfRB4vuWXGTfJKs3UD5_LuptHOhM2=c4_JB6Obavw@mail.gmail.com>
Message-ID: <50D06F10.7010808@creativetrax.com>

On 12/18/12 6:16 AM, Nicholas Bollweg wrote:
> Good brainstorming. I think as I*nb becomes more mainstream, we'll see
> more code generators (or is it the other way?), so these are good things
> to hash out now.
>
> Offsets are tricky... this would make it such that editing the cell
> without %load_ext blockly is almost guaranteed to break it even if you
> don't edit the blockly region's content.

The point is that CodeMirror keeps track of the offsets.  When a cell is 
loaded into the editor, all marks stored in the cell metadata are set 
and then Codemirror makes sure they follow the text.  When the cell is 
saved, the offsets are updated.  This could be a general mechanism using 
cell metadata for providing marked regions in cells, so we don't have to 
have a special extension loaded to load the marks and save them.

Thanks,

Jason



From nick.bollweg at gmail.com  Tue Dec 18 08:50:01 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Tue, 18 Dec 2012 08:50:01 -0500
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
Message-ID: <CACejjWz0knKLD=YBQuwAufoxFcURhpva_fjoX_bk5vEERAS1zw@mail.gmail.com>

Welcome nissim!

sorry for no real links, on my phone!

good ideas! smart to ask the list, first, too! I failed to do so, and ended
up doing a lot of stuff that falls under a big upcoming JS/html refactor.

Check out the "custom web notebook cells" threads for the current state of
thinking. also, don't get discouraged if the answer is "not just yet"! the
core devs are interested in our ideas, and will help us understand the core
design principles.

I recommend grabbing ellisonbg/ipython at jsonhandlers and  the
ipython/jsplugins off Github for the current state of the art (d3graph is a
good learning example, and pytutor is... Crazy cool)... I have built a
proof of concept visual code editor which uses two
cells(bollwyvl/ipython at blockly), and we're trying to figure out how to make
it better, specifically with closer integration with CM3.

There is also IPEP5 on the main ipython/ipython Github wiki, which talks
about some of the larger trends in JS... I don't think first-class JS
editing is mentioned there, but as you will hear frequently, the "python"
in "ipython" is just a stand-in for "everything"! this means the web
notebook is not the only driver of the technologies... What would JS be
like in the qt notebook? Would we embed a webkit frame?

I look forward to working with you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/9db9ed31/attachment.html>

From nick.bollweg at gmail.com  Tue Dec 18 08:50:01 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Tue, 18 Dec 2012 08:50:01 -0500
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <50D06F10.7010808@creativetrax.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
	<CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
	<50D06713.2090500@creativetrax.com>
	<CACejjWzHuVfRB4vuWXGTfJKs3UD5_LuptHOhM2=c4_JB6Obavw@mail.gmail.com>
	<50D06F10.7010808@creativetrax.com>
Message-ID: <CACejjWw6QVHLPgU-sra1SWuV01HP7795QTh7XjWvQKRzQqe4Rw@mail.gmail.com>

maybe, but what about the qt console? command line? I guess I would rather
have something that was a bit more explicit and durable. of course, the cm
offsets would be totally clutch during live editing, as seeing the effects
of your actions is a great way to learn.... Blocky goes out of its way to
generate readable code.
On Dec 18, 2012 8:26 AM, "Jason Grout" <jason-sage at creativetrax.com> wrote:

> On 12/18/12 6:16 AM, Nicholas Bollweg wrote:
> > Good brainstorming. I think as I*nb becomes more mainstream, we'll see
> > more code generators (or is it the other way?), so these are good things
> > to hash out now.
> >
> > Offsets are tricky... this would make it such that editing the cell
> > without %load_ext blockly is almost guaranteed to break it even if you
> > don't edit the blockly region's content.
>
> The point is that CodeMirror keeps track of the offsets.  When a cell is
> loaded into the editor, all marks stored in the cell metadata are set
> and then Codemirror makes sure they follow the text.  When the cell is
> saved, the offsets are updated.  This could be a general mechanism using
> cell metadata for providing marked regions in cells, so we don't have to
> have a special extension loaded to load the marks and save them.
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/eca70504/attachment.html>

From jason-sage at creativetrax.com  Tue Dec 18 08:55:36 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 06:55:36 -0700
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWw6QVHLPgU-sra1SWuV01HP7795QTh7XjWvQKRzQqe4Rw@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
	<CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
	<50D06713.2090500@creativetrax.com>
	<CACejjWzHuVfRB4vuWXGTfJKs3UD5_LuptHOhM2=c4_JB6Obavw@mail.gmail.com>
	<50D06F10.7010808@creativetrax.com>
	<CACejjWw6QVHLPgU-sra1SWuV01HP7795QTh7XjWvQKRzQqe4Rw@mail.gmail.com>
Message-ID: <50D075D8.8000608@creativetrax.com>

On 12/18/12 6:50 AM, Nicholas Bollweg wrote:
> maybe, but what about the qt console? command line? I guess I would
> rather have something that was a bit more explicit and durable. of
> course, the cm offsets would be totally clutch during live editing, as
> seeing the effects of your actions is a great way to learn.... Blocky
> goes out of its way to generate readable code.

It would be up to the editor to maintain the character offsets in a way 
that makes sense.  If an editor doesn't have a way of easily maintaining 
them, the editor could always literally insert some obscure unicode 
character in the code to act as delimiters, or even just insert your 
comments.  But that would be less than ideal, I think.

Thanks,

Jason



From takowl at gmail.com  Tue Dec 18 09:54:25 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 18 Dec 2012 14:54:25 +0000
Subject: [IPython-dev] IPython nosetests extension
In-Reply-To: <79F9959B-4ACC-46F9-B48A-E7781DEE669B@gmail.com>
References: <CAKrk_d_081PjEcCqiFTUQPM=uLQ+f3aOF=GWQhPdczScnmhTGw@mail.gmail.com>
	<CAH4pYpQRBXCNuyndEcZosmrG-4jgXUK4bU+MFK0apNEV_2uhug@mail.gmail.com>
	<CAKrk_d8wL93isdsZQg1ENfatESncmZina2=wEn1b=xbdVXgyYw@mail.gmail.com>
	<CAH4pYpQZCvHJe4rA-v6gAuEFBF_R2wjh7Sd3vdMe0HXhYR8Dzw@mail.gmail.com>
	<CAKrk_d9+x=Y6_ODeV+xxqAXTAG-0dazV2jkaqubdmmy6SX9ESg@mail.gmail.com>
	<79F9959B-4ACC-46F9-B48A-E7781DEE669B@gmail.com>
Message-ID: <CAOvn4qh5Tkv9c0QF-oMgqpQVJZ6zfAc0sDDe2BX4WMPfzpAfxQ@mail.gmail.com>

On 18 December 2012 08:13, Matthias BUSSONNIER <bussonniermatthias at gmail.com
> wrote:

> Haven't read the all thread, but, why not having a object that have
> both an _repr_ and and _repr_html_ (supported by qtconsole)
>

It would certainly be good to build tracebacks as objects using the display
protocol. ultratb deeply embeds the formatting using terminal control
codes. I think it's some of the oldest code still in IPython - but hats off
to it, it still works.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/e8dfa7ae/attachment.html>

From takowl at gmail.com  Tue Dec 18 10:38:42 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 18 Dec 2012 15:38:42 +0000
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
Message-ID: <CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>

On 17 December 2012 20:00, Brian Granger <ellisonbg at gmail.com> wrote:

> * How do we manage communication?  Verbal communication is much more
> efficient than emails or even IRC.  The 4 people at Berkeley will have
> an incredible advantage in being able to talk daily.  We don't want to
> cripple or remove that advantage, but we need to figure out how to
> include other core devs and people from the community.  This is
> particularly relevant to myself as I am the only person involved in
> the Sloan work that is not at Berkeley.
>

I think this issue - communication - is becoming key. I spent a day away
from my computer, and came back to >40 new e-mails in my IPython folder (in
addition to the 20 odd unread that I'm planning to get round to one day).
That's a mixture of our two mailing lists and the Github notification
messages. I get the feeling we're approaching a critical point, where it's
no longer possible for us as individuals to keep up with all the
discussions going on.

To my mind, we need to split things up. We already have an informal
separation of interests - for instance, I leave most of the discussion
about the notebook front-end to others, but get more involved with
IPython.core. I think we need to make this a bit (not too much) more
formal, so that no-one falls down the cracks as everything speeds up.

This could mean, e.g. more specialised mailing lists, or having a
consistent process for handling incoming issues.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/bf6f3ae9/attachment.html>

From jason-sage at creativetrax.com  Tue Dec 18 11:03:39 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 09:03:39 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
Message-ID: <50D093DB.50400@creativetrax.com>

On 12/18/12 8:38 AM, Thomas Kluyver wrote:
> On 17 December 2012 20:00, Brian Granger <ellisonbg at gmail.com
> <mailto:ellisonbg at gmail.com>> wrote:
>
>     * How do we manage communication?  Verbal communication is much more
>     efficient than emails or even IRC.  The 4 people at Berkeley will have
>     an incredible advantage in being able to talk daily.  We don't want to
>     cripple or remove that advantage, but we need to figure out how to
>     include other core devs and people from the community.  This is
>     particularly relevant to myself as I am the only person involved in
>     the Sloan work that is not at Berkeley.
>
>
> I think this issue - communication - is becoming key. I spent a day away
> from my computer, and came back to >40 new e-mails in my IPython folder
> (in addition to the 20 odd unread that I'm planning to get round to one
> day). That's a mixture of our two mailing lists and the Github
> notification messages. I get the feeling we're approaching a critical
> point, where it's no longer possible for us as individuals to keep up
> with all the discussions going on.
>
> To my mind, we need to split things up. We already have an informal
> separation of interests - for instance, I leave most of the discussion
> about the notebook front-end to others, but get more involved with
> IPython.core. I think we need to make this a bit (not too much) more
> formal, so that no-one falls down the cracks as everything speeds up.
>
> This could mean, e.g. more specialised mailing lists, or having a
> consistent process for handling incoming issues.

If Github had a better issue tracker, it'd be much easier.  For example, 
if you could get notifications for certain labels, and if random people 
could add labels (or only add certain types of labels), things would be 
much better.

I'd suggest not splitting up the project mailing lists, but rather 
ignoring emails that appear to touch things you aren't interested in. 
Just at the critical time, you don't want to fragment the (still small) 
community.

Thanks,

Jason



From jason-sage at creativetrax.com  Tue Dec 18 11:08:39 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 09:08:39 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <50D093DB.50400@creativetrax.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
	<50D093DB.50400@creativetrax.com>
Message-ID: <50D09507.9020708@creativetrax.com>

On 12/18/12 9:03 AM, Jason Grout wrote:
> For example,
> if you could get notifications for certain labels,

This part I guess you could do with the API and a 3rd party service.

However, there still remains the issue that I, as a user, can't classify 
my own issues as notebook related, core related, etc.

Thanks,

Jason



From takowl at gmail.com  Tue Dec 18 11:12:03 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 18 Dec 2012 16:12:03 +0000
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <50D093DB.50400@creativetrax.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
	<50D093DB.50400@creativetrax.com>
Message-ID: <CAOvn4qhNTPdkMSt-k6yU10NLMHW4h6oALdTANLU_bBFifFu_LA@mail.gmail.com>

On 18 December 2012 16:03, Jason Grout <jason-sage at creativetrax.com> wrote:

> If Github had a better issue tracker, it'd be much easier.  For example,
> if you could get notifications for certain labels, and if random people
> could add labels (or only add certain types of labels), things would be
> much better.
>

Perhaps, but I think we could also make better use of the tools that are
there on Github. For instance, we could try to assign all bugs as they come
in. You don't have to personally resolve issues assigned to you, but you're
the point of contact until it's resolved or reassigned.

It would certainly be nice if non-core developers - especially bug
submitters - could do labelling. Maybe we should lobby Github to make this
possible. They've certainly listened to users in the past.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/369010a0/attachment.html>

From jason-sage at creativetrax.com  Tue Dec 18 11:28:54 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 18 Dec 2012 09:28:54 -0700
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAOvn4qhNTPdkMSt-k6yU10NLMHW4h6oALdTANLU_bBFifFu_LA@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
	<50D093DB.50400@creativetrax.com>
	<CAOvn4qhNTPdkMSt-k6yU10NLMHW4h6oALdTANLU_bBFifFu_LA@mail.gmail.com>
Message-ID: <50D099C6.5000108@creativetrax.com>

On 12/18/12 9:12 AM, Thomas Kluyver wrote:
> On 18 December 2012 16:03, Jason Grout <jason-sage at creativetrax.com
> <mailto:jason-sage at creativetrax.com>> wrote:
>
>     If Github had a better issue tracker, it'd be much easier.  For example,
>     if you could get notifications for certain labels, and if random people
>     could add labels (or only add certain types of labels), things would be
>     much better.
>
>
> Perhaps, but I think we could also make better use of the tools that are
> there on Github. For instance, we could try to assign all bugs as they
> come in. You don't have to personally resolve issues assigned to you,
> but you're the point of contact until it's resolved or reassigned.
>
> It would certainly be nice if non-core developers - especially bug
> submitters - could do labelling. Maybe we should lobby Github to make
> this possible. They've certainly listened to users in the past.

Fernando has brought this up in the past, but in addition, if there were 
any way to have key-value labels like Google Code, plus filters:

http://code.google.com/p/support/wiki/IssueTracker#Filter_rules

that would go a long way to making the issue tracker more usable.  To be 
fair, filters could be done with service hooks.

Thanks,

Jason




From d.warde.farley at gmail.com  Tue Dec 18 12:02:11 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Tue, 18 Dec 2012 12:02:11 -0500
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
Message-ID: <CALc6Xo7MoC7j-4AyVT7VZ7qzpp2fp_M=tu40+=Wf7OAOyHmR5A@mail.gmail.com>

On Mon, Dec 17, 2012 at 7:02 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> I know it probably sounds heretical, but I think that part of the
> solution has to be to reduce the number of issues.  I don't mean by
> closing issues by working harder and coding more, I mean simply
> *closing* them outright to reflect the reality that we are not going
> to work on them anytime soon.  There is no universe where it is useful
> to have thousands of issues.  How many can each developer
> realistically keep track of?  A few dozen?

Would additionally adding a "Someday" (slightly more diplomatic than
"When Hell Freezes Over") label help track those that were closed not
due to invalidity but due to lack of bandwidth?

Cheers,

David


From bussonniermatthias at gmail.com  Tue Dec 18 12:40:03 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 18:40:03 +0100
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
Message-ID: <C9265E72-6169-4A4A-9BC3-7901B3456621@gmail.com>

Hi, 

Le 18 d?c. 2012 ? 14:02, Nissim Karpenstein a ?crit :

> Hi,
> 
> I'm new here and I wanted to run this by the group before I start working on it.  I'm interested in building a facility to use the notebook to edit and execute javascript code for presenting data from the python kernel.  I know there is a magic function and there is some sort of way to have python code render js in <script> tags, and you can put javascript in the Markdown cells, but I was envisioning something more like a standard codecell, but execution is handled by the browser rather than the kernel.
> 
> Let's say you've loaded some data and run some computations in your python kernel and now you want to display results in the notebook using some javascript library like d3, datatables or processing.  If you want to do this by writing the javascript code in a CodeCell and evaluating that code in the browser you need a couple of things:
> 1) ability to change a code cell language to javascipt in the notebook frontend.  I saw that there is a language field in the notebook file format for CodeCell but it is always = python.
> 
> 2) facility to have the browser javascript engine evaluate the contents of the javascript CodeCell when it is executed by the user
> 
> 3) http REST api that will return json representations of Python objects that exist in the kernel.  (this may already exist, I don't really understand the kernel communication protocol yet).
> 
> There are probably some more things like the ability to add CSS, HTML, js and image resources to the server that it can then serve back to the client.  I see that there's been a lot of discussion related to this in the plugins thread, but maybe there can also be a method for accessing resources from the network and then telling the notebook server to serve them from some path.
> I'd like to implement this and I had a couple of questions:
> 
> 1) Does this fit in with your ideas for the notebook or is editing non python code in CodeCell outside of the vision?
> 
> 2) Should there be another Dropdown for the language of the CodeCell or should there be a javascript option added to the existing select box.
> 
> Thanks
> 
>    -Nissim
> 

Hi, 

My answer will be short, but there are lot more too say here, 
I absolutely don't want to discourage you here, I'm sure there are way to do what you want to seamlessly 
for the user.

No, a direct javascript cell is not envisageable for security reason, and we will soon block js in markdown.
It will be optional but js will be on purpose **painfully** and  opt-in.

But, that what the json handler pull request is made for.
There is a prototype on d3 graph visualization using json handler.

We already have prototype of node/ruby kernel so you might be able to do js/ruby kernel side if needed.

I don't think we have decide yet of multi-language notebook on a per-cell basis without magics.
I wouldn't recommend it as it forces user to think of the kernel where the variable are defined.

But language on a per-notebook basis is totally conceivable, we just need devs from other language...

There are also issues where kernel and server might be on different machines, which we should keep in mind so avoid server as much as possible.

I know there are plan on making ability in browser to run JS in a **safe** sandbox, which are not really ready yet I think, 
when this comes to maturity the ability to execute js in browser, might be revisited, still i might not have access to the DOM directly.
-- 
Matthias


> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/d6d48730/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 18 12:44:09 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 18:44:09 +0100
Subject: [IPython-dev] custom web notebook cells
In-Reply-To: <CACejjWw6QVHLPgU-sra1SWuV01HP7795QTh7XjWvQKRzQqe4Rw@mail.gmail.com>
References: <CACejjWwVsaAquU4X0ORW7rhrwF2Jq50iOvx8t-yoBfNjRmNwbA@mail.gmail.com>
	<CAH4pYpT8ise-2iLSQ6NcQ+grC9+H83G_pysKhp7oEo6Nue0hvw@mail.gmail.com>
	<50CCE9A2.8000905@creativetrax.com>
	<50CCF809.4090808@creativetrax.com>
	<50CFA219.4020703@creativetrax.com>
	<CACejjWwGk0B07n-EQC-u6psZAUKHi++cq6Xc-fE0FtC9YhZX5g@mail.gmail.com>
	<5F351738-9220-4C00-8B2C-1BAF2C1A6AF8@gmail.com>
	<CACejjWzQAQSmEvU54+q=7W_+BEMG=0O5X-arLWAT8Qcf2NTdNA@mail.gmail.com>
	<50D06713.2090500@creativetrax.com>
	<CACejjWzHuVfRB4vuWXGTfJKs3UD5_LuptHOhM2=c4_JB6Obavw@mail.gmail.com>
	<50D06F10.7010808@creativetrax.com>
	<CACejjWw6QVHLPgU-sra1SWuV01HP7795QTh7XjWvQKRzQqe4Rw@mail.gmail.com>
Message-ID: <939930DC-4920-4AC7-9C52-FC6407CBA867@gmail.com>


Le 18 d?c. 2012 ? 14:50, Nicholas Bollweg a ?crit :

> maybe, but what about the qt console? command line? I guess I would rather have something that was a bit more explicit and durable. of course, the cm offsets would be totally clutch during live editing, as seeing the effects of your actions is a great way to learn.... Blocky goes out of its way to generate readable code.
> 
The argument still apply for command line, but I had a prototype of webkit console, that is to say, looks like qtconsole, 
but edit in code mirror, Mathjax, iframes? etc. 

Qt webkit is painfully slow, never had the time to look at chromium embedded. 
-- 
M





> On Dec 18, 2012 8:26 AM, "Jason Grout" <jason-sage at creativetrax.com> wrote:
> On 12/18/12 6:16 AM, Nicholas Bollweg wrote:
> > Good brainstorming. I think as I*nb becomes more mainstream, we'll see
> > more code generators (or is it the other way?), so these are good things
> > to hash out now.
> >
> > Offsets are tricky... this would make it such that editing the cell
> > without %load_ext blockly is almost guaranteed to break it even if you
> > don't edit the blockly region's content.
> 
> The point is that CodeMirror keeps track of the offsets.  When a cell is
> loaded into the editor, all marks stored in the cell metadata are set
> and then Codemirror makes sure they follow the text.  When the cell is
> saved, the offsets are updated.  This could be a general mechanism using
> cell metadata for providing marked regions in cells, so we don't have to
> have a special extension loaded to load the marks and save them.
> 
> Thanks,
> 
> Jason
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/8c0fd71c/attachment.html>

From d.warde.farley at gmail.com  Tue Dec 18 13:26:33 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Tue, 18 Dec 2012 13:26:33 -0500
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
Message-ID: <CALc6Xo4DU2oZFSufk-rV1eS+KnKeUOfaLwv0AWYSKqOTe6c4yQ@mail.gmail.com>

On Mon, Dec 17, 2012 at 5:36 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
> I remember reading about a study that said that open source projects
> that have funded developers actually get less contributions (or at
> least fewer contributors) because there is a mindset of, "why should I
> contribute when there is someone who is paid to do it? Surely that
> person/persons will get around to fixing the issue themselves."  I
> think the email Matthias received is indicative of this mindset.
> Clearly it is wrong (obviously, more contributions are still more
> contributions), but I think you should really think about how to make
> this permeate through your community, so that you minimize the number
> of people who end up thinking this way.

One thing to emphasize (and I know I'm preaching to the choir here,
but this should be emphasized to *other* people) is that, as much
faith as I have in the core developers, they are neither omniscient
nor psychic. Participation in the form of bug reports and feature
requests is therefore valuable, but the fastest path to a new feature
will always be to take a stab at implementing it yourself. Even if
your implementation is hacky, imperfect, or incomplete, it gets the
conversation started in a much more tangible way. It also reduces the
bandwidth commitment of a core developer, somewhat, because they no
longer have to figure out exactly what you mean based on a mere
description of it.

Cheers,

David


From bussonniermatthias at gmail.com  Tue Dec 18 13:40:59 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 19:40:59 +0100
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
Message-ID: <AA51BA3B-B987-42F5-9B67-105AA44A9105@gmail.com>

Hi again, 

I'll moderate what I said an hour ago, 
I might see one way to maybe do something close to what you wish, 
It might not be strait forward, and I will thing about it. 
It might be incorporated in the JS widget we planed on implementing, I'll discusses of that with Brian
to better now what his plans are.

In any way it will probably change the notebook format introducing incompatibilities with older one, so 
we will think about it a lot before releasing it.

We would still be happy to see what a prototype does, maybe you'll find how to do that in a way that
prevent javascript injection at load time when exchanging ipynb files. 
-- 
Matthias

Le 18 d?c. 2012 ? 14:02, Nissim Karpenstein a ?crit :

> Hi,
> 
> I'm new here and I wanted to run this by the group before I start working on it.  I'm interested in building a facility to use the notebook to edit and execute javascript code for presenting data from the python kernel.  I know there is a magic function and there is some sort of way to have python code render js in <script> tags, and you can put javascript in the Markdown cells, but I was envisioning something more like a standard codecell, but execution is handled by the browser rather than the kernel.
> 
> Let's say you've loaded some data and run some computations in your python kernel and now you want to display results in the notebook using some javascript library like d3, datatables or processing.  If you want to do this by writing the javascript code in a CodeCell and evaluating that code in the browser you need a couple of things:
> 1) ability to change a code cell language to javascipt in the notebook frontend.  I saw that there is a language field in the notebook file format for CodeCell but it is always = python.
> 
> 2) facility to have the browser javascript engine evaluate the contents of the javascript CodeCell when it is executed by the user
> 
> 3) http REST api that will return json representations of Python objects that exist in the kernel.  (this may already exist, I don't really understand the kernel communication protocol yet).
> 
> There are probably some more things like the ability to add CSS, HTML, js and image resources to the server that it can then serve back to the client.  I see that there's been a lot of discussion related to this in the plugins thread, but maybe there can also be a method for accessing resources from the network and then telling the notebook server to serve them from some path.
> I'd like to implement this and I had a couple of questions:
> 
> 1) Does this fit in with your ideas for the notebook or is editing non python code in CodeCell outside of the vision?
> 
> 2) Should there be another Dropdown for the language of the CodeCell or should there be a javascript option added to the existing select box.
> 
> Thanks
> 
>    -Nissim
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From nissimk at gmail.com  Tue Dec 18 13:57:06 2012
From: nissimk at gmail.com (Nissim Karpenstein)
Date: Tue, 18 Dec 2012 13:57:06 -0500
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <AA51BA3B-B987-42F5-9B67-105AA44A9105@gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
	<AA51BA3B-B987-42F5-9B67-105AA44A9105@gmail.com>
Message-ID: <CAFhnF0BtD+yyj3ck4RYevS70cO6np6SfV+hrytVWYJeRAmJjRg@mail.gmail.com>

Can you please elaborate further on the potential security risks.
I thought that code cells are not executed when opening an ipynb file.  It
would have to be a little different than python CodeCells because the
output of javascript cells could not really be stored in the ipynb, but we
could still require each codecell to be manually executed by the user
before being eval'd in the browser.

Have you seen jsFiddle?  My suggestion is sort of a mashup of iPython
notebook and jsFiddle.







On Tue, Dec 18, 2012 at 1:40 PM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> Hi again,
>
> I'll moderate what I said an hour ago,
> I might see one way to maybe do something close to what you wish,
> It might not be strait forward, and I will thing about it.
> It might be incorporated in the JS widget we planed on implementing, I'll
> discusses of that with Brian
> to better now what his plans are.
>
> In any way it will probably change the notebook format introducing
> incompatibilities with older one, so
> we will think about it a lot before releasing it.
>
> We would still be happy to see what a prototype does, maybe you'll find
> how to do that in a way that
> prevent javascript injection at load time when exchanging ipynb files.
> --
> Matthias
>
> Le 18 d?c. 2012 ? 14:02, Nissim Karpenstein a ?crit :
>
> > Hi,
> >
> > I'm new here and I wanted to run this by the group before I start
> working on it.  I'm interested in building a facility to use the notebook
> to edit and execute javascript code for presenting data from the python
> kernel.  I know there is a magic function and there is some sort of way to
> have python code render js in <script> tags, and you can put javascript in
> the Markdown cells, but I was envisioning something more like a standard
> codecell, but execution is handled by the browser rather than the kernel.
> >
> > Let's say you've loaded some data and run some computations in your
> python kernel and now you want to display results in the notebook using
> some javascript library like d3, datatables or processing.  If you want to
> do this by writing the javascript code in a CodeCell and evaluating that
> code in the browser you need a couple of things:
> > 1) ability to change a code cell language to javascipt in the notebook
> frontend.  I saw that there is a language field in the notebook file format
> for CodeCell but it is always = python.
> >
> > 2) facility to have the browser javascript engine evaluate the contents
> of the javascript CodeCell when it is executed by the user
> >
> > 3) http REST api that will return json representations of Python objects
> that exist in the kernel.  (this may already exist, I don't really
> understand the kernel communication protocol yet).
> >
> > There are probably some more things like the ability to add CSS, HTML,
> js and image resources to the server that it can then serve back to the
> client.  I see that there's been a lot of discussion related to this in the
> plugins thread, but maybe there can also be a method for accessing
> resources from the network and then telling the notebook server to serve
> them from some path.
> > I'd like to implement this and I had a couple of questions:
> >
> > 1) Does this fit in with your ideas for the notebook or is editing non
> python code in CodeCell outside of the vision?
> >
> > 2) Should there be another Dropdown for the language of the CodeCell or
> should there be a javascript option added to the existing select box.
> >
> > Thanks
> >
> >    -Nissim
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/903a9df2/attachment.html>

From benjaminrk at gmail.com  Tue Dec 18 14:02:11 2012
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 18 Dec 2012 11:02:11 -0800
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CALc6Xo7MoC7j-4AyVT7VZ7qzpp2fp_M=tu40+=Wf7OAOyHmR5A@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAKgW=6KH84=Rjc+w9GUWHM9KUT7VZjLtwFetnADU66vfKbVTKQ@mail.gmail.com>
	<CAH4pYpT9yHTNYBgpn3PrvmQQ3irjZme72sM602o2aoAM26a0Mw@mail.gmail.com>
	<CALc6Xo7MoC7j-4AyVT7VZ7qzpp2fp_M=tu40+=Wf7OAOyHmR5A@mail.gmail.com>
Message-ID: <CAHNn8BXFunMyOT9HoYnM+nfUqd73S6UABZwxht_A8uyxHgULTg@mail.gmail.com>

On Tue, Dec 18, 2012 at 9:02 AM, David Warde-Farley <
d.warde.farley at gmail.com> wrote:

> Would additionally adding a "Someday" (slightly more diplomatic than
> "When Hell Freezes Over") label help track those that were closed not
> due to invalidity but due to lack of bandwidth?
>

We've been using the 'wishlist' milestone for exactly this - issues we have
no immediate intention of addressing (previously, we used the
status-dormant label).
To me, there's no practical difference between open issues appropriately
labeled and closed ones that haven't actually been addressed.

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/5484d5a0/attachment.html>

From bussonniermatthias at gmail.com  Tue Dec 18 15:15:34 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 18 Dec 2012 21:15:34 +0100
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0BtD+yyj3ck4RYevS70cO6np6SfV+hrytVWYJeRAmJjRg@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
	<AA51BA3B-B987-42F5-9B67-105AA44A9105@gmail.com>
	<CAFhnF0BtD+yyj3ck4RYevS70cO6np6SfV+hrytVWYJeRAmJjRg@mail.gmail.com>
Message-ID: <4B5934CF-E0B7-4A35-849D-BFBFB42E2708@gmail.com>


Le 18 d?c. 2012 ? 19:57, Nissim Karpenstein a ?crit :

> Can you please elaborate further on the potential security risks.
> I thought that code cells are not executed when opening an ipynb file.  It would have to be a little different than python CodeCells because the output of javascript cells could not really be stored in the ipynb, but we could still require each codecell to be manually executed by the user before being eval'd in the browser.

No code cell are no executed, put if the output contain a <script> tag...

To protect that, we'll probably prevent JS in output by stripping it. 

Obviously a Js cell would be able to go around that by manipulating the notebook structure itself. 
and where a server side kernel can "easily" be isolated, it is much harder to do on browser side.

In a context where user might be logged in to a service that provide an IPython notebook, and other service, 
this is dangerous.

I'm not an expert on Javscript and security, but we warned not so long ago someone who was using IPython 
notebook that they had a security breach after adding a tiny feature that seemed irrelevant. So I'm a little cautious 
when it deals with javascript in browser now. 

This is the reason why we are moving toward json plugin. The server owner could chose that some services 
are available to user. You could of course build a service that allows to execute custom Javascipt.

I totally agree that user should be responsible for what thy execute, but if we build that we totally need parental control 
for that. 

I still think that having the ability to execute js in notebook is something really worth considering at least as an advance feature 
or a plugin. 

-- 
Matthias


> Have you seen jsFiddle?  My suggestion is sort of a mashup of iPython notebook and jsFiddle.
> 
> 
> 
> 
> 
> 
> 
> On Tue, Dec 18, 2012 at 1:40 PM, Matthias BUSSONNIER <bussonniermatthias at gmail.com> wrote:
> Hi again,
> 
> I'll moderate what I said an hour ago,
> I might see one way to maybe do something close to what you wish,
> It might not be strait forward, and I will thing about it.
> It might be incorporated in the JS widget we planed on implementing, I'll discusses of that with Brian
> to better now what his plans are.
> 
> In any way it will probably change the notebook format introducing incompatibilities with older one, so
> we will think about it a lot before releasing it.
> 
> We would still be happy to see what a prototype does, maybe you'll find how to do that in a way that
> prevent javascript injection at load time when exchanging ipynb files.
> --
> Matthias
> 
> Le 18 d?c. 2012 ? 14:02, Nissim Karpenstein a ?crit :
> 
> > Hi,
> >
> > I'm new here and I wanted to run this by the group before I start working on it.  I'm interested in building a facility to use the notebook to edit and execute javascript code for presenting data from the python kernel.  I know there is a magic function and there is some sort of way to have python code render js in <script> tags, and you can put javascript in the Markdown cells, but I was envisioning something more like a standard codecell, but execution is handled by the browser rather than the kernel.
> >
> > Let's say you've loaded some data and run some computations in your python kernel and now you want to display results in the notebook using some javascript library like d3, datatables or processing.  If you want to do this by writing the javascript code in a CodeCell and evaluating that code in the browser you need a couple of things:
> > 1) ability to change a code cell language to javascipt in the notebook frontend.  I saw that there is a language field in the notebook file format for CodeCell but it is always = python.
> >
> > 2) facility to have the browser javascript engine evaluate the contents of the javascript CodeCell when it is executed by the user
> >
> > 3) http REST api that will return json representations of Python objects that exist in the kernel.  (this may already exist, I don't really understand the kernel communication protocol yet).
> >
> > There are probably some more things like the ability to add CSS, HTML, js and image resources to the server that it can then serve back to the client.  I see that there's been a lot of discussion related to this in the plugins thread, but maybe there can also be a method for accessing resources from the network and then telling the notebook server to serve them from some path.
> > I'd like to implement this and I had a couple of questions:
> >
> > 1) Does this fit in with your ideas for the notebook or is editing non python code in CodeCell outside of the vision?
> >
> > 2) Should there be another Dropdown for the language of the CodeCell or should there be a javascript option added to the existing select box.
> >
> > Thanks
> >
> >    -Nissim
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From ellisonbg at gmail.com  Tue Dec 18 18:01:43 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 18 Dec 2012 15:01:43 -0800
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
Message-ID: <CAH4pYpSzYXgstA8W3f4a7TMWE-U2aHjMkKb40YByG5cpY6Mhxg@mail.gmail.com>

Hi,

I am going to be travelling for a day, but will reply soon.

Brian

On Tue, Dec 18, 2012 at 5:02 AM, Nissim Karpenstein <nissimk at gmail.com> wrote:
> Hi,
>
> I'm new here and I wanted to run this by the group before I start working on
> it.  I'm interested in building a facility to use the notebook to edit and
> execute javascript code for presenting data from the python kernel.  I know
> there is a magic function and there is some sort of way to have python code
> render js in <script> tags, and you can put javascript in the Markdown
> cells, but I was envisioning something more like a standard codecell, but
> execution is handled by the browser rather than the kernel.
>
> Let's say you've loaded some data and run some computations in your python
> kernel and now you want to display results in the notebook using some
> javascript library like d3, datatables or processing.  If you want to do
> this by writing the javascript code in a CodeCell and evaluating that code
> in the browser you need a couple of things:
>
> 1) ability to change a code cell language to javascipt in the notebook
> frontend.  I saw that there is a language field in the notebook file format
> for CodeCell but it is always = python.
>
> 2) facility to have the browser javascript engine evaluate the contents of
> the javascript CodeCell when it is executed by the user
>
> 3) http REST api that will return json representations of Python objects
> that exist in the kernel.  (this may already exist, I don't really
> understand the kernel communication protocol yet).
>
> There are probably some more things like the ability to add CSS, HTML, js
> and image resources to the server that it can then serve back to the client.
> I see that there's been a lot of discussion related to this in the plugins
> thread, but maybe there can also be a method for accessing resources from
> the network and then telling the notebook server to serve them from some
> path.
>
> I'd like to implement this and I had a couple of questions:
>
> 1) Does this fit in with your ideas for the notebook or is editing non
> python code in CodeCell outside of the vision?
>
> 2) Should there be another Dropdown for the language of the CodeCell or
> should there be a javascript option added to the existing select box.
>
> Thanks
>
>    -Nissim
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Dec 18 18:05:22 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 18 Dec 2012 15:05:22 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
Message-ID: <CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>

I think we want both the tab completion and tooltip to work at the
same time = not be mutually exclusive.  I think we should preserve TAB
for the completion logic and move tooltip to something else to be
consistent.  This new tab completion logic is really important.  Let's
brain storm about ways of allowing both to coexist.

On Tue, Dec 18, 2012 at 2:03 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 18 d?c. 2012 ? 10:52, Robert McGibbon a ?crit :
>
> Matthias,
>
> Good call. That's it exactly. Unfortunately, I think dealing with this issue
> is a little above my expertise/experience.
>
> When I apply this patch (pasted below) to the PR, the tab completions show
> in the notebook for foo(<TAB>.
>
> The problem is that behavior kind of conflicts with the tooltip feature,
> which is not a feature of the terminal console.
>
>
> without your patch,
> You could set
> this.tooltip_on_tab = true;
> L25 in codecell.js to false.
> It will trigger the tooltip only if you stay after a opening bracket for a
> certain amount of time.
> The the completer should work as expected.
>
> It should be made configurable.
>
> we could try to tweak the behavior so that (<tab> trigger tooltip
> ([space]<tab> trigger completion.
>
> But I guess that's beyond the extend of your PR.
> --
> Matthias
>
>
>
> -Robert
>
> p.s.
>
> I think I'm finally starting to see the wisdom of the comment on line 1046
> of IPython/core/complete.py,
>
>         # FIXME: we should extend our api to return a dict with completions
> for
>         # different types of objects.  The rlcomplete() method could then
>         # simply collapse the dict into a list for readline, but we'd have
>         # richer completion semantics in other evironments.
>
>
> $ git diff
> diff --git a/IPython/frontend/html/notebook/static/js/codecell.js
> b/IPython/frontend/html/notebook/static/js/codecell.js
> index ea34719..000734b 100644
> --- a/IPython/frontend/html/notebook/static/js/codecell.js
> +++ b/IPython/frontend/html/notebook/static/js/codecell.js
> @@ -155,6 +155,8 @@ var IPython = (function (IPython) {
>                  IPython.tooltip.request(that);
>                  // Prevent the event from bubbling up.
>                  event.stop();
> +                console.log('RTM DEBUG');
> +                this.completer.startCompletion();
>                  // Prevent CodeMirror from handling the tab.
>                  return true;
>              } else {
> diff --git a/IPython/frontend/html/notebook/static/js/completer.js
> b/IPython/frontend/html/notebook/static/js/completer.js
> index ac2d95c..94aff4b 100644
> --- a/IPython/frontend/html/notebook/static/js/completer.js
> +++ b/IPython/frontend/html/notebook/static/js/completer.js
> @@ -105,8 +105,9 @@ var IPython = (function (IPython) {
>          // we need to check that we are still on a word boundary
>          // because while typing the completer is still reinvoking itself
>          if (!/[0-9a-z._/\\:~-]/i.test(pre_cursor)) {
> -            this.close();
> -            return;
> +            //this.close();
> +            //return;
> +            console.log('RTM DEBUG2');
>          }
>
>          this.autopick = false;
>
>
> On Dec 18, 2012, at 1:17 AM, Matthias Bussonnier wrote:
>
> Ah... Then it's probably the tooltip code that take over.
>
> Same folder, codecell.js look for TAB or pre_cursor.
>
> Completion is not triggerd if char in front of cursor is space or opening
> bracket.
>
> Short from my phone.
>
> Le 18 d?c. 2012 09:28, "Robert McGibbon" <rmcgibbo at gmail.com> a ?crit :
>>
>> Marrhias
>>
>> the baz(notevaluated).<TAB> is working in the notebook.
>>
>> it's the foo(<TAB> that's not working... yet.
>>
>> my fridge is not out of beer, so i'm still working :)
>>
>> -Robert
>>
>> On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:
>>
>>
>> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
>>
>> It's not working in the notebook with the argument specific completions,
>> but the return value completion is working. There must be some issue with a
>> difference in the line splitting with readline and without. I'll track it
>> down.
>>
>>
>> That's probably the js that removes everything between bracket by nothing
>> (IIRC, first completer was doing that.)
>>
>> baz(notevaluated).<tab>
>> to
>> baz.<tab>
>>
>> before completion. (or is it tooltip... that does it)
>>
>> Or a security about 'as you type'
>>
>> Actually every key press recall the completer once it has been invoked and
>> it decide to 'kill itself'
>> on certain things (like closing bracket, spaces...)
>>
>> That would be
>>
>> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
>> --
>> Matthias
>>
>>
>>
>>
>> -Robert
>>
>>
>> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
>>
>>
>> This looks really great.  Have you tested it with the notebook yet?  I
>>
>> would imagine that it should already work there as well.
>>
>>
>> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com>
>> wrote:
>>
>> Hey,
>>
>>
>> I just posted a new PR for annotation based custom tab completion.
>>
>> https://github.com/ipython/ipython/pull/2701
>>
>>
>> The code is fairly complex (~1000 new lines), so I assume that there are
>>
>> bugs. For that reason (and because its fun!), I encourage people
>>
>> to try taking it out for a spin.
>>
>>
>>
>> It lets you do things like this:
>>
>>
>> In [1]: from IPython.extensions.completion import tab_complete, globs_to,
>>
>> literal
>>
>>
>> In [2]: @tab_complete
>>
>> ...: def baz(x) -> str:
>>
>> ...:     pass
>>
>> ...:
>>
>>
>> In [3]: baz(notevaluated).<TAB>
>>
>> .capitalize  .find        .isspace     .partition   .rstrip
>> .translate
>>
>> .center      .format      .istitle     .replace     .split       .upper
>>
>> .count       .index       .isupper     .rfind       .splitlines  .zfill
>>
>> .decode      .isalnum     .join        .rindex      .startswith
>>
>> .encode      .isalpha     .ljust       .rjust       .strip
>>
>> .endswith    .isdigit     .lower       .rpartition  .swapcase
>>
>> .expandtabs  .islower     .lstrip      .rsplit      .title
>>
>>
>>
>> And this:
>>
>>
>> In [4]: @tab_complete
>>
>> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>>
>> ...:     pass
>>
>> ...:
>>
>>
>> In [5]: foo(<TAB>
>>
>> 'COPYING.txt'        'dist/'              'setupext/'
>>
>> 'IPython/'           'docs/'              'tools/'
>>
>> '__pycache__/'       'ipython.egg-info/'
>>
>> 'build/'             'scripts/'
>>
>>
>> In[6]: foo('COPYING.txt', <TAB>
>>
>> 'r', 'w'
>>
>>
>>
>>
>> -Robert
>>
>>
>> _______________________________________________
>>
>> IPython-dev mailing list
>>
>> IPython-dev at scipy.org
>>
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>>
>>
>> --
>>
>> Brian E. Granger
>>
>> Cal Poly State University, San Luis Obispo
>>
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>
>> _______________________________________________
>>
>> IPython-dev mailing list
>>
>> IPython-dev at scipy.org
>>
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>> _______________________________________________
>>
>> IPython-dev mailing list
>>
>> IPython-dev at scipy.org
>>
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From nick.bollweg at gmail.com  Tue Dec 18 18:51:53 2012
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Tue, 18 Dec 2012 18:51:53 -0500
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
Message-ID: <CACejjWwb=AsPQLEzHYHPE2rLftHV6056r9PE=DCrJtPw2_LAfQ@mail.gmail.com>

open question: where can automation help?

what is causing the bulk of time waste in doing community management to
really dig into what a particular issue or PR is trying to accomplish?
perhaps a cloud asset, along the lines of Travis-ci, that allowed single
click access to a running kernel, web notebook or other aspect of the
system?

while GH is great, and is cited as one of the factors leading to
involvement blowing up, would a more workflow-enabled system like jira or
rt offer better triage? with the API, Github could still be an entry point
into the ecosystem.

too much success seems like a happy problem... I hope everything works out!
On Dec 18, 2012 10:39 AM, "Thomas Kluyver" <takowl at gmail.com> wrote:

> On 17 December 2012 20:00, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> * How do we manage communication?  Verbal communication is much more
>> efficient than emails or even IRC.  The 4 people at Berkeley will have
>> an incredible advantage in being able to talk daily.  We don't want to
>> cripple or remove that advantage, but we need to figure out how to
>> include other core devs and people from the community.  This is
>> particularly relevant to myself as I am the only person involved in
>> the Sloan work that is not at Berkeley.
>>
>
> I think this issue - communication - is becoming key. I spent a day away
> from my computer, and came back to >40 new e-mails in my IPython folder (in
> addition to the 20 odd unread that I'm planning to get round to one day).
> That's a mixture of our two mailing lists and the Github notification
> messages. I get the feeling we're approaching a critical point, where it's
> no longer possible for us as individuals to keep up with all the
> discussions going on.
>
> To my mind, we need to split things up. We already have an informal
> separation of interests - for instance, I leave most of the discussion
> about the notebook front-end to others, but get more involved with
> IPython.core. I think we need to make this a bit (not too much) more
> formal, so that no-one falls down the cracks as everything speeds up.
>
> This could mean, e.g. more specialised mailing lists, or having a
> consistent process for handling incoming issues.
>
> Best wishes,
> Thomas
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121218/8afd2823/attachment.html>

From bussonniermatthias at gmail.com  Wed Dec 19 02:41:29 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Wed, 19 Dec 2012 08:41:29 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
	<CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
Message-ID: <CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>

We could detect (? It is no valid python and seems appropriate

ITOH tooltip can also give info on non callable. One can trigger tooltip
onlt after ?<tab> and remove the ? To allow fast typing.
Le 19 d?c. 2012 00:05, "Brian Granger" <ellisonbg at gmail.com> a ?crit :

> I think we want both the tab completion and tooltip to work at the
> same time = not be mutually exclusive.  I think we should preserve TAB
> for the completion logic and move tooltip to something else to be
> consistent.  This new tab completion logic is really important.  Let's
> brain storm about ways of allowing both to coexist.
>
> On Tue, Dec 18, 2012 at 2:03 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
> >
> > Le 18 d?c. 2012 ? 10:52, Robert McGibbon a ?crit :
> >
> > Matthias,
> >
> > Good call. That's it exactly. Unfortunately, I think dealing with this
> issue
> > is a little above my expertise/experience.
> >
> > When I apply this patch (pasted below) to the PR, the tab completions
> show
> > in the notebook for foo(<TAB>.
> >
> > The problem is that behavior kind of conflicts with the tooltip feature,
> > which is not a feature of the terminal console.
> >
> >
> > without your patch,
> > You could set
> > this.tooltip_on_tab = true;
> > L25 in codecell.js to false.
> > It will trigger the tooltip only if you stay after a opening bracket for
> a
> > certain amount of time.
> > The the completer should work as expected.
> >
> > It should be made configurable.
> >
> > we could try to tweak the behavior so that (<tab> trigger tooltip
> > ([space]<tab> trigger completion.
> >
> > But I guess that's beyond the extend of your PR.
> > --
> > Matthias
> >
> >
> >
> > -Robert
> >
> > p.s.
> >
> > I think I'm finally starting to see the wisdom of the comment on line
> 1046
> > of IPython/core/complete.py,
> >
> >         # FIXME: we should extend our api to return a dict with
> completions
> > for
> >         # different types of objects.  The rlcomplete() method could then
> >         # simply collapse the dict into a list for readline, but we'd
> have
> >         # richer completion semantics in other evironments.
> >
> >
> > $ git diff
> > diff --git a/IPython/frontend/html/notebook/static/js/codecell.js
> > b/IPython/frontend/html/notebook/static/js/codecell.js
> > index ea34719..000734b 100644
> > --- a/IPython/frontend/html/notebook/static/js/codecell.js
> > +++ b/IPython/frontend/html/notebook/static/js/codecell.js
> > @@ -155,6 +155,8 @@ var IPython = (function (IPython) {
> >                  IPython.tooltip.request(that);
> >                  // Prevent the event from bubbling up.
> >                  event.stop();
> > +                console.log('RTM DEBUG');
> > +                this.completer.startCompletion();
> >                  // Prevent CodeMirror from handling the tab.
> >                  return true;
> >              } else {
> > diff --git a/IPython/frontend/html/notebook/static/js/completer.js
> > b/IPython/frontend/html/notebook/static/js/completer.js
> > index ac2d95c..94aff4b 100644
> > --- a/IPython/frontend/html/notebook/static/js/completer.js
> > +++ b/IPython/frontend/html/notebook/static/js/completer.js
> > @@ -105,8 +105,9 @@ var IPython = (function (IPython) {
> >          // we need to check that we are still on a word boundary
> >          // because while typing the completer is still reinvoking itself
> >          if (!/[0-9a-z._/\\:~-]/i.test(pre_cursor)) {
> > -            this.close();
> > -            return;
> > +            //this.close();
> > +            //return;
> > +            console.log('RTM DEBUG2');
> >          }
> >
> >          this.autopick = false;
> >
> >
> > On Dec 18, 2012, at 1:17 AM, Matthias Bussonnier wrote:
> >
> > Ah... Then it's probably the tooltip code that take over.
> >
> > Same folder, codecell.js look for TAB or pre_cursor.
> >
> > Completion is not triggerd if char in front of cursor is space or opening
> > bracket.
> >
> > Short from my phone.
> >
> > Le 18 d?c. 2012 09:28, "Robert McGibbon" <rmcgibbo at gmail.com> a ?crit :
> >>
> >> Marrhias
> >>
> >> the baz(notevaluated).<TAB> is working in the notebook.
> >>
> >> it's the foo(<TAB> that's not working... yet.
> >>
> >> my fridge is not out of beer, so i'm still working :)
> >>
> >> -Robert
> >>
> >> On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:
> >>
> >>
> >> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
> >>
> >> It's not working in the notebook with the argument specific completions,
> >> but the return value completion is working. There must be some issue
> with a
> >> difference in the line splitting with readline and without. I'll track
> it
> >> down.
> >>
> >>
> >> That's probably the js that removes everything between bracket by
> nothing
> >> (IIRC, first completer was doing that.)
> >>
> >> baz(notevaluated).<tab>
> >> to
> >> baz.<tab>
> >>
> >> before completion. (or is it tooltip... that does it)
> >>
> >> Or a security about 'as you type'
> >>
> >> Actually every key press recall the completer once it has been invoked
> and
> >> it decide to 'kill itself'
> >> on certain things (like closing bracket, spaces...)
> >>
> >> That would be
> >>
> >> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
> >> --
> >> Matthias
> >>
> >>
> >>
> >>
> >> -Robert
> >>
> >>
> >> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
> >>
> >>
> >> This looks really great.  Have you tested it with the notebook yet?  I
> >>
> >> would imagine that it should already work there as well.
> >>
> >>
> >> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com>
> >> wrote:
> >>
> >> Hey,
> >>
> >>
> >> I just posted a new PR for annotation based custom tab completion.
> >>
> >> https://github.com/ipython/ipython/pull/2701
> >>
> >>
> >> The code is fairly complex (~1000 new lines), so I assume that there are
> >>
> >> bugs. For that reason (and because its fun!), I encourage people
> >>
> >> to try taking it out for a spin.
> >>
> >>
> >>
> >> It lets you do things like this:
> >>
> >>
> >> In [1]: from IPython.extensions.completion import tab_complete,
> globs_to,
> >>
> >> literal
> >>
> >>
> >> In [2]: @tab_complete
> >>
> >> ...: def baz(x) -> str:
> >>
> >> ...:     pass
> >>
> >> ...:
> >>
> >>
> >> In [3]: baz(notevaluated).<TAB>
> >>
> >> .capitalize  .find        .isspace     .partition   .rstrip
> >> .translate
> >>
> >> .center      .format      .istitle     .replace     .split       .upper
> >>
> >> .count       .index       .isupper     .rfind       .splitlines  .zfill
> >>
> >> .decode      .isalnum     .join        .rindex      .startswith
> >>
> >> .encode      .isalpha     .ljust       .rjust       .strip
> >>
> >> .endswith    .isdigit     .lower       .rpartition  .swapcase
> >>
> >> .expandtabs  .islower     .lstrip      .rsplit      .title
> >>
> >>
> >>
> >> And this:
> >>
> >>
> >> In [4]: @tab_complete
> >>
> >> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
> >>
> >> ...:     pass
> >>
> >> ...:
> >>
> >>
> >> In [5]: foo(<TAB>
> >>
> >> 'COPYING.txt'        'dist/'              'setupext/'
> >>
> >> 'IPython/'           'docs/'              'tools/'
> >>
> >> '__pycache__/'       'ipython.egg-info/'
> >>
> >> 'build/'             'scripts/'
> >>
> >>
> >> In[6]: foo('COPYING.txt', <TAB>
> >>
> >> 'r', 'w'
> >>
> >>
> >>
> >>
> >> -Robert
> >>
> >>
> >> _______________________________________________
> >>
> >> IPython-dev mailing list
> >>
> >> IPython-dev at scipy.org
> >>
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Brian E. Granger
> >>
> >> Cal Poly State University, San Luis Obispo
> >>
> >> bgranger at calpoly.edu and ellisonbg at gmail.com
> >>
> >> _______________________________________________
> >>
> >> IPython-dev mailing list
> >>
> >> IPython-dev at scipy.org
> >>
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >> _______________________________________________
> >>
> >> IPython-dev mailing list
> >>
> >> IPython-dev at scipy.org
> >>
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121219/aafcabf7/attachment.html>

From bussonniermatthias at gmail.com  Wed Dec 19 03:12:40 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 19 Dec 2012 09:12:40 +0100
Subject: [IPython-dev] Should I still contribute to IPython ?
In-Reply-To: <CACejjWwb=AsPQLEzHYHPE2rLftHV6056r9PE=DCrJtPw2_LAfQ@mail.gmail.com>
References: <B7E1B8E6-9224-4FBA-B7A7-2C4DE99D7169@gmail.com>
	<CAH4pYpTK4JR_fQBX89fUPZKTjuYVNB8+i33rriu+T=3o_-qnGA@mail.gmail.com>
	<CAOvn4qj=uye7-7ihspQQp6xsR9-ZtU0Uq-bUxbPtgnjtH+OsZw@mail.gmail.com>
	<CACejjWwb=AsPQLEzHYHPE2rLftHV6056r9PE=DCrJtPw2_LAfQ@mail.gmail.com>
Message-ID: <1B6EE590-4EBC-4557-9CFE-C141127B54A7@gmail.com>


Le 19 d?c. 2012 ? 00:51, Nicholas Bollweg a ?crit :

> open question: where can automation help?
> 
> what is causing the bulk of time waste in doing community management to really dig into what a particular issue or PR is trying to accomplish? perhaps a cloud asset, along the lines of Travis-ci, that allowed single click access to a running kernel, web notebook or other aspect of the system?

I think the problem is you don't really know who is responsible for which PR/issue, so you lost time wondering wether you have to be involved, 
or if you are in a good place to review. I have no problem getting the code to run. For me it's a one line

$ git checkout origin/pull/2463 ; ipython notebook

Sometime 2 if I need to do a git fetch before and adding /head or /merge at the end is enough to know which state of the PR I want to test.

The longer part is to read the PR, unpacked the new UV laser, and wondering what the other dev thought about that are. 
I miss a small "involvement indicator" on PR that say "involved"/"neutral"/"don't care" on the side to know which PR I should 
take care of. 

Also the "who agree for merging" is a little confusing sometime. 
I would like a "ok to merge" button, that you just click and you get the count of +1 vote that goes back to 0 if a new commit 
is pushed.

I think this would highly improved the merging /review rate with :

 * This pr has 3 "merge ok", I just have to read add mine and I can merge. 
 * This PR has already 2 dev with are "involved" I can worry of others.
 * This one has nobody I'll look after it.

-- 
Matthias


> 
> while GH is great, and is cited as one of the factors leading to involvement blowing up, would a more workflow-enabled system like jira or rt offer better triage? with the API, Github could still be an entry point into the ecosystem.
> 
> too much success seems like a happy problem... I hope everything works out!
> 
> On Dec 18, 2012 10:39 AM, "Thomas Kluyver" <takowl at gmail.com> wrote:
> On 17 December 2012 20:00, Brian Granger <ellisonbg at gmail.com> wrote:
> * How do we manage communication?  Verbal communication is much more
> efficient than emails or even IRC.  The 4 people at Berkeley will have
> an incredible advantage in being able to talk daily.  We don't want to
> cripple or remove that advantage, but we need to figure out how to
> include other core devs and people from the community.  This is
> particularly relevant to myself as I am the only person involved in
> the Sloan work that is not at Berkeley.
> 
> I think this issue - communication - is becoming key. I spent a day away from my computer, and came back to >40 new e-mails in my IPython folder (in addition to the 20 odd unread that I'm planning to get round to one day). That's a mixture of our two mailing lists and the Github notification messages. I get the feeling we're approaching a critical point, where it's no longer possible for us as individuals to keep up with all the discussions going on.
> 
> To my mind, we need to split things up. We already have an informal separation of interests - for instance, I leave most of the discussion about the notebook front-end to others, but get more involved with IPython.core. I think we need to make this a bit (not too much) more formal, so that no-one falls down the cracks as everything speeds up.
> 
> This could mean, e.g. more specialised mailing lists, or having a consistent process for handling incoming issues.
> 
> Best wishes,
> Thomas
> 
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From damianavila at gmail.com  Wed Dec 19 07:50:40 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Wed, 19 Dec 2012 09:50:40 -0300
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
Message-ID: <50D1B820.2030300@gmail.com>

El 03/12/12 09:14, Carl Smith escribi?:
>
> I've not had chance to run the code, but the demo was good. You should 
> maybe start trying to get the styles closer to the Notebook though. 
> Yours looked pretty, but too different. I found myself looking more at 
> what CSS can do than at the slide functionality itself. Good work though.
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Carl, a closer style as you requested...check this out: 
http://www.slideviper.oquanta.info/nbcreveal/example_nb_tour_slides.html 
<http://www.slideviper.oquanta.info/nbcreveal/example_nb_tour_slides.html#/>

Dami?n.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121219/5dc09c54/attachment.html>

From javi.martinez.lopez at gmail.com  Wed Dec 19 08:25:17 2012
From: javi.martinez.lopez at gmail.com (=?ISO-8859-1?Q?Javier_Mart=EDnez=2DL=F3pez?=)
Date: Wed, 19 Dec 2012 14:25:17 +0100
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <50D1B820.2030300@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
	<50D1B820.2030300@gmail.com>
Message-ID: <CAKgwV9n_Yeyjp40NAWJLUYNQGsq=8UJo28Vvzun8HHV=d_WV-Q@mail.gmail.com>

It looks great! I am looking forward to using it in the stable version!
Thank you, Javier

On Wed, Dec 19, 2012 at 1:50 PM, Dami?n Avila <damianavila at gmail.com> wrote:

>  El 03/12/12 09:14, Carl Smith escribi?:
>
> I've not had chance to run the code, but the demo was good. You should
> maybe start trying to get the styles closer to the Notebook though. Yours
> looked pretty, but too different. I found myself looking more at what CSS
> can do than at the slide functionality itself. Good work though.
>
>
> _______________________________________________
> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> Carl, a closer style as you requested...check this out:
> http://www.slideviper.oquanta.info/nbcreveal/example_nb_tour_slides.html<http://www.slideviper.oquanta.info/nbcreveal/example_nb_tour_slides.html#/>
>
> Dami?n.
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121219/fa2ee270/attachment.html>

From carl.input at gmail.com  Wed Dec 19 10:01:06 2012
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 19 Dec 2012 15:01:06 +0000
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <50D1B820.2030300@gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
	<50D1B820.2030300@gmail.com>
Message-ID: <CAP-uhDcZyF51cm8tvJ7Mx9hkhwNe2hKPTkjRsJgbOAU+E7i_Zg@mail.gmail.com>

Very nice work. This has really come together well. I can see this becoming
a really popular feature too.

Nice hacking chap. Well done.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121219/d6ba8bf5/attachment.html>

From damianavila at gmail.com  Thu Dec 20 00:36:21 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Thu, 20 Dec 2012 02:36:21 -0300
Subject: [IPython-dev] slider converter in nbconvert
In-Reply-To: <CAP-uhDcZyF51cm8tvJ7Mx9hkhwNe2hKPTkjRsJgbOAU+E7i_Zg@mail.gmail.com>
References: <50BC2CBB.9060200@gmail.com>
	<CAP-uhDdXyVh5B1ndFFFhC7ktSJCu1=tOCBanMuxHQzAEhW43yA@mail.gmail.com>
	<50D1B820.2030300@gmail.com>
	<CAP-uhDcZyF51cm8tvJ7Mx9hkhwNe2hKPTkjRsJgbOAU+E7i_Zg@mail.gmail.com>
Message-ID: <50D2A3D5.1040500@gmail.com>

El 19/12/12 12:01, Carl Smith escribi?:
>
> Very nice work. This has really come together well. I can see this 
> becoming a really popular feature too.
>
> Nice hacking chap. Well done.
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

OK, guys... here in the last improved slyling: 
http://www.slideviper.oquanta.info/nbcreveal/example_nb_tour_slides.html 
<http://www.slideviper.oquanta.info/nbcreveal/example_nb_tour_slides.html#/>

I am almost there... hehe.

Regards.

Dami?n.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/d1987ade/attachment.html>

From javi.martinez.lopez at gmail.com  Thu Dec 20 08:13:32 2012
From: javi.martinez.lopez at gmail.com (=?ISO-8859-1?Q?Javier_Mart=EDnez=2DL=F3pez?=)
Date: Thu, 20 Dec 2012 14:13:32 +0100
Subject: [IPython-dev] ipython notebook and ggplot2
Message-ID: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>

Dear all,

I am sorry if this is a particular question but I am working with R using
the ipython notebook and I cannot plot grahps using the ggplot2 library. I
do not know why, when I run the code nothing comes up! The same code works
fine using the usual R interface and all other non ggplot2 graphs work fine
under the nb. Of course, I previously loaded the ggplot2 library before
plotting. Is there a trick, does anyone knows anything about this?

Thank you very much for your help,

Javier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/b34f11b8/attachment.html>

From rmcgibbo at gmail.com  Thu Dec 20 13:29:57 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Thu, 20 Dec 2012 10:29:57 -0800
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
Message-ID: <6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>

The trick is that in the rmagic code, it calls dev.off() and does some handling for base graphics after every R code cell is executed.

When I use ggplot2 in the terminal app (not notebook), what I do is call 

ggsave('temp.png'); system('open temp.png') 

At the end of my plotting session. This pops up the plot in my image editor.

For the notebook, I think it's a little more complicated because you have to send the png to the front end through the display pub. 

Sent from my iPhone

On Dec 20, 2012, at 5:13 AM, Javier Mart?nez-L?pez <javi.martinez.lopez at gmail.com> wrote:

> Dear all,
> 
> I am sorry if this is a particular question but I am working with R using the ipython notebook and I cannot plot grahps using the ggplot2 library. I do not know why, when I run the code nothing comes up! The same code works fine using the usual R interface and all other non ggplot2 graphs work fine under the nb. Of course, I previously loaded the ggplot2 library before plotting. Is there a trick, does anyone knows anything about this?
> 
> Thank you very much for your help,
> 
> Javier
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From takowl at gmail.com  Thu Dec 20 13:45:36 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 20 Dec 2012 18:45:36 +0000
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
	<6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
Message-ID: <CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>

On 20 December 2012 18:29, Robert McGibbon <rmcgibbo at gmail.com> wrote:

> ggsave('temp.png'); system('open temp.png')
>
> At the end of my plotting session. This pops up the plot in my image
> editor.
>
> For the notebook, I think it's a little more complicated because you have
> to send the png to the front end through the display pub.


In fact, the notebook just sets up a specific temporary directory for your
R code, and then displays the png files saved in there. Maybe we should
expose that to the R code as something like _ipy_tmpd, so that you can save
arbitrary png files?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/06038c44/attachment.html>

From rmcgibbo at gmail.com  Thu Dec 20 14:05:34 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Thu, 20 Dec 2012 11:05:34 -0800
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
	<6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
	<CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
Message-ID: <C40AF51D-0724-47D7-B115-423D922363F8@gmail.com>

Another option would be to present the ability for the user to register post completion hooks with rmagic that would run some python code
after every %%R code cell. The default post completion hook would implement the current behavior of calling dev.off(), but others could call ggsave() or whatever.

-Robert


On Dec 20, 2012, at 10:45 AM, Thomas Kluyver wrote:

> On 20 December 2012 18:29, Robert McGibbon <rmcgibbo at gmail.com> wrote:
> ggsave('temp.png'); system('open temp.png')
> 
> At the end of my plotting session. This pops up the plot in my image editor.
> 
> For the notebook, I think it's a little more complicated because you have to send the png to the front end through the display pub.
> 
> In fact, the notebook just sets up a specific temporary directory for your R code, and then displays the png files saved in there. Maybe we should expose that to the R code as something like _ipy_tmpd, so that you can save arbitrary png files?
> 
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/1f640218/attachment.html>

From rmcgibbo at gmail.com  Thu Dec 20 14:09:19 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Thu, 20 Dec 2012 11:09:19 -0800
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
	<6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
	<CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
Message-ID: <1EBCD378-D949-4695-B115-8623AC9310C6@gmail.com>

Oh. Never mind. Since the R cell is executed in that temporary directory, just calling ggsave() with a relative path should just work. That will save the plot in the current
temporary directory, and then ipython will find it and ship it to the browser.

-Robert

On Dec 20, 2012, at 10:45 AM, Thomas Kluyver wrote:

> On 20 December 2012 18:29, Robert McGibbon <rmcgibbo at gmail.com> wrote:
> ggsave('temp.png'); system('open temp.png')
> 
> At the end of my plotting session. This pops up the plot in my image editor.
> 
> For the notebook, I think it's a little more complicated because you have to send the png to the front end through the display pub.
> 
> In fact, the notebook just sets up a specific temporary directory for your R code, and then displays the png files saved in there. Maybe we should expose that to the R code as something like _ipy_tmpd, so that you can save arbitrary png files?
> 
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/cb1a9b76/attachment.html>

From javi.martinez.lopez at gmail.com  Thu Dec 20 14:31:55 2012
From: javi.martinez.lopez at gmail.com (=?ISO-8859-1?Q?Javier_Mart=EDnez=2DL=F3pez?=)
Date: Thu, 20 Dec 2012 20:31:55 +0100
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <1EBCD378-D949-4695-B115-8623AC9310C6@gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
	<6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
	<CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
	<1EBCD378-D949-4695-B115-8623AC9310C6@gmail.com>
Message-ID: <CAKgwV9mwLMqT9pEbM-2uD+qS2Y5mBbpMLu-M4EhwzrS4Mcd_ug@mail.gmail.com>

Thank you very much. I saved it and opened it with the 'Image' python
command, but I still do not understand what is the difference between the
usual and the ggplot2 graphs!

Javier

On Thu, Dec 20, 2012 at 8:09 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:

> Oh. Never mind. Since the R cell is executed in that temporary directory,
> just calling ggsave() with a relative path should just work. That will save
> the plot in the current
> temporary directory, and then ipython will find it and ship it to the
> browser.
>
> -Robert
>
> On Dec 20, 2012, at 10:45 AM, Thomas Kluyver wrote:
>
> On 20 December 2012 18:29, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>
>> ggsave('temp.png'); system('open temp.png')
>>
>> At the end of my plotting session. This pops up the plot in my image
>> editor.
>>
>> For the notebook, I think it's a little more complicated because you have
>> to send the png to the front end through the display pub.
>
>
> In fact, the notebook just sets up a specific temporary directory for your
> R code, and then displays the png files saved in there. Maybe we should
> expose that to the R code as something like _ipy_tmpd, so that you can save
> arbitrary png files?
>
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/27a7a8bf/attachment.html>

From rmcgibbo at gmail.com  Thu Dec 20 18:52:33 2012
From: rmcgibbo at gmail.com (Robert McGibbon)
Date: Thu, 20 Dec 2012 15:52:33 -0800
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <CAKgwV9mwLMqT9pEbM-2uD+qS2Y5mBbpMLu-M4EhwzrS4Mcd_ug@mail.gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
	<6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
	<CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
	<1EBCD378-D949-4695-B115-8623AC9310C6@gmail.com>
	<CAKgwV9mwLMqT9pEbM-2uD+qS2Y5mBbpMLu-M4EhwzrS4Mcd_ug@mail.gmail.com>
Message-ID: <019B4168-A17D-4E57-91BC-0D42598B0496@gmail.com>

ggplot2 and base graphics in R use a totally different plotting mechanism and api. The IPython rmagic extension has built-in support for the base graphics system, but doesn't have corresponding support for 	ggplot.

The R cell isn't actually executed in that temporary directory, so its impossible to call the `save ggplot` command from R with the right path.

I just submitted a two line PR that should fix the issue for ggplot.

https://github.com/ipython/ipython/pull/2711

-Robert

On Dec 20, 2012, at 11:31 AM, Javier Mart?nez-L?pez wrote:

> Thank you very much. I saved it and opened it with the 'Image' python command, but I still do not understand what is the difference between the usual and the ggplot2 graphs!
> 
> Javier
> 
> On Thu, Dec 20, 2012 at 8:09 PM, Robert McGibbon <rmcgibbo at gmail.com> wrote:
> Oh. Never mind. Since the R cell is executed in that temporary directory, just calling ggsave() with a relative path should just work. That will save the plot in the current
> temporary directory, and then ipython will find it and ship it to the browser.
> 
> -Robert
> 
> On Dec 20, 2012, at 10:45 AM, Thomas Kluyver wrote:
> 
>> On 20 December 2012 18:29, Robert McGibbon <rmcgibbo at gmail.com> wrote:
>> ggsave('temp.png'); system('open temp.png')
>> 
>> At the end of my plotting session. This pops up the plot in my image editor.
>> 
>> For the notebook, I think it's a little more complicated because you have to send the png to the front end through the display pub.
>> 
>> In fact, the notebook just sets up a specific temporary directory for your R code, and then displays the png files saved in there. Maybe we should expose that to the R code as something like _ipy_tmpd, so that you can save arbitrary png files?
>> 
>> Thomas
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121220/8283044d/attachment.html>

From javi.martinez.lopez at gmail.com  Fri Dec 21 05:28:18 2012
From: javi.martinez.lopez at gmail.com (=?ISO-8859-1?Q?Javier_Mart=EDnez=2DL=F3pez?=)
Date: Fri, 21 Dec 2012 11:28:18 +0100
Subject: [IPython-dev] ipython notebook and ggplot2
In-Reply-To: <CAMEX5syqQU4Oixf0ho_EYvXL1tdG93kj=+Fb6KwS2Em=PJDibg@mail.gmail.com>
References: <CAKgwV9=Jz3y=ZTCi+TjvXjdoA9v2++NNM=HNWwo-cws=j-yVrA@mail.gmail.com>
	<6C0D9DA6-5AF0-414E-B8A4-514834603E40@gmail.com>
	<CAOvn4qh1q9dzqxekA_PZro7u9YFrdpnL24Rm2WVaOU2aWBkNzQ@mail.gmail.com>
	<1EBCD378-D949-4695-B115-8623AC9310C6@gmail.com>
	<CAKgwV9mwLMqT9pEbM-2uD+qS2Y5mBbpMLu-M4EhwzrS4Mcd_ug@mail.gmail.com>
	<019B4168-A17D-4E57-91BC-0D42598B0496@gmail.com>
	<CAMEX5syqQU4Oixf0ho_EYvXL1tdG93kj=+Fb6KwS2Em=PJDibg@mail.gmail.com>
Message-ID: <CAKgwV9nT-vNAyVBKVVDTVN9ifRktAd2EHH6LQswkHb6hMqgRiQ@mail.gmail.com>

That works! I noticed already that I had to add the print command for
seeing variables and summaries but I didn't know it would work with plots.
Thank you very much.

Javier

On Fri, Dec 21, 2012 at 1:23 AM, Charlie Sharpsteen <chuck at sharpsteen.net>wrote:

> On Thu, Dec 20, 2012 at 3:52 PM, Robert McGibbon <rmcgibbo at gmail.com>wrote:
>
>> The IPython rmagic extension has built-in support for the base graphics
>> system, but doesn't have corresponding support for ggplot.
>>
>
> (NOTE: Looks like Mailman may be eating my responses for some reason, so
> CCing those directly involved.)
>
> All you need for "corresponding support" is to call `print()` on the
> output of grid graphics (such as ggplot2) when writing R code.
>
> The plotting functions in the base graphics system immediately paint
> results to the output device when the function is executed. In contrast,
> the plotting functions in the grid system return objects. This
> object-oriented nature is what makes ggplot2 work so well---complicated
> graphics can be built in an intuitive fashion because each function is
> returning objects that can be combined via overloaded addition operators.
> grid graphics aren't painted to the output device until the `print()`
> method is called on the object.
>
> The source of confusion is that the native R REPL (i.e. the thing that
> people play with when using the R GUI or the R command line interface)
> implicitly calls `print()` on the result of every computation. When running
> R programs outside of the official REPL, such as through RScript or in this
> case rpy2, the calls to `print()` must be explicitly added by the
> programmer in order to get code that works the same way.
>
> This is true for several R objects, but is especially noticeable with grid
> graphics.
>
> -Charlie
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121221/b01fdcb0/attachment.html>

From jed.ludlow at gmail.com  Fri Dec 21 18:20:22 2012
From: jed.ludlow at gmail.com (Jed Ludlow)
Date: Fri, 21 Dec 2012 16:20:22 -0700
Subject: [IPython-dev] (no subject)
Message-ID: <CAFszPdhq_vsrjosvQiXhgWV9Y6h5dkz6F-FsyuDpUbDuYecjCg@mail.gmail.com>

Hi.

Quick question from the Spyder project. I'm working on some debugging
related features, and I can't quite figure out the right way to communicate
with IPython during debugging.

As background, Spyder can launch instances of IPythonQtConsoleApp in the
IDE which make use of RichIPythonWidget. These get associated with an
IPython kernel instance. As part of the IDE there are commands and toolbars
to, say, execute the current editor contents in an IPython console. We are
currently using the execute() method (inherited from
IPython's HistoryConsoleWidget) to send these commands, and as best I can
tell this is the method by which typical keyboard input is also sent.
Anyway, this approach works fine for sending commands directly to
conventional input prompts.

Difficulty arises when IPython is in debug mode. I'm
currently experimenting with a simple debugging toolbar that will send the
typical pdb commands (next, step, continue, etc.) down to the shell during
a debug session. But using the execute() method as described above does not
work while debugging. It seems like pdb commands must be taking a different
route to get to the kernel, but I can't seem to figure out what it is. Any
guidance you can provide would be most appreciated.

Jed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121221/1750c4eb/attachment.html>

From jed.ludlow at gmail.com  Sat Dec 22 02:19:20 2012
From: jed.ludlow at gmail.com (Jed Ludlow)
Date: Sat, 22 Dec 2012 00:19:20 -0700
Subject: [IPython-dev] Manipulate stdin to IPythonQtConsoleApp
Message-ID: <CAFszPdgC0AhQ2LEe7SCkqMz3+=dbDd8Me5cKsEsM=SU8wZae8w@mail.gmail.com>

> Quick question from the Spyder project. I'm working on some debugging
> related features, and I can't quite figure out the right way to communicate
> with IPython during debugging.
>
> As background, Spyder can launch instances of IPythonQtConsoleApp in the
> IDE which make use of RichIPythonWidget. These get associated with an
> IPython kernel instance. As part of the IDE there are commands and toolbars
> to, say, execute the current editor contents in an IPython console. We are
> currently using the execute() method (inherited from
> IPython's HistoryConsoleWidget) to send these commands, and as best I can
> tell this is the method by which typical keyboard input is also sent.
> Anyway, this approach works fine for sending commands directly to
> conventional input prompts.
>
> Difficulty arises when IPython is in debug mode. I'm
> currently experimenting with a simple debugging toolbar that will send the
> typical pdb commands (next, step, continue, etc.) down to the shell during
> a debug session. But using the execute() method as described above does not
> work while debugging. It seems like pdb commands must be taking a different
> route to get to the kernel, but I can't seem to figure out what it is. Any
> guidance you can provide would be most appreciated.
>
>
Sorry for the empty subject on my previous message above on this topic.
It's been that kind of day.

After studying the problem further, I suspect what I'm looking for is a way
to write directly to stdin. The execute() method works well to send Python
code to the the interpreter itself. But it seems that once the debugger is
active it's just looking for commands on stdin. Is there a mechanism to
manipulate this?

Jed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121222/5fb174a7/attachment.html>

From takowl at gmail.com  Sat Dec 22 07:10:40 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 22 Dec 2012 12:10:40 +0000
Subject: [IPython-dev] (no subject)
In-Reply-To: <CAFszPdhq_vsrjosvQiXhgWV9Y6h5dkz6F-FsyuDpUbDuYecjCg@mail.gmail.com>
References: <CAFszPdhq_vsrjosvQiXhgWV9Y6h5dkz6F-FsyuDpUbDuYecjCg@mail.gmail.com>
Message-ID: <CAOvn4qgAWO-wCCx2LBtT4sEB0raHyGDDZ7euO4H8spN45=uq+Q@mail.gmail.com>

On 21 December 2012 23:20, Jed Ludlow <jed.ludlow at gmail.com> wrote:

> Difficulty arises when IPython is in debug mode. I'm
> currently experimenting with a simple debugging toolbar that will send the
> typical pdb commands (next, step, continue, etc.) down to the shell during
> a debug session. But using the execute() method as described above does not
> work while debugging. It seems like pdb commands must be taking a different
> route to get to the kernel, but I can't seem to figure out what it is. Any
> guidance you can provide would be most appreciated.
>

The IPython protocol doesn't 'know about' the debugger, so the kernel is
essentially running code that calls raw_input(). When that happens, the
kernel sends an input_request message to the frontend, and waits for an
input_reply message:

http://ipython.org/ipython-doc/stable/development/messaging.html#messages-on-the-stdin-router-dealer-sockets

Hope that helps,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121222/f0ac867d/attachment.html>

From jason-sage at creativetrax.com  Sat Dec 29 11:27:11 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 29 Dec 2012 09:27:11 -0700
Subject: [IPython-dev] interactive javascript controls
Message-ID: <50DF19DF.7040701@creativetrax.com>

Just FYI, I'm experimenting with ways to add javascript controls with 
bidirectional communication to have interactive controls.  I'm 
implementing these in the Sage cell server, but using the IPython 
infrastructure and communication protocols (but with my own custom 
messages for control updates and the like).  Here is the thread about it 
on sage-notebook:

https://groups.google.com/forum/?fromgroups=#!topic/sage-notebook/KHRyuUMxnN8

and here is a currently-working example:

http://aleph.sagemath.org/?q=504efe00-85d4-4047-9315-1398ffb0efb9&lang=sage

showing how to implement a new control (that lets you pick a 2d point) 
and have other sliders associated with the variables that interactively 
update with the following code:

with interactive_namespace(x=4,y=20):
     Point2D('x','y').create()
     slider('x', (0,100)).create()
     slider('y', (0,100)).create()
     ExpressionBox('x').create()

Again, just FYI...I'm still experimenting quite a bit, but trying to 
keep things general enough to be useful to go into IPython eventually.

Thanks,

Jason

--
Jason Grout


From ellisonbg at gmail.com  Sat Dec 29 13:29:33 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 29 Dec 2012 10:29:33 -0800
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
Message-ID: <CAH4pYpT8_HsOMot9a0AnuRr466b-c=_05ueqv7+nmHzqHjrP6g@mail.gmail.com>

Nissim,

Hi, glad to see you thinking about these issues, we are definitely
interested in them.  There is actually a lot going on with this
stuff...

> I'm new here and I wanted to run this by the group before I start working on
> it.  I'm interested in building a facility to use the notebook to edit and
> execute javascript code for presenting data from the python kernel.  I know
> there is a magic function and there is some sort of way to have python code
> render js in <script> tags, and you can put javascript in the Markdown
> cells, but I was envisioning something more like a standard codecell, but
> execution is handled by the browser rather than the kernel.

For security reasons, we are going to disable all user written
JavaScript in the notebook.  As the notebook is used in more and more
multiuser settings, it is too dangerous to run JavaScript in a context
that basically gives the browser full shell access to that users
assets.

> Let's say you've loaded some data and run some computations in your python
> kernel and now you want to display results in the notebook using some
> javascript library like d3, datatables or processing.  If you want to do
> this by writing the javascript code in a CodeCell and evaluating that code
> in the browser you need a couple of things:

For something like this, we are already working on a JavaScript plugin
system that will enable people to write this JavaScript and load it
into the notebook server before it starts.  Here is the PR that
implements this:

https://github.com/ipython/ipython/pull/2518

This is still very much a work in progress, so we would love feedback.

> 1) ability to change a code cell language to javascipt in the notebook
> frontend.  I saw that there is a language field in the notebook file format
> for CodeCell but it is always = python.

We don't want to allow notebooks that mix different languages at the
CodeCell level.  It introduces too many complications.  However, we do
want to create a JavaScript based kernel that would allow you to
create an entire JavaScript based notebook.  However, we don't know
how to handle that in a secure manner, so the work is stalled (in
addition to not having anyone that is pushing on it - maybe that will
be you ;) ).

> 2) facility to have the browser JavaScript engine evaluate the contents of
> the javascript CodeCell when it is executed by the user

This is possible, but again the security is the crux we have to solve.

> 3) http REST api that will return json representations of Python objects
> that exist in the kernel.  (this may already exist, I don't really
> understand the kernel communication protocol yet).

This is already implemented in the PR that I link to above.

> There are probably some more things like the ability to add CSS, HTML, js
> and image resources to the server that it can then serve back to the client.
> I see that there's been a lot of discussion related to this in the plugins
> thread, but maybe there can also be a method for accessing resources from
> the network and then telling the notebook server to serve them from some
> path.

This is exactly what the JavaScript plugin idea above adds.  We even
have a d3 example.

> I'd like to implement this and I had a couple of questions:
>
> 1) Does this fit in with your ideas for the notebook or is editing non
> python code in CodeCell outside of the vision?

Yes, we want the notebook to support other languages.

> 2) Should there be another Dropdown for the language of the CodeCell or
> should there be a javascript option added to the existing select box.

No, at this point we want each notebook to be tied to a kernel in a
single language.  What this means is that as we add kernels in other
languages (this is already happening) we will add a UI for selecting
the language of the notebook upon creation.  But this has to be done
at notebook creation time when the kernel is created.  IOW, it won't
be possible to *change* the language of a notebook once it is started.

But again, the core problem with the JavaScript kernel is one if
security.  There was some talk of working with Mozilla on this, but
that has stalled for now.

For now, I would say that the best thing to do is to help us review
and test the JavaScript plugin PR above.  That will be the first
method of getting better JavaScript support in the notebook.

Cheers,

Brian


> Thanks
>
>    -Nissim
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sat Dec 29 13:32:35 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 29 Dec 2012 10:32:35 -0800
Subject: [IPython-dev] interactive javascript controls
In-Reply-To: <50DF19DF.7040701@creativetrax.com>
References: <50DF19DF.7040701@creativetrax.com>
Message-ID: <CAH4pYpS34MG7G_PkbGS3ALK+1z=Xd40YmDsfjhe_Q7xAgBj-BA@mail.gmail.com>

Jason,

Great, we will look this over and keep it in mind as we get started
with that work. You are definitely going to be a key part of that work
:)  We will keep you in the loop as we start to move forward.

Cheers,

Brian

On Sat, Dec 29, 2012 at 8:27 AM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> Just FYI, I'm experimenting with ways to add javascript controls with
> bidirectional communication to have interactive controls.  I'm
> implementing these in the Sage cell server, but using the IPython
> infrastructure and communication protocols (but with my own custom
> messages for control updates and the like).  Here is the thread about it
> on sage-notebook:
>
> https://groups.google.com/forum/?fromgroups=#!topic/sage-notebook/KHRyuUMxnN8
>
> and here is a currently-working example:
>
> http://aleph.sagemath.org/?q=504efe00-85d4-4047-9315-1398ffb0efb9&lang=sage
>
> showing how to implement a new control (that lets you pick a 2d point)
> and have other sliders associated with the variables that interactively
> update with the following code:
>
> with interactive_namespace(x=4,y=20):
>      Point2D('x','y').create()
>      slider('x', (0,100)).create()
>      slider('y', (0,100)).create()
>      ExpressionBox('x').create()
>
> Again, just FYI...I'm still experimenting quite a bit, but trying to
> keep things general enough to be useful to go into IPython eventually.
>
> Thanks,
>
> Jason
>
> --
> Jason Grout
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sat Dec 29 15:14:28 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 29 Dec 2012 12:14:28 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
	<CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
	<CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>
Message-ID: <CAH4pYpSMTp0_pF1shyfKfem6GqO_Y+qYb3QAwZceK-9jtBb5KQ@mail.gmail.com>

We could do the (?.  What about shift-tab?  Would that be too
difficult to manage with all the other meanings of tab?

On Tue, Dec 18, 2012 at 11:41 PM, Matthias Bussonnier
<bussonniermatthias at gmail.com> wrote:
> We could detect (? It is no valid python and seems appropriate
>
> ITOH tooltip can also give info on non callable. One can trigger tooltip
> onlt after ?<tab> and remove the ? To allow fast typing.
>
> Le 19 d?c. 2012 00:05, "Brian Granger" <ellisonbg at gmail.com> a ?crit :
>
>> I think we want both the tab completion and tooltip to work at the
>> same time = not be mutually exclusive.  I think we should preserve TAB
>> for the completion logic and move tooltip to something else to be
>> consistent.  This new tab completion logic is really important.  Let's
>> brain storm about ways of allowing both to coexist.
>>
>> On Tue, Dec 18, 2012 at 2:03 AM, Matthias BUSSONNIER
>> <bussonniermatthias at gmail.com> wrote:
>> >
>> > Le 18 d?c. 2012 ? 10:52, Robert McGibbon a ?crit :
>> >
>> > Matthias,
>> >
>> > Good call. That's it exactly. Unfortunately, I think dealing with this
>> > issue
>> > is a little above my expertise/experience.
>> >
>> > When I apply this patch (pasted below) to the PR, the tab completions
>> > show
>> > in the notebook for foo(<TAB>.
>> >
>> > The problem is that behavior kind of conflicts with the tooltip feature,
>> > which is not a feature of the terminal console.
>> >
>> >
>> > without your patch,
>> > You could set
>> > this.tooltip_on_tab = true;
>> > L25 in codecell.js to false.
>> > It will trigger the tooltip only if you stay after a opening bracket for
>> > a
>> > certain amount of time.
>> > The the completer should work as expected.
>> >
>> > It should be made configurable.
>> >
>> > we could try to tweak the behavior so that (<tab> trigger tooltip
>> > ([space]<tab> trigger completion.
>> >
>> > But I guess that's beyond the extend of your PR.
>> > --
>> > Matthias
>> >
>> >
>> >
>> > -Robert
>> >
>> > p.s.
>> >
>> > I think I'm finally starting to see the wisdom of the comment on line
>> > 1046
>> > of IPython/core/complete.py,
>> >
>> >         # FIXME: we should extend our api to return a dict with
>> > completions
>> > for
>> >         # different types of objects.  The rlcomplete() method could
>> > then
>> >         # simply collapse the dict into a list for readline, but we'd
>> > have
>> >         # richer completion semantics in other evironments.
>> >
>> >
>> > $ git diff
>> > diff --git a/IPython/frontend/html/notebook/static/js/codecell.js
>> > b/IPython/frontend/html/notebook/static/js/codecell.js
>> > index ea34719..000734b 100644
>> > --- a/IPython/frontend/html/notebook/static/js/codecell.js
>> > +++ b/IPython/frontend/html/notebook/static/js/codecell.js
>> > @@ -155,6 +155,8 @@ var IPython = (function (IPython) {
>> >                  IPython.tooltip.request(that);
>> >                  // Prevent the event from bubbling up.
>> >                  event.stop();
>> > +                console.log('RTM DEBUG');
>> > +                this.completer.startCompletion();
>> >                  // Prevent CodeMirror from handling the tab.
>> >                  return true;
>> >              } else {
>> > diff --git a/IPython/frontend/html/notebook/static/js/completer.js
>> > b/IPython/frontend/html/notebook/static/js/completer.js
>> > index ac2d95c..94aff4b 100644
>> > --- a/IPython/frontend/html/notebook/static/js/completer.js
>> > +++ b/IPython/frontend/html/notebook/static/js/completer.js
>> > @@ -105,8 +105,9 @@ var IPython = (function (IPython) {
>> >          // we need to check that we are still on a word boundary
>> >          // because while typing the completer is still reinvoking
>> > itself
>> >          if (!/[0-9a-z._/\\:~-]/i.test(pre_cursor)) {
>> > -            this.close();
>> > -            return;
>> > +            //this.close();
>> > +            //return;
>> > +            console.log('RTM DEBUG2');
>> >          }
>> >
>> >          this.autopick = false;
>> >
>> >
>> > On Dec 18, 2012, at 1:17 AM, Matthias Bussonnier wrote:
>> >
>> > Ah... Then it's probably the tooltip code that take over.
>> >
>> > Same folder, codecell.js look for TAB or pre_cursor.
>> >
>> > Completion is not triggerd if char in front of cursor is space or
>> > opening
>> > bracket.
>> >
>> > Short from my phone.
>> >
>> > Le 18 d?c. 2012 09:28, "Robert McGibbon" <rmcgibbo at gmail.com> a ?crit :
>> >>
>> >> Marrhias
>> >>
>> >> the baz(notevaluated).<TAB> is working in the notebook.
>> >>
>> >> it's the foo(<TAB> that's not working... yet.
>> >>
>> >> my fridge is not out of beer, so i'm still working :)
>> >>
>> >> -Robert
>> >>
>> >> On Dec 18, 2012, at 12:22 AM, Matthias BUSSONNIER wrote:
>> >>
>> >>
>> >> Le 18 d?c. 2012 ? 09:03, Robert McGibbon a ?crit :
>> >>
>> >> It's not working in the notebook with the argument specific
>> >> completions,
>> >> but the return value completion is working. There must be some issue
>> >> with a
>> >> difference in the line splitting with readline and without. I'll track
>> >> it
>> >> down.
>> >>
>> >>
>> >> That's probably the js that removes everything between bracket by
>> >> nothing
>> >> (IIRC, first completer was doing that.)
>> >>
>> >> baz(notevaluated).<tab>
>> >> to
>> >> baz.<tab>
>> >>
>> >> before completion. (or is it tooltip... that does it)
>> >>
>> >> Or a security about 'as you type'
>> >>
>> >> Actually every key press recall the completer once it has been invoked
>> >> and
>> >> it decide to 'kill itself'
>> >> on certain things (like closing bracket, spaces...)
>> >>
>> >> That would be
>> >>
>> >> IPython/frontend/html/notebook/static/js/completer.js ~L 100 smth.
>> >> --
>> >> Matthias
>> >>
>> >>
>> >>
>> >>
>> >> -Robert
>> >>
>> >>
>> >> On Dec 17, 2012, at 11:51 PM, Brian Granger wrote:
>> >>
>> >>
>> >> This looks really great.  Have you tested it with the notebook yet?  I
>> >>
>> >> would imagine that it should already work there as well.
>> >>
>> >>
>> >> On Mon, Dec 17, 2012 at 11:27 PM, Robert McGibbon <rmcgibbo at gmail.com>
>> >> wrote:
>> >>
>> >> Hey,
>> >>
>> >>
>> >> I just posted a new PR for annotation based custom tab completion.
>> >>
>> >> https://github.com/ipython/ipython/pull/2701
>> >>
>> >>
>> >> The code is fairly complex (~1000 new lines), so I assume that there
>> >> are
>> >>
>> >> bugs. For that reason (and because its fun!), I encourage people
>> >>
>> >> to try taking it out for a spin.
>> >>
>> >>
>> >>
>> >> It lets you do things like this:
>> >>
>> >>
>> >> In [1]: from IPython.extensions.completion import tab_complete,
>> >> globs_to,
>> >>
>> >> literal
>> >>
>> >>
>> >> In [2]: @tab_complete
>> >>
>> >> ...: def baz(x) -> str:
>> >>
>> >> ...:     pass
>> >>
>> >> ...:
>> >>
>> >>
>> >> In [3]: baz(notevaluated).<TAB>
>> >>
>> >> .capitalize  .find        .isspace     .partition   .rstrip
>> >> .translate
>> >>
>> >> .center      .format      .istitle     .replace     .split       .upper
>> >>
>> >> .count       .index       .isupper     .rfind       .splitlines  .zfill
>> >>
>> >> .decode      .isalnum     .join        .rindex      .startswith
>> >>
>> >> .encode      .isalpha     .ljust       .rjust       .strip
>> >>
>> >> .endswith    .isdigit     .lower       .rpartition  .swapcase
>> >>
>> >> .expandtabs  .islower     .lstrip      .rsplit      .title
>> >>
>> >>
>> >>
>> >> And this:
>> >>
>> >>
>> >> In [4]: @tab_complete
>> >>
>> >> ...: def foo(x : globs_to('*.txt'), mode : literal('r', 'w')):
>> >>
>> >> ...:     pass
>> >>
>> >> ...:
>> >>
>> >>
>> >> In [5]: foo(<TAB>
>> >>
>> >> 'COPYING.txt'        'dist/'              'setupext/'
>> >>
>> >> 'IPython/'           'docs/'              'tools/'
>> >>
>> >> '__pycache__/'       'ipython.egg-info/'
>> >>
>> >> 'build/'             'scripts/'
>> >>
>> >>
>> >> In[6]: foo('COPYING.txt', <TAB>
>> >>
>> >> 'r', 'w'
>> >>
>> >>
>> >>
>> >>
>> >> -Robert
>> >>
>> >>
>> >> _______________________________________________
>> >>
>> >> IPython-dev mailing list
>> >>
>> >> IPython-dev at scipy.org
>> >>
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> Brian E. Granger
>> >>
>> >> Cal Poly State University, San Luis Obispo
>> >>
>> >> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >>
>> >> _______________________________________________
>> >>
>> >> IPython-dev mailing list
>> >>
>> >> IPython-dev at scipy.org
>> >>
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >> _______________________________________________
>> >>
>> >> IPython-dev mailing list
>> >>
>> >> IPython-dev at scipy.org
>> >>
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Sat Dec 29 16:06:34 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 29 Dec 2012 22:06:34 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <CAH4pYpSMTp0_pF1shyfKfem6GqO_Y+qYb3QAwZceK-9jtBb5KQ@mail.gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
	<CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
	<CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>
	<CAH4pYpSMTp0_pF1shyfKfem6GqO_Y+qYb3QAwZceK-9jtBb5KQ@mail.gmail.com>
Message-ID: <A6BE8B66-C833-4870-BD9D-55C0C35EBA6A@gmail.com>


Le 29 d?c. 2012 ? 21:14, Brian Granger a ?crit :

> We could do the (?.  What about shift-tab?  Would that be too
> difficult to manage with all the other meanings of tab?

No, it should be doable. 
The only other use is unindent. 

What about shit tab triggers : 
	Unindent (default code mirror option) if multiline selection.
        ObjectInfoRequest (tooltip) on selection if something selected.
	Smart ObjectInfoRequest if nothing selected. (same as actual [bracket]<tab>)

It is not incompatible with (?<tab>

And as some user requested invoking help without executing cell, we can bind ?<tab> 
to "execute" just current line to get help on pager. 


-- 
Matthias

From bussonniermatthias at gmail.com  Sat Dec 29 17:34:55 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 29 Dec 2012 23:34:55 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <A6BE8B66-C833-4870-BD9D-55C0C35EBA6A@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
	<CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
	<CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>
	<CAH4pYpSMTp0_pF1shyfKfem6GqO_Y+qYb3QAwZceK-9jtBb5KQ@mail.gmail.com>
	<A6BE8B66-C833-4870-BD9D-55C0C35EBA6A@gmail.com>
Message-ID: <4B882D58-B874-4ACF-BF00-BB8F9C347000@gmail.com>


Le 29 d?c. 2012 ? 22:06, Matthias BUSSONNIER a ?crit :

> 
> Le 29 d?c. 2012 ? 21:14, Brian Granger a ?crit :
> 
>> We could do the (?.  What about shift-tab?  Would that be too
>> difficult to manage with all the other meanings of tab?
> 
> No, it should be doable. 
> The only other use is unindent. 
> 
> What about shit tab triggers : 
> 	Unindent (default code mirror option) if multiline selection.
>        ObjectInfoRequest (tooltip) on selection if something selected.
> 	Smart ObjectInfoRequest if nothing selected. (same as actual [bracket]<tab>)


Done, cf 
https://github.com/ipython/ipython/pull/2728

And made configurable via custom JS. 
Current behavior is unchanged and just need to be switched to false.
-- 
Matthias



> 
> It is not incompatible with (?<tab>
> 
> And as some user requested invoking help without executing cell, we can bind ?<tab> 
> to "execute" just current line to get help on pager. 
> 
> 
> -- 
> Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121229/409a77f7/attachment.html>

From ellisonbg at gmail.com  Sun Dec 30 23:16:57 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 30 Dec 2012 20:16:57 -0800
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <4B882D58-B874-4ACF-BF00-BB8F9C347000@gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
	<CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
	<CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>
	<CAH4pYpSMTp0_pF1shyfKfem6GqO_Y+qYb3QAwZceK-9jtBb5KQ@mail.gmail.com>
	<A6BE8B66-C833-4870-BD9D-55C0C35EBA6A@gmail.com>
	<4B882D58-B874-4ACF-BF00-BB8F9C347000@gmail.com>
Message-ID: <CAH4pYpS-KFzteRZhZmJhPAwjJFC5o29mKy0OK1u=Kporx8023A@mail.gmail.com>

I think the default should always use TAB for completions.  If we do
that then the new function argument completions will work be default
with TAB right?  I like that.

On Sat, Dec 29, 2012 at 2:34 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 29 d?c. 2012 ? 22:06, Matthias BUSSONNIER a ?crit :
>
>
> Le 29 d?c. 2012 ? 21:14, Brian Granger a ?crit :
>
> We could do the (?.  What about shift-tab?  Would that be too
>
> difficult to manage with all the other meanings of tab?
>
>
> No, it should be doable.
> The only other use is unindent.
>
> What about shit tab triggers :
> Unindent (default code mirror option) if multiline selection.
>        ObjectInfoRequest (tooltip) on selection if something selected.
> Smart ObjectInfoRequest if nothing selected. (same as actual [bracket]<tab>)
>
>
>
> Done, cf
> https://github.com/ipython/ipython/pull/2728
>
> And made configurable via custom JS.
> Current behavior is unchanged and just need to be switched to false.
> --
> Matthias
>
>
>
>
> It is not incompatible with (?<tab>
>
> And as some user requested invoking help without executing cell, we can bind
> ?<tab>
> to "execute" just current line to get help on pager.
>
>
> --
> Matthias
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Mon Dec 31 03:45:29 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 31 Dec 2012 09:45:29 +0100
Subject: [IPython-dev] function specific tab completion via annotations
In-Reply-To: <CAH4pYpS-KFzteRZhZmJhPAwjJFC5o29mKy0OK1u=Kporx8023A@mail.gmail.com>
References: <9813FBC3-5151-46F1-B318-E06EC2936959@gmail.com>
	<CAH4pYpRDh-xqPR2O7EeFZrRM1+HNQ5pTO62-_iTuLtqxCoQ3_Q@mail.gmail.com>
	<880999E2-7D6E-4DDD-AA42-3AC2883C3F1C@gmail.com>
	<B947E28F-0B70-4C22-BA4D-3E57421E2D7F@gmail.com>
	<93AEB1BD-4F59-4A5B-AFD6-CE62C969E9F0@gmail.com>
	<CANJQusU9LhRMFk2BRZ5sYsbXT3_vSMENo8xMEeKAffjSq9TaAQ@mail.gmail.com>
	<9CA20672-CCCE-4E29-A42A-0D3D71F7F1FC@gmail.com>
	<7D2A284A-54BE-4A66-9B4A-4CCF27C8CFE2@gmail.com>
	<CAH4pYpRWGcTU=vSx1ekFtJeujMu48FCQJiLPiVnp4N2T7KqsSw@mail.gmail.com>
	<CANJQusXwbJG5TNSoNFtjQ_c7n_6qaCkSvGpu6DDzit0iWMPiww@mail.gmail.com>
	<CAH4pYpSMTp0_pF1shyfKfem6GqO_Y+qYb3QAwZceK-9jtBb5KQ@mail.gmail.com>
	<A6BE8B66-C833-4870-BD9D-55C0C35EBA6A@gmail.com>
	<4B882D58-B874-4ACF-BF00-BB8F9C347000@gmail.com>
	<CAH4pYpS-KFzteRZhZmJhPAwjJFC5o29mKy0OK1u=Kporx8023A@mail.gmail.com>
Message-ID: <CANJQusVtMdvU3ms+jybT4sy1ZFEk3xsOhLkRvu2Dx46j6+SgQw@mail.gmail.com>

Yes with the PR I sent, you just switch a flag and only completion will be
triggerd on tab.
-- 
Matthias
Le 31 d?c. 2012 05:17, "Brian Granger" <ellisonbg at gmail.com> a ?crit :

> I think the default should always use TAB for completions.  If we do
> that then the new function argument completions will work be default
> with TAB right?  I like that.
>
> On Sat, Dec 29, 2012 at 2:34 PM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
> >
> > Le 29 d?c. 2012 ? 22:06, Matthias BUSSONNIER a ?crit :
> >
> >
> > Le 29 d?c. 2012 ? 21:14, Brian Granger a ?crit :
> >
> > We could do the (?.  What about shift-tab?  Would that be too
> >
> > difficult to manage with all the other meanings of tab?
> >
> >
> > No, it should be doable.
> > The only other use is unindent.
> >
> > What about shit tab triggers :
> > Unindent (default code mirror option) if multiline selection.
> >        ObjectInfoRequest (tooltip) on selection if something selected.
> > Smart ObjectInfoRequest if nothing selected. (same as actual
> [bracket]<tab>)
> >
> >
> >
> > Done, cf
> > https://github.com/ipython/ipython/pull/2728
> >
> > And made configurable via custom JS.
> > Current behavior is unchanged and just need to be switched to false.
> > --
> > Matthias
> >
> >
> >
> >
> > It is not incompatible with (?<tab>
> >
> > And as some user requested invoking help without executing cell, we can
> bind
> > ?<tab>
> > to "execute" just current line to get help on pager.
> >
> >
> > --
> > Matthias
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121231/6525c9bc/attachment.html>

From jason-sage at creativetrax.com  Mon Dec 31 11:17:51 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Mon, 31 Dec 2012 09:17:51 -0700
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAH4pYpT8_HsOMot9a0AnuRr466b-c=_05ueqv7+nmHzqHjrP6g@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
	<CAH4pYpT8_HsOMot9a0AnuRr466b-c=_05ueqv7+nmHzqHjrP6g@mail.gmail.com>
Message-ID: <50E1BAAF.8000802@creativetrax.com>

On 12/29/12 11:29 AM, Brian Granger wrote:
> We don't want to allow notebooks that mix different languages at the
> CodeCell level.

Of course, cell magics alleviate this restriction, as we can right now 
do %%r, %%cython, etc., to effectively get different languages in 
different cells.

I don't see why we can't have a %%javascript that then just echoes the 
javascript back to the browser to execute.  What would be cool is for 
the syntax highlighting to also change if the cell detects that it is a 
%%r cell, etc.

Thanks,

Jason



From nissimk at gmail.com  Mon Dec 31 12:02:49 2012
From: nissimk at gmail.com (Nissim Karpenstein)
Date: Mon, 31 Dec 2012 12:02:49 -0500
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <50E1BAAF.8000802@creativetrax.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
	<CAH4pYpT8_HsOMot9a0AnuRr466b-c=_05ueqv7+nmHzqHjrP6g@mail.gmail.com>
	<50E1BAAF.8000802@creativetrax.com>
Message-ID: <CAFhnF0D8t5UExpkGeKY7OSpPO6fJnDZNOKJiFhCaL6pVK4cJNA@mail.gmail.com>

Syntax highlight changing for the %%language cells sounds closer to what I
was thinking of.

Can you guys explain to me the security concerns?  There are several
JS+CSS+HTML web based editors which execute your code in your browser, like
JSFiddle and JSBin.  http://jsfiddle.net and http://jsbin.com .  Wouldn't
allowing arbitrary users to execute python code on your server be much more
of a security risk than allowing arbitrary javascript code to run in the
browser?  Doesn't the browser provide some security by segmenting resources
by origin so any javascript code executed by the notebook will not be able
to access resources stored by other sites?  I suppose javascript in the
notebook could be used to crash the browser, or to make the browser send
too many requests to some server, but could it really access user's data?
 Do you mean something else by your security concerns?  Is the notebook
storing sensitive data in the browser's local storage or cookies?

I did find this thing, but it really sounds like overkill to me for a
programmer's tool: https://developers.google.com/caja/docs/about/




On Mon, Dec 31, 2012 at 11:17 AM, Jason Grout
<jason-sage at creativetrax.com>wrote:

> On 12/29/12 11:29 AM, Brian Granger wrote:
> > We don't want to allow notebooks that mix different languages at the
> > CodeCell level.
>
> Of course, cell magics alleviate this restriction, as we can right now
> do %%r, %%cython, etc., to effectively get different languages in
> different cells.
>
> I don't see why we can't have a %%javascript that then just echoes the
> javascript back to the browser to execute.  What would be cool is for
> the syntax highlighting to also change if the cell detects that it is a
> %%r cell, etc.
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121231/f2ab070e/attachment.html>

From bussonniermatthias at gmail.com  Mon Dec 31 12:06:28 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 31 Dec 2012 18:06:28 +0100
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <50E1BAAF.8000802@creativetrax.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
	<CAH4pYpT8_HsOMot9a0AnuRr466b-c=_05ueqv7+nmHzqHjrP6g@mail.gmail.com>
	<50E1BAAF.8000802@creativetrax.com>
Message-ID: <CANJQusUd-EWHw_V-pnH411aSjjExTY86LQ2TD77MjJCVm2+ffw@mail.gmail.com>

Le 31 d?c. 2012 17:27, "Jason Grout" <jason-sage at creativetrax.com> a ?crit :
>
> On 12/29/12 11:29 AM, Brian Granger wrote:
> > We don't want to allow notebooks that mix different languages at the
> > CodeCell level.

>
> Of course, cell magics alleviate this restriction, as we can right now
> do %%r, %%cython, etc., to effectively get different languages in
> different cells.
But that applies only to ipython kernel.

>
> I don't see why we can't have a %%javascript that then just echoes the
> javascript back to the browser to execute.
It alredy exist but is dangerous so will be blocked by the security we will
add.

What would be cool is for
> the syntax highlighting to also change if the cell detects that it is a
> %%r cell, etc.
Doesn't 0.13.1 do it. Dev does.
-- 
Matthias

>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121231/946216a7/attachment.html>

From bussonniermatthias at gmail.com  Mon Dec 31 12:43:41 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 31 Dec 2012 18:43:41 +0100
Subject: [IPython-dev] Notebook CodeCell for editing and executing
	javascript
In-Reply-To: <CAFhnF0D8t5UExpkGeKY7OSpPO6fJnDZNOKJiFhCaL6pVK4cJNA@mail.gmail.com>
References: <CAFhnF0DaU-AyuhR-_uK_jauOSqnRD4GPDMFGVGwR9MzEpvPatw@mail.gmail.com>
	<CAH4pYpT8_HsOMot9a0AnuRr466b-c=_05ueqv7+nmHzqHjrP6g@mail.gmail.com>
	<50E1BAAF.8000802@creativetrax.com>
	<CAFhnF0D8t5UExpkGeKY7OSpPO6fJnDZNOKJiFhCaL6pVK4cJNA@mail.gmail.com>
Message-ID: <CANJQusULKyWqvxnMin9q+rcZSKPTKPcFwt0qCV__PcuSX+4ZNQ@mail.gmail.com>

Short from my phone.

Problem is not executing js in itself.
Problem is executed js could execute python on kernel side.

So we need to prevent inlined js in cell output as notebook store cell
output that woul be executed at load time.

Stripping js from cell output render the ability to display js useless.

Also I suppose everybody trust its own js. But you can't always trust files
you receive from others.

Finally json plugin can be use to developp a plugin that allow to execute
arbitrary JS. It will just not be supported by the core team.
-- 
Matthias
Le 31 d?c. 2012 18:02, "Nissim Karpenstein" <nissimk at gmail.com> a ?crit :

> Syntax highlight changing for the %%language cells sounds closer to what I
> was thinking of.
>
> Can you guys explain to me the security concerns?  There are several
> JS+CSS+HTML web based editors which execute your code in your browser, like
> JSFiddle and JSBin.  http://jsfiddle.net and http://jsbin.com .  Wouldn't
> allowing arbitrary users to execute python code on your server be much more
> of a security risk than allowing arbitrary javascript code to run in the
> browser?  Doesn't the browser provide some security by segmenting resources
> by origin so any javascript code executed by the notebook will not be able
> to access resources stored by other sites?  I suppose javascript in the
> notebook could be used to crash the browser, or to make the browser send
> too many requests to some server, but could it really access user's data?
>  Do you mean something else by your security concerns?  Is the notebook
> storing sensitive data in the browser's local storage or cookies?
>
> I did find this thing, but it really sounds like overkill to me for a
> programmer's tool: https://developers.google.com/caja/docs/about/
>
>
>
>
> On Mon, Dec 31, 2012 at 11:17 AM, Jason Grout <jason-sage at creativetrax.com
> > wrote:
>
>> On 12/29/12 11:29 AM, Brian Granger wrote:
>> > We don't want to allow notebooks that mix different languages at the
>> > CodeCell level.
>>
>> Of course, cell magics alleviate this restriction, as we can right now
>> do %%r, %%cython, etc., to effectively get different languages in
>> different cells.
>>
>> I don't see why we can't have a %%javascript that then just echoes the
>> javascript back to the browser to execute.  What would be cool is for
>> the syntax highlighting to also change if the cell detects that it is a
>> %%r cell, etc.
>>
>> Thanks,
>>
>> Jason
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20121231/e8c367a4/attachment.html>