From ijstokes at alumni.uwaterloo.ca  Sat Jan  2 13:25:41 2016
From: ijstokes at alumni.uwaterloo.ca (Ian Stokes Rees)
Date: Sat, 2 Jan 2016 13:25:41 -0500
Subject: [IPython-dev] How to load an extension into every notebook on a
 server
In-Reply-To: <CADqKjdT=E4pQzOUpNyuYVNmS5_r+2VcDnF7j4ukcyh7irQ80kA@mail.gmail.com>
References: <CADqKjdQo8+-D8zAGYHr5MW1xUqyGQSrk+GKuhWHhgO=eN8QGww@mail.gmail.com>
 <CADqKjdT=E4pQzOUpNyuYVNmS5_r+2VcDnF7j4ukcyh7irQ80kA@mail.gmail.com>
Message-ID: <56881625.3020002@alumni.uwaterloo.ca>

Hi Ted (and Thomas),

At Continuum we are also interested in having pre-configured notebook 
environments and are actively working with the Jupyter team to figure 
out good ways of enabling "universal" configuration of extensions in 
Jupyter 4 (for others: prior to the complete reworking of the 
"extension" mechanism which will arrive with Jupyter 5).

I'm offline now, otherwise I'd provide you with some links to the 
discussion, but you can dig up a reference if you look at the JEP area 
of the Jupyter GitHub repos.  I think this is the link:

https://github.com/jupyter/enhancement-proposals/pull/7

That is probably way too much detail for what you're looking for, but I, 
for one, would like to come up with a solution that would make your 
GeneExpression extension "pre-loadable" with no extra work on the part 
of the user.

Cheers,

Ian

On 12/24/15 8:13 AM, Ted Liefeld wrote:
> /Aside:  Wish I knew how to reply to a thread in the digest on mailman 
> too.../
>
> Thomas
>
> what you says makes sense about having the magic be explicit in the 
> notebook.  For the Docker server though I'd still like to not have to 
> make users remember to insert it themselves  because they have just 
> started the GP-noteook-jupyter container so they (or at least I) would 
> expect it to just work.
>
> So whats your opinion of if I were to modify the jupyter in the 
> container (assuming its possible) to make the default empty python 3 
> notebook have one cell at its top with the magic line in it?
>
> Ted
>
>
>
>     Date: Wed, 23 Dec 2015 22:27:00 +0000
>     From: Thomas Kluyver <takowl at gmail.com <mailto:takowl at gmail.com>>
>     To: IPython developers list <ipython-dev at scipy.org
>     <mailto:ipython-dev at scipy.org>>
>     Subject: Re: [IPython-dev] How to load an extension into every
>             notebook on a  server
>     Message-ID:
>     <CAOvn4qgeSpaVVRW4h+jzn19gB6tTXkuY=QvfK2JLDmqso9MFaQ at mail.gmail.com <mailto:QvfK2JLDmqso9MFaQ at mail.gmail.com>>
>     Content-Type: text/plain; charset="utf-8"
>
>     Looking at the Genepattern code [1], it's using display() to
>     publish data.
>     The notebook interface will only actually use the Javascript that
>     sends if
>     it's coming from code run in a cell, so that part of the loading
>     can't work
>     at startup.
>
>     I think we'd probably encourage you to keep putting the command to
>     load the
>     extension at the top of every notebook, by analogy with a similar
>     situation
>     we had with pylab. It used to be possible to run "ipython notebook
>     --pylab", and have a lot of numpy & matplotlib functions
>     pre-loaded into
>     the namespace. But this produces notebooks which can only be run
>     by someone
>     starting the notebook server in the same way. We found it better
>     to use a
>     '%pylab' magic inside the notebook when we want the namespace set
>     up like
>     that. I think loading the genepattern extension is a similar
>     thing: it's
>     clearer what's going on when that lives inside the notebook,
>     rather than
>     hidden in your config.
>
>     [1]
>     https://github.com/genepattern/genepattern-notebook/blob/master/profile/extensions/genepattern.py
>
>     Thomas
>
>     On 23 December 2015 at 22:06, Ted Liefeld
>     <liefeld at broadinstitute.org <mailto:liefeld at broadinstitute.org>>
>     wrote:
>
>     > I am building a Docker image to have a Jupyter server with the GenePattern
>     > notebook extension in it.  So far so good except that in every notebook I
>     > have to enter
>     >
>     >      %reload_ext genepattern
>     >
>     > into the first cell to make the extension active.  I have been prowling
>     > through the docs and have tried putting this command in lots of places,
>     > none of which have worked (e.g. ~/.jupyter/jupyter_notebook_config.py,
>     >~/.ipython/profile_default/startup/ipython_notebook_config.py) both
>     > normally (ie as above) and also as a config setting
>     > c.InteractiveShellApp.exec_lines = ['%reload_ext genepattern']
>     >
>     > It seems with all the layers of config I am not finding the right one to
>     > insert this at. Can anyone point me in the right direction?  My current
>     > container is using Jupyter 4.0.6, IPython 4.0.1 and Python 3.4.3.
>     >
>     > Thanks
>     >
>     > Ted
>     >
>     >
>
>
> On Wed, Dec 23, 2015 at 2:06 PM, Ted Liefeld 
> <liefeld at broadinstitute.org <mailto:liefeld at broadinstitute.org>> wrote:
>
>     I am building a Docker image to have a Jupyter server with the
>     GenePattern notebook extension in it.  So far so good except that
>     in every notebook I have to enter
>
>          %reload_ext genepattern
>
>     into the first cell to make the extension active.  I have been
>     prowling through the docs and have tried putting this command in
>     lots of places, none of which have worked (e.g.
>     ~/.jupyter/jupyter_notebook_config.py,
>      ~/.ipython/profile_default/startup/ipython_notebook_config.py)
>     both normally (ie as above) and also as a config setting
>             c.InteractiveShellApp.exec_lines = ['%reload_ext genepattern']
>
>     It seems with all the layers of config I am not finding the right
>     one to insert this at. Can anyone point me in the right
>     direction?  My current container is using Jupyter 4.0.6, IPython
>     4.0.1 and Python 3.4.3.
>
>     Thanks
>
>     Ted
>
>
>     -- 
>     Ted Liefeld  UC San Diego
>     Mesirov Lab liefeld at ucsd.edu <mailto:liefeld at ucsd.edu>
>     Office 2A24, BRF-II 858-534-2010 <tel:858-534-2010>
>
>
>
>
> -- 
> Ted Liefeld  UC San Diego
> Mesirov Lab liefeld at ucsd.edu <mailto:liefeld at ucsd.edu>
> Office 2A24, BRF-II  858-534-2010
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev

Ian Stokes-Rees
617.942.0218

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160102/ef390330/attachment.html>

From ijstokes at alumni.uwaterloo.ca  Sun Jan  3 06:58:41 2016
From: ijstokes at alumni.uwaterloo.ca (Ian Stokes Rees)
Date: Sun, 3 Jan 2016 06:58:41 -0500
Subject: [IPython-dev] How to load an extension into every notebook on a
 server
In-Reply-To: <56881625.3020002@alumni.uwaterloo.ca>
References: <CADqKjdQo8+-D8zAGYHr5MW1xUqyGQSrk+GKuhWHhgO=eN8QGww@mail.gmail.com>
 <CADqKjdT=E4pQzOUpNyuYVNmS5_r+2VcDnF7j4ukcyh7irQ80kA@mail.gmail.com>
 <56881625.3020002@alumni.uwaterloo.ca>
Message-ID: <56890CF1.3050206@alumni.uwaterloo.ca>

Ted,

Now that I'm back online, the reference I should have provided was this one:

https://github.com/jupyter/notebook/issues/878

That issue is the core discussion around Jupyter 4.2 modifications to 
support better extension configuration.  A number of PRs have already 
been created and merged, but I believe the "main" open one is:

https://github.com/jupyter/notebook/pull/879

Regards.

Ian

On 1/2/16 1:25 PM, Ian Stokes Rees wrote:
> Hi Ted (and Thomas),
>
> At Continuum we are also interested in having pre-configured notebook 
> environments and are actively working with the Jupyter team to figure 
> out good ways of enabling "universal" configuration of extensions in 
> Jupyter 4 (for others: prior to the complete reworking of the 
> "extension" mechanism which will arrive with Jupyter 5).
>
> I'm offline now, otherwise I'd provide you with some links to the 
> discussion, but you can dig up a reference if you look at the JEP area 
> of the Jupyter GitHub repos.  I think this is the link:
>
> https://github.com/jupyter/enhancement-proposals/pull/7
>
> That is probably way too much detail for what you're looking for, but 
> I, for one, would like to come up with a solution that would make your 
> GeneExpression extension "pre-loadable" with no extra work on the part 
> of the user.
>
> Cheers,
>
> Ian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160103/02f40e59/attachment.html>

From jason at jasongrout.org  Tue Jan  5 14:19:06 2016
From: jason at jasongrout.org (Jason Grout)
Date: Tue, 5 Jan 2016 12:19:06 -0700
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
Message-ID: <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>

(cross-posting to ipython-dev)

Jon,

At the recent San Francisco meetings, we talked about this.  What do you
think about:

1. keeping track of the size of the io messages sent from any specific
kernel execution
2. When the total size of io reaches some specific size
(user-configurable), transmitting a special "throwing away output, but
here's how to save the output to a file if you want in the future, or how
to increase the limit" message
3. keep a running buffer of the last bit of output attempted to be sent,
and send it when the execution finishes (so basically a ring buffer that
overwrites the oldest message)

This:

* allows small output through
* provides an explanatory message
* provides the last bit of output as well

One thing to figure out: a limit on size of output that is text may not be
appropriate for output that is images, etc.

Thanks,

Jason


On Tue, Jan 5, 2016 at 12:11 PM, Jason Grout <jason at jasongrout.org> wrote:

>
> ---------- Forwarded message ----------
> From: Jonathan Frederic <jon.freder at gmail.com>
> Date: Tue, Jan 5, 2016 at 11:42 AM
> Subject: Re: [sage-devel] Re: Jupyter notebook by default?
> To: Jason Grout <grout.jason at gmail.com>
> Cc: sage-devel <sage-devel at googlegroups.com>
>
>
> Jason,
>
> Thanks for pulling me in on this.
>
> William,
>
> I agree, getting a bunch of people to agree on stuff can seem impossible.
> However, you mention Sage offers a couple options to mitigate output
> overflows, can you point me to those options?  The Jupyter Notebook should
> provide multiple options too - this will also make it easier for everyone
> to agree.
>
> Also, in you experience, which of these options work the best?
>
> I was thinking initially of doing something simple, like hard limiting
> data/time, then printing an error if that's exceeded.  In the Jupyter
> Notebook, we have to worry about
> - Too many messages sent on the websocket
> - The notebook json file growing too large and consequently becoming
> unopenable
> - Too much data being appended to the DOM, crashing the browser
>
>
> Thanks!
> -Jon
>
> On Tue, Jan 5, 2016 at 10:19 AM, Jason Grout <grout.jason at gmail.com>
> wrote:
>
>>
>>
>> On Tuesday, January 5, 2016 at 8:17:45 AM UTC-7, William wrote:
>>>
>>>
>>> One example of a subtle feature in Sage (notebook and worksheets) not
>>> in Jupyter, which I was just reminded of, is output limiting.  In Sage
>>> there are numerous rules/options to deal with people doing stuff like:
>>>
>>> while True:
>>>    print "hi!"
>>>
>>> ... which is exactly what students will tend to do by accident...
>>> Jupyter doesn't deal with this, but it might not be too hard to
>>> implement in theory.  One of the main problems is figuring out what
>>> the arbitrary rate limiting defaults "should" be; it's arbitrary, and
>>> depends a lot on whether everything is local, over the web, etc. so
>>> getting a bunch of people to agree is hard, which might mean they will
>>> never implement anything.
>>>
>>
>>
>> William,
>>
>> Jon Frederic in the Jupyter dev meeting happening right now said that he
>> will be working on output limiting as one of his next things.
>>
>> Jason
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160105/3f4d56aa/attachment.html>

From wstein at gmail.com  Tue Jan  5 14:32:05 2016
From: wstein at gmail.com (William Stein)
Date: Tue, 5 Jan 2016 11:32:05 -0800
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
 <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
Message-ID: <CACLE5GAOyHABesZmZyBb_W8eOcmWYCBCD0DLqV1apwEC8PJH+A@mail.gmail.com>

On Tue, Jan 5, 2016 at 11:19 AM, Jason Grout <jason at jasongrout.org> wrote:

> (cross-posting to ipython-dev)
>
> Jon,
>
> At the recent San Francisco meetings, we talked about this.  What do you
> think about:
>
> 1. keeping track of the size of the io messages sent from any specific
> kernel execution
> 2. When the total size of io reaches some specific size
> (user-configurable), transmitting a special "throwing away output, but
> here's how to save the output to a file if you want in the future, or how
> to increase the limit" message
> 3. keep a running buffer of the last bit of output attempted to be sent,
> and send it when the execution finishes (so basically a ring buffer that
> overwrites the oldest message)
>
> This:
>
> * allows small output through
> * provides an explanatory message
> * provides the last bit of output as well
>
> One thing to figure out: a limit on size of output that is text may not be
> appropriate for output that is images, etc.
>

The above strategy is a good start, but I've found it to be too naive in
practice.  For example, in SMC there is a different limit on the amount of
output that will be rendered with mathjax, versus the amount of stdout
output.  This is because mathjax rendering is vastly more resource
intensive than text rendering.  Also, if your graphics images are just
contents of messages in base64 (say), they can be relatively large, but
easy to render quickly.

You might also want to distinguish between local users and people using
Jupyter via a remote server.  If everything is running on your laptop, the
network situation is completely different than a remote server talking to a
cell phone.

I know you wrote "user-configurable" above, but it's a possibly bad sign
when user configuration is required.

Have fun at the Jupyter dev meeting!

William



>
> Thanks,
>
> Jason
>
>
> On Tue, Jan 5, 2016 at 12:11 PM, Jason Grout <jason at jasongrout.org> wrote:
>
>>
>> ---------- Forwarded message ----------
>> From: Jonathan Frederic <jon.freder at gmail.com>
>> Date: Tue, Jan 5, 2016 at 11:42 AM
>> Subject: Re: [sage-devel] Re: Jupyter notebook by default?
>> To: Jason Grout <grout.jason at gmail.com>
>> Cc: sage-devel <sage-devel at googlegroups.com>
>>
>>
>> Jason,
>>
>> Thanks for pulling me in on this.
>>
>> William,
>>
>> I agree, getting a bunch of people to agree on stuff can seem
>> impossible.  However, you mention Sage offers a couple options to mitigate
>> output overflows, can you point me to those options?  The Jupyter Notebook
>> should provide multiple options too - this will also make it easier for
>> everyone to agree.
>>
>> Also, in you experience, which of these options work the best?
>>
>> I was thinking initially of doing something simple, like hard limiting
>> data/time, then printing an error if that's exceeded.  In the Jupyter
>> Notebook, we have to worry about
>> - Too many messages sent on the websocket
>> - The notebook json file growing too large and consequently becoming
>> unopenable
>> - Too much data being appended to the DOM, crashing the browser
>>
>>
>> Thanks!
>> -Jon
>>
>> On Tue, Jan 5, 2016 at 10:19 AM, Jason Grout <grout.jason at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Tuesday, January 5, 2016 at 8:17:45 AM UTC-7, William wrote:
>>>>
>>>>
>>>> One example of a subtle feature in Sage (notebook and worksheets) not
>>>> in Jupyter, which I was just reminded of, is output limiting.  In Sage
>>>> there are numerous rules/options to deal with people doing stuff like:
>>>>
>>>> while True:
>>>>    print "hi!"
>>>>
>>>> ... which is exactly what students will tend to do by accident...
>>>> Jupyter doesn't deal with this, but it might not be too hard to
>>>> implement in theory.  One of the main problems is figuring out what
>>>> the arbitrary rate limiting defaults "should" be; it's arbitrary, and
>>>> depends a lot on whether everything is local, over the web, etc. so
>>>> getting a bunch of people to agree is hard, which might mean they will
>>>> never implement anything.
>>>>
>>>
>>>
>>> William,
>>>
>>> Jon Frederic in the Jupyter dev meeting happening right now said that he
>>> will be working on output limiting as one of his next things.
>>>
>>> Jason
>>>
>>
>>
>>
>


-- 
William (http://wstein.org)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160105/f2e37367/attachment.html>

From liefeld at broadinstitute.org  Tue Jan  5 17:35:42 2016
From: liefeld at broadinstitute.org (Ted Liefeld)
Date: Tue, 5 Jan 2016 14:35:42 -0800
Subject: [IPython-dev] How to load an extension into every notebook on a
	server
In-Reply-To: <56890CF1.3050206@alumni.uwaterloo.ca>
References: <CADqKjdQo8+-D8zAGYHr5MW1xUqyGQSrk+GKuhWHhgO=eN8QGww@mail.gmail.com>
 <CADqKjdT=E4pQzOUpNyuYVNmS5_r+2VcDnF7j4ukcyh7irQ80kA@mail.gmail.com>
 <56881625.3020002@alumni.uwaterloo.ca> <56890CF1.3050206@alumni.uwaterloo.ca>
Message-ID: <CADqKjdQz3X6PG-s=WQnBrcOHRqouWnMGs2YQX8zYL3OhfReBQw@mail.gmail.com>

Ian

So it seems that (delayed response due to the holiday season) its not
actually possible to do this now using the example in 879
<https://github.com/jupyter/notebook/issues/879> unless I go to a
development build, or is there currently a way to do that by editing my
notebook.json file which seems to be implied by the discussion at the top
of 878 <https://github.com/jupyter/notebook/issues/878>.  For the moment
this would work fine for our container since it is single user and we
control the files in the container.

Speaking of which I cannot seem to figure out the right mojo to get the
notebook.json file (in the intro in 878 it refers to paths.jupyter_path('
nbextensions') but the "paths" variable is not present in my notebook, do I
need to run python in the container to access this variable?

Also, what is the root path for loading extensions like this?  The examples
don't make that clear, so I have this at the moment (note this is in a
docker container so its not really running as root)

# pwd
/root/.jupyter/nbconfig
# more notebook.json
{
  "load_extensions" : {
       "genepattern" : true
       }

}


while the extension installed (via pip) in

/root/.jupyter/custom/genepattern


This, of course, does not actually work though

Thanks

Ted

On Sun, Jan 3, 2016 at 3:58 AM, Ian Stokes Rees <
ijstokes at alumni.uwaterloo.ca> wrote:

> Ted,
>
> Now that I'm back online, the reference I should have provided was this
> one:
>
> https://github.com/jupyter/notebook/issues/878
>
> That issue is the core discussion around Jupyter 4.2 modifications to
> support better extension configuration.  A number of PRs have already been
> created and merged, but I believe the "main" open one is:
>
> https://github.com/jupyter/notebook/pull/879
>
> Regards.
>
> Ian
>
> On 1/2/16 1:25 PM, Ian Stokes Rees wrote:
>
> Hi Ted (and Thomas),
>
> At Continuum we are also interested in having pre-configured notebook
> environments and are actively working with the Jupyter team to figure out
> good ways of enabling "universal" configuration of extensions in Jupyter 4
> (for others: prior to the complete reworking of the "extension" mechanism
> which will arrive with Jupyter 5).
>
> I'm offline now, otherwise I'd provide you with some links to the
> discussion, but you can dig up a reference if you look at the JEP area of
> the Jupyter GitHub repos.  I think this is the link:
>
> https://github.com/jupyter/enhancement-proposals/pull/7
>
> That is probably way too much detail for what you're looking for, but I,
> for one, would like to come up with a solution that would make your
> GeneExpression extension "pre-loadable" with no extra work on the part of
> the user.
>
> Cheers,
>
> Ian
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Ted Liefeld                                      UC San Diego
Mesirov Lab                                    liefeld at ucsd.edu

Office 2A24, BRF-II                        858-534-2010
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160105/5a86d145/attachment.html>

From moorepants at gmail.com  Wed Jan  6 00:13:52 2016
From: moorepants at gmail.com (Jason Moore)
Date: Tue, 5 Jan 2016 21:13:52 -0800
Subject: [IPython-dev] Supporting the qtconsole and notebook display system
	simultaneously
Message-ID: <CAP7f1AiYBVBmk_NRgh03Uw4vBknazZhbUxH2Ar63xV6igsudJg@mail.gmail.com>

SymPy has a comprehensive printing system and we leverage IPython's
qtconsole and notebook display abilities to show mathematics in unicode,
mathjax, and png outputs. Typically all three of these are generated and
the IPython frontend can choose what to display.

We've hit a bug that we can't figure out how to fix though. Matplotlib has
a minimal LaTeX implementation and can output PNGs. If a user doesn't have
a full LaTeX suit installed we fall back to that. But since matplotlib
doesn't handle LaTeX environments, it can't render things like matrices.

Now if a qtconsole user tries to display a matrix, this fails if matplotlib
is the only available renderer. But if the user is using the notebook then
the matplotlib renderer will fail but mathjax will print properly. We'd
like to fall back to unicode printing if matplotlib fails to print **only
in the qtconsole**, but render with mathjax in the notebook.

So, as far as we can tell, we need to know which frontend is trying to
display our object so that we can handle errors properly. We haven't been
able to think of a workaround that is agnostic of the frontend.

There is some discussion in this PR:
https://github.com/sympy/sympy/pull/9976 (among others).

Any suggestions?

Jason
moorepants.info
+01 530-601-9791
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160105/7518a8f6/attachment.html>

From bussonniermatthias at gmail.com  Wed Jan  6 02:42:37 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Wed, 6 Jan 2016 08:42:37 +0100
Subject: [IPython-dev] Supporting the qtconsole and notebook display
	system simultaneously
In-Reply-To: <CAP7f1AiYBVBmk_NRgh03Uw4vBknazZhbUxH2Ar63xV6igsudJg@mail.gmail.com>
References: <CAP7f1AiYBVBmk_NRgh03Uw4vBknazZhbUxH2Ar63xV6igsudJg@mail.gmail.com>
Message-ID: <61E4E307-67BF-4F7B-A6E0-73D64314AE5B@gmail.com>


> On Jan 6, 2016, at 06:13, Jason Moore <moorepants at gmail.com> wrote:
> 
> SymPy has a comprehensive printing system and we leverage IPython's qtconsole and notebook display abilities to show mathematics in unicode, mathjax, and png outputs. Typically all three of these are generated and the IPython frontend can choose what to display.
> 
> We've hit a bug that we can't figure out how to fix though. Matplotlib has a minimal LaTeX implementation and can output PNGs. If a user doesn't have a full LaTeX suit installed we fall back to that. But since matplotlib doesn't handle LaTeX environments, it can't render things like matrices.
> 
> Now if a qtconsole user tries to display a matrix, this fails if matplotlib is the only available renderer. But if the user is using the notebook then the matplotlib renderer will fail but mathjax will print properly. We'd like to fall back to unicode printing if matplotlib fails to print **only in the qtconsole**, but render with mathjax in the notebook.

Technically you can only distinguish a non-zmq from a ZMQ front-end, so **only** in qtconsole is not reliable.


> So, as far as we can tell, we need to know which frontend is trying to display our object so that we can handle errors properly. We haven't been able to think of a workaround that is agnostic of the frontend.

Can?t you register 2 display formatter (tex and pretty unicode) so that it display correctly on both ?

Though IIUC there might also be a bug where our Tex to Png that should not return empty PNG.
-- 
M

> 
> There is some discussion in this PR: https://github.com/sympy/sympy/pull/9976 <https://github.com/sympy/sympy/pull/9976> (among others).
> 
> Any suggestions?
> 
> Jason
> moorepants.info <http://moorepants.info/>
> +01 530-601-9791
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160106/8e23cf40/attachment.html>

From vbraun.name at gmail.com  Wed Jan  6 04:05:32 2016
From: vbraun.name at gmail.com (Volker Braun)
Date: Wed, 6 Jan 2016 01:05:32 -0800 (PST)
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
 <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
Message-ID: <59b5c876-b745-4352-9b7e-38f9fe3f2800@googlegroups.com>

IMHO output capture into a web browser isn't really different from the 
scrollback buffer of a terminal. We obviously enjoy the infinite scrollback 
but do not want an unbounded drawing surface in the terminal (= dom nodes 
in the web browser). And certainly nobody wants a piece of their output 
discarded in a long-running computation. 

The technical implementation is virtual scrolling, this is what the 
terminal does and this is how the browser should do it, too.


On Tuesday, January 5, 2016 at 8:21:53 PM UTC+1, Jason Grout wrote:
>
> (cross-posting to ipython-dev)
>
> Jon,
>
> At the recent San Francisco meetings, we talked about this.  What do you 
> think about:
>
> 1. keeping track of the size of the io messages sent from any specific 
> kernel execution
> 2. When the total size of io reaches some specific size 
> (user-configurable), transmitting a special "throwing away output, but 
> here's how to save the output to a file if you want in the future, or how 
> to increase the limit" message
> 3. keep a running buffer of the last bit of output attempted to be sent, 
> and send it when the execution finishes (so basically a ring buffer that 
> overwrites the oldest message)
>
> This:
>
> * allows small output through
> * provides an explanatory message
> * provides the last bit of output as well
>
> One thing to figure out: a limit on size of output that is text may not be 
> appropriate for output that is images, etc.
>
> Thanks,
>
> Jason
>
>
> On Tue, Jan 5, 2016 at 12:11 PM, Jason Grout <ja... at jasongrout.org 
> <javascript:>> wrote:
>
>>
>> ---------- Forwarded message ----------
>> From: Jonathan Frederic <jon.f... at gmail.com <javascript:>>
>> Date: Tue, Jan 5, 2016 at 11:42 AM
>> Subject: Re: [sage-devel] Re: Jupyter notebook by default?
>> To: Jason Grout <grout... at gmail.com <javascript:>>
>> Cc: sage-devel <sage-... at googlegroups.com <javascript:>>
>>
>>
>> Jason,
>>
>> Thanks for pulling me in on this.  
>>
>> William,
>>
>> I agree, getting a bunch of people to agree on stuff can seem 
>> impossible.  However, you mention Sage offers a couple options to mitigate 
>> output overflows, can you point me to those options?  The Jupyter Notebook 
>> should provide multiple options too - this will also make it easier for 
>> everyone to agree.
>>
>> Also, in you experience, which of these options work the best?  
>>
>> I was thinking initially of doing something simple, like hard limiting 
>> data/time, then printing an error if that's exceeded.  In the Jupyter 
>> Notebook, we have to worry about
>> - Too many messages sent on the websocket
>> - The notebook json file growing too large and consequently becoming 
>> unopenable
>> - Too much data being appended to the DOM, crashing the browser
>>
>>
>> Thanks!
>> -Jon
>>
>> On Tue, Jan 5, 2016 at 10:19 AM, Jason Grout <grout... at gmail.com 
>> <javascript:>> wrote:
>>
>>>
>>>
>>> On Tuesday, January 5, 2016 at 8:17:45 AM UTC-7, William wrote:
>>>>
>>>>
>>>> One example of a subtle feature in Sage (notebook and worksheets) not 
>>>> in Jupyter, which I was just reminded of, is output limiting.  In Sage 
>>>> there are numerous rules/options to deal with people doing stuff like: 
>>>>
>>>> while True: 
>>>>    print "hi!" 
>>>>
>>>> ... which is exactly what students will tend to do by accident... 
>>>> Jupyter doesn't deal with this, but it might not be too hard to 
>>>> implement in theory.  One of the main problems is figuring out what 
>>>> the arbitrary rate limiting defaults "should" be; it's arbitrary, and 
>>>> depends a lot on whether everything is local, over the web, etc. so 
>>>> getting a bunch of people to agree is hard, which might mean they will 
>>>> never implement anything. 
>>>>
>>>
>>>
>>> William,
>>>
>>> Jon Frederic in the Jupyter dev meeting happening right now said that he 
>>> will be working on output limiting as one of his next things.
>>>
>>> Jason
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160106/66c89112/attachment.html>

From benjaminrk at gmail.com  Wed Jan  6 05:42:53 2016
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 6 Jan 2016 11:42:53 +0100
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <59b5c876-b745-4352-9b7e-38f9fe3f2800@googlegroups.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
 <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
 <59b5c876-b745-4352-9b7e-38f9fe3f2800@googlegroups.com>
Message-ID: <CAHNn8BVGfbVeziyovM-W3vGa7kn00FdGTqLZk14iLYBdxR8Pvg@mail.gmail.com>

On Wed, Jan 6, 2016 at 10:05 AM, Volker Braun <vbraun.name at gmail.com> wrote:

IMHO output capture into a web browser isn't really different from the
> scrollback buffer of a terminal. We obviously enjoy the infinite scrollback
> but do not want an unbounded drawing surface in the terminal (= dom nodes
> in the web browser). And certainly nobody wants a piece of their output
> discarded in a long-running computation.
>
> The technical implementation is virtual scrolling, this is what the
> terminal does and this is how the browser should do it, too.
>
Jon mentioned that there are a few levels for large output to cause
problems. The lowest bar is putting the output on the page, which is by far
the easiest to hit, causing an unresponsive browser. This is the level that
can be addressed by virtual scrolling / truncating output in UI.
Fortunately, it?s also the easiest one to implement.

If we truncate instead of virtual-scroll, then we have a choice for whether
truncated output is included in the document or not, which alleviates the
problem of opening notebooks that have a problematic amount of output. But
it?s putting that on the page that?s ~always the problem, not loading the
notebook JSON itself, so I?m somewhat less concerned about that.

The next level where it can cause problems is the output coming over the
network in the first place. We can throttle this in the notebook server, as
was implemented for 4.2 <https://github.com/jupyter/notebook/pull/506>
months ago. Again, this moves the bar for when output causes trouble, but
isn?t a complete solution. Dumping truncated output to a file is
complicated a bit by the separations we have in place, but it should be
doable.

-MinRK


>
> On Tuesday, January 5, 2016 at 8:21:53 PM UTC+1, Jason Grout wrote:
>>
>> (cross-posting to ipython-dev)
>>
>> Jon,
>>
>> At the recent San Francisco meetings, we talked about this.  What do you
>> think about:
>>
>> 1. keeping track of the size of the io messages sent from any specific
>> kernel execution
>> 2. When the total size of io reaches some specific size
>> (user-configurable), transmitting a special "throwing away output, but
>> here's how to save the output to a file if you want in the future, or how
>> to increase the limit" message
>> 3. keep a running buffer of the last bit of output attempted to be sent,
>> and send it when the execution finishes (so basically a ring buffer that
>> overwrites the oldest message)
>>
>> This:
>>
>> * allows small output through
>> * provides an explanatory message
>> * provides the last bit of output as well
>>
>> One thing to figure out: a limit on size of output that is text may not
>> be appropriate for output that is images, etc.
>>
>> Thanks,
>>
>> Jason
>>
>>
>> On Tue, Jan 5, 2016 at 12:11 PM, Jason Grout <ja... at jasongrout.org>
>> wrote:
>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Jonathan Frederic <jon.f... at gmail.com>
>>> Date: Tue, Jan 5, 2016 at 11:42 AM
>>> Subject: Re: [sage-devel] Re: Jupyter notebook by default?
>>> To: Jason Grout <grout... at gmail.com>
>>> Cc: sage-devel <sage-... at googlegroups.com>
>>>
>>>
>>> Jason,
>>>
>>> Thanks for pulling me in on this.
>>>
>>> William,
>>>
>>> I agree, getting a bunch of people to agree on stuff can seem
>>> impossible.  However, you mention Sage offers a couple options to mitigate
>>> output overflows, can you point me to those options?  The Jupyter Notebook
>>> should provide multiple options too - this will also make it easier for
>>> everyone to agree.
>>>
>>> Also, in you experience, which of these options work the best?
>>>
>>> I was thinking initially of doing something simple, like hard limiting
>>> data/time, then printing an error if that's exceeded.  In the Jupyter
>>> Notebook, we have to worry about
>>> - Too many messages sent on the websocket
>>> - The notebook json file growing too large and consequently becoming
>>> unopenable
>>> - Too much data being appended to the DOM, crashing the browser
>>>
>>>
>>> Thanks!
>>> -Jon
>>>
>>> On Tue, Jan 5, 2016 at 10:19 AM, Jason Grout <grout... at gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Tuesday, January 5, 2016 at 8:17:45 AM UTC-7, William wrote:
>>>>>
>>>>>
>>>>> One example of a subtle feature in Sage (notebook and worksheets) not
>>>>> in Jupyter, which I was just reminded of, is output limiting.  In Sage
>>>>> there are numerous rules/options to deal with people doing stuff like:
>>>>>
>>>>> while True:
>>>>>    print "hi!"
>>>>>
>>>>> ... which is exactly what students will tend to do by accident...
>>>>> Jupyter doesn't deal with this, but it might not be too hard to
>>>>> implement in theory.  One of the main problems is figuring out what
>>>>> the arbitrary rate limiting defaults "should" be; it's arbitrary, and
>>>>> depends a lot on whether everything is local, over the web, etc. so
>>>>> getting a bunch of people to agree is hard, which might mean they will
>>>>> never implement anything.
>>>>>
>>>>
>>>>
>>>> William,
>>>>
>>>> Jon Frederic in the Jupyter dev meeting happening right now said that
>>>> he will be working on output limiting as one of his next things.
>>>>
>>>> Jason
>>>>
>>>
>>>
>>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
> ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160106/6f6e217d/attachment.html>

From benjaminrk at gmail.com  Wed Jan  6 07:16:51 2016
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 6 Jan 2016 13:16:51 +0100
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <fe6aa349-bc3a-4c48-a3ed-de5044c5a9fe@googlegroups.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
 <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
 <59b5c876-b745-4352-9b7e-38f9fe3f2800@googlegroups.com>
 <76291a54-daea-4aeb-ae11-1047bdf291c8@googlegroups.com>
 <fe6aa349-bc3a-4c48-a3ed-de5044c5a9fe@googlegroups.com>
Message-ID: <CAHNn8BVRpw3SCdO78hqJt23fSZ_iss+J_+mFgBrTzEqj1GZNqQ@mail.gmail.com>

On Wed, Jan 6, 2016 at 1:02 PM, Volker Braun <vbraun.name at gmail.com> wrote:

> On Wednesday, January 6, 2016 at 11:55:36 AM UTC+1, Min RK wrote:
>>
>> If we truncate instead of virtual-scroll, then we have a choice for
>> whether truncated output is included in the document or not, which
>> alleviates the problem of opening notebooks that have a problematic amount
>> of output
>>
>
> There is no fundamental problem with large amounts of output (really, any
> content), and there is essentially only a single way to do it right:
>
> The view (dom) needs only a fixed number of dom nodes for a virtual scroll.
>
> The in-browser view model can lazily load the current scroll position,
> with a suitable cache. Fixed amount of browser JS memory.
>
> The server can just mmap the output file, or alternatively seek around in
> the file. With a suitable index. Fixed amount of server-side memory.
>

Files aren't used for output. The filesystem should only be involved, if at
all, in the exceptional case of output overflow.


>
> The kernel has to block if the notebook server can't append output fast
> enough, thats normal flow control just like in a pipe. Fixed memory usage
> in the kernel.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sage-devel/8erxWppKxXM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> sage-devel+unsubscribe at googlegroups.com.
> To post to this group, send email to sage-devel at googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160106/f514ae7d/attachment.html>

From vbraun.name at gmail.com  Wed Jan  6 07:46:01 2016
From: vbraun.name at gmail.com (Volker Braun)
Date: Wed, 6 Jan 2016 04:46:01 -0800 (PST)
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <CAHNn8BVRpw3SCdO78hqJt23fSZ_iss+J_+mFgBrTzEqj1GZNqQ@mail.gmail.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
 <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
 <59b5c876-b745-4352-9b7e-38f9fe3f2800@googlegroups.com>
 <76291a54-daea-4aeb-ae11-1047bdf291c8@googlegroups.com>
 <fe6aa349-bc3a-4c48-a3ed-de5044c5a9fe@googlegroups.com>
 <CAHNn8BVRpw3SCdO78hqJt23fSZ_iss+J_+mFgBrTzEqj1GZNqQ@mail.gmail.com>
Message-ID: <c17acd7e-78a3-4067-a42b-89131d215837@googlegroups.com>

On Wednesday, January 6, 2016 at 1:17:12 PM UTC+1, Min RK wrote:
>
> Files aren't used for output. The filesystem should only be involved, if 
> at all, in the exceptional case of output overflow.
>

Everything is a file of sorts... map is just ram with filesystem backing. 

You can put large stuff that you don't continuously access into ram (which 
will then be paged out -> swapfs) or you put it into a temp file (either 
tmpfs->swap or disk). However you call it, large data that is not 
continuously accessed must end up on the disk because thats what it is good 
for.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160106/839820ee/attachment.html>

From benjaminrk at gmail.com  Thu Jan  7 05:48:05 2016
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 7 Jan 2016 11:48:05 +0100
Subject: [IPython-dev] [sage-devel] Re: Jupyter notebook by default?
In-Reply-To: <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
References: <5673E187.8080102@cage.ugent.be>
 <499de9f3-e9f9-40eb-8dbb-5674eeba9705@googlegroups.com>
 <653b4de9-f5c6-4528-a963-5f1d2c944c70@googlegroups.com>
 <b3f859c8-f7bf-4972-8094-cd0c42adb82d@googlegroups.com>
 <ca97aa46-d91d-4a24-88aa-51de4c0d5cf9@googlegroups.com>
 <12b8f87d-6e51-4ea0-a033-46e5ce686cf9@googlegroups.com>
 <217e530c-8808-433e-a4f2-946266b54f35@googlegroups.com>
 <3d8e6d3c-2a06-469f-a4ca-4b2ca88815ef@googlegroups.com>
 <CACLE5GCyethVtLDFtnZXHXiYcM2e993MkO-yOzNDcu+gLbGMcw@mail.gmail.com>
 <c94d645b-2c72-4ec8-a3f2-c4b416fa67ca@googlegroups.com>
 <CACLE5GBJ0NBm1NY=HU_RkowBabK=FC-jQkfXT7QSyxJV3TzBYg@mail.gmail.com>
 <296621c4-e568-40f1-af31-ed30193129df@googlegroups.com>
 <CACLE5GBbfW6XJ8ODrVa=CO6uOKu=UqqV0HPQ3JB4+LfUTyzgOA@mail.gmail.com>
 <5a340c99-9630-41df-9a88-279dc79d2e41@googlegroups.com>
 <CAAoBLw2Scu=OHsDuf6JJiyN2d5eD3XhW0n7THeqaX864SbCFXA@mail.gmail.com>
 <CAPDWZHwOCXBUgu_2Z=UHhto-K2Ko+eoKbYwGihgp0Yz3LHBJ8Q@mail.gmail.com>
 <CAPDWZHx_WY9jpk4AvhDw_fKv7Q-7+A8iS2QfENWbK-zmj9dXAA@mail.gmail.com>
Message-ID: <CAHNn8BWnrvJhYnKqfMtf82fuGXMLPWfB5=cF=5cKhzuwFVXaUQ@mail.gmail.com>

Thanks Jason for cross-posting.

Since the issue of funding was brought up, I think supporting projects like
this is exactly the sort of thing we should be doing with the funding we
have, whether the work sits on the Jupyter or Sage side (I assume there
will be both). It?s a bit tricky to keep track of all the points in an
email thread, but if we could aggregate the things that are blockers and
the things that would be nice, especially changes you need from Jupyter, we
should be able to start ticking boxes.

A summary of what I?ve seen so far:

   - sage interacts
   - language cells
   - document conversion from sagenb to ipynb
   - low-level output capturing
   - gracefully handling large output

Some comments:

Re: language cells, I assume it?s referring to things like %%bash, %%R, and
%%cython. While these *look* similar, there is a significant difference in
how they are implemented. For instance, the R magic (provided by rpy2) runs
an R interpreter in-memory, and talks to it, capturing output, etc.. Where
many of these magics, such as bash, ruby, perl, come from is some ?script
magic? machinery in IPython, which populates the default magics with
shortcuts to running a script in a given interpreter. They are essentially
shortcuts to cat <cell> | <interpreter>. It?s not a fundamental limitation,
or anything dire like that. If sage has an implementation of running code
in a persistent alternate interpreter, then it should not be much work to
represent that in magics, since cell magics are any Python functions called
with two string arguments (the rest of the line and the cell), and can be
defined at any time, for instance:

def mymagic(line, cell):
    do_stuff_with(cell)

get_ipython().register_magic_function(mymagic, 'cell')

Re: output capturing, Thomas Kluyver and I were at CERN last month working
on the Cling kernel, and one of the things we did was C-level capturing of
output. Now that we have that working, integrating it into the IPython
kernel should not be much work, and if it?s really important, libraries can
use the same technique themselves without waiting for IPython to catch up.

Interacts are perhaps the hardest piece. I think it should be doable to get
sage?s own interacts working in the notebook, rather than forcing people to
adopt the much more basic interact provided by the IPython widgets.

I can?t speak to the UI transition part of the problem whenever you change
defaults, which is a big challenge, but I think we can at least mitigate
most of the things on the Jupyter side that are getting in your way.

-MinRK
?

On Tue, Jan 5, 2016 at 8:19 PM, Jason Grout <jason at jasongrout.org> wrote:

> (cross-posting to ipython-dev)
>
> Jon,
>
> At the recent San Francisco meetings, we talked about this.  What do you
> think about:
>
> 1. keeping track of the size of the io messages sent from any specific
> kernel execution
> 2. When the total size of io reaches some specific size
> (user-configurable), transmitting a special "throwing away output, but
> here's how to save the output to a file if you want in the future, or how
> to increase the limit" message
> 3. keep a running buffer of the last bit of output attempted to be sent,
> and send it when the execution finishes (so basically a ring buffer that
> overwrites the oldest message)
>
> This:
>
> * allows small output through
> * provides an explanatory message
> * provides the last bit of output as well
>
> One thing to figure out: a limit on size of output that is text may not be
> appropriate for output that is images, etc.
>
> Thanks,
>
> Jason
>
>
> On Tue, Jan 5, 2016 at 12:11 PM, Jason Grout <jason at jasongrout.org> wrote:
>
>>
>> ---------- Forwarded message ----------
>> From: Jonathan Frederic <jon.freder at gmail.com>
>> Date: Tue, Jan 5, 2016 at 11:42 AM
>> Subject: Re: [sage-devel] Re: Jupyter notebook by default?
>> To: Jason Grout <grout.jason at gmail.com>
>> Cc: sage-devel <sage-devel at googlegroups.com>
>>
>>
>> Jason,
>>
>> Thanks for pulling me in on this.
>>
>> William,
>>
>> I agree, getting a bunch of people to agree on stuff can seem
>> impossible.  However, you mention Sage offers a couple options to mitigate
>> output overflows, can you point me to those options?  The Jupyter Notebook
>> should provide multiple options too - this will also make it easier for
>> everyone to agree.
>>
>> Also, in you experience, which of these options work the best?
>>
>> I was thinking initially of doing something simple, like hard limiting
>> data/time, then printing an error if that's exceeded.  In the Jupyter
>> Notebook, we have to worry about
>> - Too many messages sent on the websocket
>> - The notebook json file growing too large and consequently becoming
>> unopenable
>> - Too much data being appended to the DOM, crashing the browser
>>
>>
>> Thanks!
>> -Jon
>>
>> On Tue, Jan 5, 2016 at 10:19 AM, Jason Grout <grout.jason at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Tuesday, January 5, 2016 at 8:17:45 AM UTC-7, William wrote:
>>>>
>>>>
>>>> One example of a subtle feature in Sage (notebook and worksheets) not
>>>> in Jupyter, which I was just reminded of, is output limiting.  In Sage
>>>> there are numerous rules/options to deal with people doing stuff like:
>>>>
>>>> while True:
>>>>    print "hi!"
>>>>
>>>> ... which is exactly what students will tend to do by accident...
>>>> Jupyter doesn't deal with this, but it might not be too hard to
>>>> implement in theory.  One of the main problems is figuring out what
>>>> the arbitrary rate limiting defaults "should" be; it's arbitrary, and
>>>> depends a lot on whether everything is local, over the web, etc. so
>>>> getting a bunch of people to agree is hard, which might mean they will
>>>> never implement anything.
>>>>
>>>
>>>
>>> William,
>>>
>>> Jon Frederic in the Jupyter dev meeting happening right now said that he
>>> will be working on output limiting as one of his next things.
>>>
>>> Jason
>>>
>>
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160107/c9568374/attachment.html>

From ccordoba12 at gmail.com  Thu Jan  7 08:39:23 2016
From: ccordoba12 at gmail.com (=?UTF-8?Q?Carlos_C=c3=b3rdoba?=)
Date: Thu, 7 Jan 2016 08:39:23 -0500
Subject: [IPython-dev] Supporting the qtconsole and notebook display
 system simultaneously
In-Reply-To: <CAP7f1AiYBVBmk_NRgh03Uw4vBknazZhbUxH2Ar63xV6igsudJg@mail.gmail.com>
References: <CAP7f1AiYBVBmk_NRgh03Uw4vBknazZhbUxH2Ar63xV6igsudJg@mail.gmail.com>
Message-ID: <568E6A8B.9050602@gmail.com>

El 06/01/16 a las 00:13, Jason Moore escribi?:
> SymPy has a comprehensive printing system and we leverage IPython's 
> qtconsole and notebook display abilities to show mathematics in 
> unicode, mathjax, and png outputs. Typically all three of these are 
> generated and the IPython frontend can choose what to display.
>
> We've hit a bug that we can't figure out how to fix though. Matplotlib 
> has a minimal LaTeX implementation and can output PNGs. If a user 
> doesn't have a full LaTeX suit installed we fall back to that. But 
> since matplotlib doesn't handle LaTeX environments, it can't render 
> things like matrices.
>
> Now if a qtconsole user tries to display a matrix, this fails if 
> matplotlib is the only available renderer. But if the user is using 
> the notebook then the matplotlib renderer will fail but mathjax will 
> print properly. We'd like to fall back to unicode printing if 
> matplotlib fails to print **only in the qtconsole**, but render with 
> mathjax in the notebook.

There's already a pull request in qtconsole about this problem:

https://github.com/jupyter/qtconsole/pull/65


Cheers,
Carlos

>
> So, as far as we can tell, we need to know which frontend is trying to 
> display our object so that we can handle errors properly. We haven't 
> been able to think of a workaround that is agnostic of the frontend.
>
> There is some discussion in this PR: 
> https://github.com/sympy/sympy/pull/9976 (among others).
>
> Any suggestions?
>
> Jason
> moorepants.info <http://moorepants.info>
> +01 530-601-9791
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160107/3c703cc0/attachment.html>

From moorepants at gmail.com  Thu Jan  7 09:21:11 2016
From: moorepants at gmail.com (Jason Moore)
Date: Thu, 7 Jan 2016 06:21:11 -0800
Subject: [IPython-dev] Supporting the qtconsole and notebook display
 system simultaneously
In-Reply-To: <568E6A8B.9050602@gmail.com>
References: <CAP7f1AiYBVBmk_NRgh03Uw4vBknazZhbUxH2Ar63xV6igsudJg@mail.gmail.com>
 <568E6A8B.9050602@gmail.com>
Message-ID: <CAP7f1AjdhsNnY=oKu30vo+S828ccLC0MMhPbk89ZY4Cg2kUowg@mail.gmail.com>

I didn't know about that PR. Thanks for pointing it. That likely solves the
issue that I've raised here. I'll try it out and discuss with Aaron.


Jason
moorepants.info
+01 530-601-9791

On Thu, Jan 7, 2016 at 5:39 AM, Carlos C?rdoba <ccordoba12 at gmail.com> wrote:

> El 06/01/16 a las 00:13, Jason Moore escribi?:
>
> SymPy has a comprehensive printing system and we leverage IPython's
> qtconsole and notebook display abilities to show mathematics in unicode,
> mathjax, and png outputs. Typically all three of these are generated and
> the IPython frontend can choose what to display.
>
> We've hit a bug that we can't figure out how to fix though. Matplotlib has
> a minimal LaTeX implementation and can output PNGs. If a user doesn't have
> a full LaTeX suit installed we fall back to that. But since matplotlib
> doesn't handle LaTeX environments, it can't render things like matrices.
>
> Now if a qtconsole user tries to display a matrix, this fails if
> matplotlib is the only available renderer. But if the user is using the
> notebook then the matplotlib renderer will fail but mathjax will print
> properly. We'd like to fall back to unicode printing if matplotlib fails to
> print **only in the qtconsole**, but render with mathjax in the notebook.
>
>
> There's already a pull request in qtconsole about this problem:
>
> https://github.com/jupyter/qtconsole/pull/65
>
>
> Cheers,
> Carlos
>
>
> So, as far as we can tell, we need to know which frontend is trying to
> display our object so that we can handle errors properly. We haven't been
> able to think of a workaround that is agnostic of the frontend.
>
> There is some discussion in this PR:
> <https://github.com/sympy/sympy/pull/9976>
> https://github.com/sympy/sympy/pull/9976 (among others).
>
> Any suggestions?
>
> Jason
> moorepants.info
> +01 530-601-9791
>
>
> _______________________________________________
> IPython-dev mailing listIPython-dev at scipy.orghttps://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160107/8bb27d04/attachment.html>

From ijstokes at alumni.uwaterloo.ca  Thu Jan  7 23:47:20 2016
From: ijstokes at alumni.uwaterloo.ca (Ian Stokes Rees)
Date: Thu, 7 Jan 2016 22:47:20 -0600
Subject: [IPython-dev] How to load an extension into every notebook on a
 server
In-Reply-To: <CADqKjdQz3X6PG-s=WQnBrcOHRqouWnMGs2YQX8zYL3OhfReBQw@mail.gmail.com>
References: <CADqKjdQo8+-D8zAGYHr5MW1xUqyGQSrk+GKuhWHhgO=eN8QGww@mail.gmail.com>
 <CADqKjdT=E4pQzOUpNyuYVNmS5_r+2VcDnF7j4ukcyh7irQ80kA@mail.gmail.com>
 <56881625.3020002@alumni.uwaterloo.ca> <56890CF1.3050206@alumni.uwaterloo.ca>
 <CADqKjdQz3X6PG-s=WQnBrcOHRqouWnMGs2YQX8zYL3OhfReBQw@mail.gmail.com>
Message-ID: <568F3F58.7030004@alumni.uwaterloo.ca>

Hi Ted,

This is "active work in progress".  We (at Continuum) came up with our 
own solution to this problem but in doing so we had an extensive 
discussion with the Jupyter team and agreed on a more general way to 
solve the issue that is "work-in-progress".  As such, I'd encourage you 
to "watch this space" until we have the work completed.  If you are 
really keen to try and get this working now then you could look at what 
we put together via:

https://github.com/Anaconda-Server/generator-nbextension
https://github.com/Anaconda-Server/nbsetuptools
https://github.com/Anaconda-Server/nb_config_manager

And here is an example:

https://github.com/Anaconda-Server/nb_helloworld

NOTE: We don't recommend that you use this right now.  It works, but it 
has been constructed very specifically for some of our current (internal 
and client-specific) use cases in environments which are 
Anaconda-centric.  You'd be much better off waiting until we have this 
adapted to the new format, if you have the luxury of time.

Cheers,

Ian

On 1/5/16 4:35 PM, Ted Liefeld wrote:
> Ian
>
> So it seems that (delayed response due to the holiday season) its not 
> actually possible to do this now using the example in 879 
> <https://github.com/jupyter/notebook/issues/879> unless I go to a 
> development build, or is there currently a way to do that by editing 
> my notebook.json file which seems to be implied by the discussion at 
> the top of 878 <https://github.com/jupyter/notebook/issues/878>. For 
> the moment this would work fine for our container since it is single 
> user and we control the files in the container.
>
> Speaking of which I cannot seem to figure out the right mojo to get 
> the notebook.json file (in the intro in 878 it refers to 
> paths.jupyter_path('nbextensions') but the "paths" variable is not 
> present in my notebook, do I need to run python in the container to 
> access this variable?
>
> Also, what is the root path for loading extensions like this?  The 
> examples don't make that clear, so I have this at the moment (note 
> this is in a docker container so its not really running as root)
>
>     # pwd
>     /root/.jupyter/nbconfig
>     # more notebook.json
>     {
>       "load_extensions" : {
>            "genepattern" : true
>            }
>
>     }
>
>
> while the extension installed (via pip) in
>
>     /root/.jupyter/custom/genepattern
>
>
> This, of course, does not actually work though
>
> Thanks
>
> Ted
>
> On Sun, Jan 3, 2016 at 3:58 AM, Ian Stokes Rees 
> <ijstokes at alumni.uwaterloo.ca <mailto:ijstokes at alumni.uwaterloo.ca>> 
> wrote:
>
>     Ted,
>
>     Now that I'm back online, the reference I should have provided was
>     this one:
>
>     https://github.com/jupyter/notebook/issues/878
>
>     That issue is the core discussion around Jupyter 4.2 modifications
>     to support better extension configuration. A number of PRs have
>     already been created and merged, but I believe the "main" open one is:
>
>     https://github.com/jupyter/notebook/pull/879
>
>     Regards.
>
>     Ian
>
>     On 1/2/16 1:25 PM, Ian Stokes Rees wrote:
>>     Hi Ted (and Thomas),
>>
>>     At Continuum we are also interested in having pre-configured
>>     notebook environments and are actively working with the Jupyter
>>     team to figure out good ways of enabling "universal"
>>     configuration of extensions in Jupyter 4 (for others: prior to
>>     the complete reworking of the "extension" mechanism which will
>>     arrive with Jupyter 5).
>>
>>     I'm offline now, otherwise I'd provide you with some links to the
>>     discussion, but you can dig up a reference if you look at the JEP
>>     area of the Jupyter GitHub repos.  I think this is the link:
>>
>>     https://github.com/jupyter/enhancement-proposals/pull/7
>>
>>     That is probably way too much detail for what you're looking for,
>>     but I, for one, would like to come up with a solution that would
>>     make your GeneExpression extension "pre-loadable" with no extra
>>     work on the part of the user.
>>
>>     Cheers,
>>
>>     Ian
>
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     https://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> -- 
> Ted Liefeld  UC San Diego
> Mesirov Lab liefeld at ucsd.edu <mailto:liefeld at ucsd.edu>
> Office 2A24, BRF-II  858-534-2010
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev

Ian Stokes-Rees
617.942.0218

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160107/a8443efe/attachment.html>

From steve at holdenweb.com  Fri Jan  8 06:38:14 2016
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 8 Jan 2016 11:38:14 +0000
Subject: [IPython-dev] Client/server security
Message-ID: <CAMofdRBetOo3g18sUQ_at7MkmeKCtHoBQJgDzBOEW+6x0hvHxg@mail.gmail.com>

Hi there,

I've taken a look at the Jupyter roadmap at
https://github.com/jupyter/roadmap and observed that there doesn't yet seem
to be any prospect of moving to secure communications between the client
and the server.

This is completely understandable, but I'd be grateful if someone better
informed than me could comment on the work that would be required to
achieve this.

Without taking too much time away from actual progress on scheduled
requirements, naturally ...

Steve Holden
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160108/634de609/attachment.html>

From benjaminrk at gmail.com  Fri Jan  8 06:51:29 2016
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 8 Jan 2016 12:51:29 +0100
Subject: [IPython-dev] Client/server security
In-Reply-To: <CAMofdRBetOo3g18sUQ_at7MkmeKCtHoBQJgDzBOEW+6x0hvHxg@mail.gmail.com>
References: <CAMofdRBetOo3g18sUQ_at7MkmeKCtHoBQJgDzBOEW+6x0hvHxg@mail.gmail.com>
Message-ID: <CAHNn8BWDuWV6tfjweuOpLYjmhOYcs6QMEb5inVvg1ezqe7wQ2Q@mail.gmail.com>

Communication between the client and server can be secured today with https
and a password. The docs
<http://jupyter-notebook.readthedocs.org/en/latest/public_server.html#securing-a-notebook-server>
cover setting up a server with a password and self-signed SSL certs. Are
you looking for something more?

-MinRK
?

On Fri, Jan 8, 2016 at 12:38 PM, Steve Holden <steve at holdenweb.com> wrote:

> Hi there,
>
> I've taken a look at the Jupyter roadmap at
> https://github.com/jupyter/roadmap and observed that there doesn't yet
> seem to be any prospect of moving to secure communications between the
> client and the server.
>
> This is completely understandable, but I'd be grateful if someone better
> informed than me could comment on the work that would be required to
> achieve this.
>
> Without taking too much time away from actual progress on scheduled
> requirements, naturally ...
>
> Steve Holden
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160108/116a5fc7/attachment.html>

From steve at holdenweb.com  Fri Jan  8 07:01:24 2016
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 8 Jan 2016 12:01:24 +0000
Subject: [IPython-dev] Client/server security
In-Reply-To: <CAHNn8BWDuWV6tfjweuOpLYjmhOYcs6QMEb5inVvg1ezqe7wQ2Q@mail.gmail.com>
References: <CAMofdRBetOo3g18sUQ_at7MkmeKCtHoBQJgDzBOEW+6x0hvHxg@mail.gmail.com>
 <CAHNn8BWDuWV6tfjweuOpLYjmhOYcs6QMEb5inVvg1ezqe7wQ2Q@mail.gmail.com>
Message-ID: <CAMofdRAfjkjxUfi_moiGQakndBAUin7Pgzq2941cqk_dS8VuHg@mail.gmail.com>

Oh, thanks for the heads-up. I suspect that will do for current purposes,
and will be back if enhancements would help!

regards
 Steve

Steve Holden

On Fri, Jan 8, 2016 at 11:51 AM, MinRK <benjaminrk at gmail.com> wrote:

> Communication between the client and server can be secured today with
> https and a password. The docs
> <http://jupyter-notebook.readthedocs.org/en/latest/public_server.html#securing-a-notebook-server>
> cover setting up a server with a password and self-signed SSL certs. Are
> you looking for something more?
>
> -MinRK
> ?
>
> On Fri, Jan 8, 2016 at 12:38 PM, Steve Holden <steve at holdenweb.com> wrote:
>
>> Hi there,
>>
>> I've taken a look at the Jupyter roadmap at
>> https://github.com/jupyter/roadmap and observed that there doesn't yet
>> seem to be any prospect of moving to secure communications between the
>> client and the server.
>>
>> This is completely understandable, but I'd be grateful if someone better
>> informed than me could comment on the work that would be required to
>> achieve this.
>>
>> Without taking too much time away from actual progress on scheduled
>> requirements, naturally ...
>>
>> Steve Holden
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160108/5677bf28/attachment.html>

From bussonniermatthias at gmail.com  Fri Jan  8 13:15:52 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 8 Jan 2016 19:15:52 +0100
Subject: [IPython-dev] [ANN] Notebook 4.1.0 out.
Message-ID: <1588E236-91B4-468B-9A9E-81EB3BAD9A11@gmail.com>

Hi all, 

I just released notebook 4.1.0 a few minutes ago, 
you can go see the announcement on our blog:

http://blog.jupyter.org/2016/01/08/notebook-4-1-release/ <http://blog.jupyter.org/2016/01/08/notebook-4-1-release/>

Thanks to everyone who helped !

Do backups, enjoy this release. Report any bugs, we candy a 4.1.1 soon
if there are critical things we miss !

Happy new year 2016 also !
-- 
M
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160108/890a8e91/attachment.html>

From fperez.net at gmail.com  Sun Jan 10 16:52:20 2016
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 10 Jan 2016 13:52:20 -0800
Subject: [IPython-dev] [jupyter] [ANN] Notebook 4.1.0 out.
In-Reply-To: <1588E236-91B4-468B-9A9E-81EB3BAD9A11@gmail.com>
References: <1588E236-91B4-468B-9A9E-81EB3BAD9A11@gmail.com>
Message-ID: <CAHAreOogtjvRTYRbaM_kxvM=xcL+UUB1o5MT5L30sEmOptPuHg@mail.gmail.com>

Congratulations, and many thanks to everyone who pushed extra hard in the
end to make this possible!!

Cheers,

f

On Fri, Jan 8, 2016 at 10:15 AM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> Hi all,
>
> I just released notebook 4.1.0 a few minutes ago,
> you can go see the announcement on our blog:
>
> http://blog.jupyter.org/2016/01/08/notebook-4-1-release/
>
> Thanks to everyone who helped !
>
> Do backups, enjoy this release. Report any bugs, we candy a 4.1.1 soon
> if there are critical things we miss !
>
> Happy new year 2016 also !
> --
> M
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscribe at googlegroups.com.
> To post to this group, send email to jupyter at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/1588E236-91B4-468B-9A9E-81EB3BAD9A11%40gmail.com
> <https://groups.google.com/d/msgid/jupyter/1588E236-91B4-468B-9A9E-81EB3BAD9A11%40gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160110/260b7d06/attachment.html>

From bussonniermatthias at gmail.com  Tue Jan 12 14:57:14 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 12 Jan 2016 11:57:14 -0800
Subject: [IPython-dev] [jupyter] API change proposal for ipywidget's
	interact
In-Reply-To: <CAK=Phk6NfcwyaCagm=RLBVTbh1oZ0YdYr8ncM1ZLH-JV=z_gYg@mail.gmail.com>
References: <CAK=Phk6NfcwyaCagm=RLBVTbh1oZ0YdYr8ncM1ZLH-JV=z_gYg@mail.gmail.com>
Message-ID: <57477D7B-6A7B-413D-AFAE-B897B0265DC1@gmail.com>

Hi all, 

This was sent on jupyter ML, but as it?s a Python-kernel only change, it might be good to CC IPython-dev, 
which I do there. 

Please keep a single thread, preferably on Jupyter groups as the thread was started there. 

Thanks
-- 
M


> On Jan 12, 2016, at 11:34, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
> 
> Hello,
> 
> Brian and Fernando advised that I send a message to the mailing list regarding this change.
> 
> Both Jupyter and Sage's interactive widget libraries provide an `@interact` function decorator which introspects the arguments of the function to create a simple GUI for interacting with it. Sage and Jupyter APIs for `@interact` are very similar, but differ in how they use abbreviations.
> 
> Abbreviations are a way to map default values to a type of widget. For example in both cases the following case
> 
>     @interact
>     def foo(x=(0, 100, 2)):
>         print(x)
> 
> both Jupyter and Sage will create a slider with from 0 to 100 with a step of 2. 
> 
> In fact, Jupyter and Sage's uses of abbreviations differ in the following way:
> 
> With Sage: When passing a list of items, the generated widget is always a selection widget.
> When passing a tuple of 2 or 3 numerical values, a slider is created and the values are interpreted as(min, max[, step])
> 
> With Jupyter, a selection widget is only used when passing sequence (tuple or list) of strings. 
> A slider is used when passing a sequence of 2 or 3 numerical values, and these values are interpreted as (min, max[, step]).
> 
> I personally prefer the API of Sage over the one of Jupyter. In my opinion, the type mapping `tuple -> slider` and `list -> selection` is simpler to reason about than a behavior that depends on the type of the items in a sequence.
> 
> Obviously, since the two APIs are not compatible, we cannot support them both at the same time. If we decide that we eventually want to adopt the Sage approach, I proposed a way to achieve a smooth transition in the pull request https://github.com/ipython/ipywidgets/pull/288 <https://github.com/ipython/ipywidgets/pull/288>. The proposed change is backward compatible with the Jupyter approach and introduces deprecation warnings when using a list of 2/3 numerical values to create a slider or if the user creates a selection widget with a tuple of strings.
> 
> The question to the list is: what do you think of adopting the same type abbreviations as Sage in Jupyter's interact?
> 
> Thanks,
> 
> Sylvain 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe at googlegroups.com <mailto:jupyter+unsubscribe at googlegroups.com>.
> To post to this group, send email to jupyter at googlegroups.com <mailto:jupyter at googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAK%3DPhk6NfcwyaCagm%3DRLBVTbh1oZ0YdYr8ncM1ZLH-JV%3Dz_gYg%40mail.gmail.com <https://groups.google.com/d/msgid/jupyter/CAK%3DPhk6NfcwyaCagm%3DRLBVTbh1oZ0YdYr8ncM1ZLH-JV%3Dz_gYg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160112/19eed0df/attachment.html>

From ra092767 at ime.unicamp.br  Mon Jan 18 14:23:22 2016
From: ra092767 at ime.unicamp.br (Raniere Silva)
Date: Mon, 18 Jan 2016 17:23:22 -0200
Subject: [IPython-dev] Google Summer of Code 2016
Message-ID: <20160118192322.GT1225@buriti.rgaiacs.com>

Hi all,

Since IPython is a NumFOCUS sponsored project
and NumFOCUS will apply to be a mentoring organization on GSoC
I want to know (1) if IPython is planning to apply this year
and (2) if want to apply under NumFOCUS umbrella.

IPython is welcome and encouraged to apply as separate mentoring
organizations directly with Google. We're happy to help you fill out your
application and improve your ideas pages, as well as link your page to help
students find you. We may also be able to be a reference for you.
It is totally fine if you want to use the NumFOCUS umbrella org
as a backup plan in case you don't get selected and we do!

Cheers,
Raniere
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160118/1e96696e/attachment.sig>

From bussonniermatthias at gmail.com  Thu Jan 21 13:07:50 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Thu, 21 Jan 2016 10:07:50 -0800
Subject: [IPython-dev] Trying to get IPython 4.1 through the door.
Message-ID: <4914E6D3-AAEF-430A-A6D4-595100CDFA0F@gmail.com>

Hi all, 

I?m trying to push for an IPython 4.1 (minor release) soon-ish, 

I think there are a few annoying bugs it would be nice to get released, and a few
minor new features that are useful. 

There are only a few issues left for 4.1, some we don?t even know how to fix/why the bug. 
Other we cannot reproduce. Any insight help on that would be appreciated. 

If you have a particular issue you think could be solve easily and would need to be in 4.1
could you please manifest yourself on the issue so that we can try to tackle it soonish. 

Branch 4.x will likely be a bug fix only branch, master will slowly diverge to bring
awesome new features for 5.0.

Thanks, 
-- 
Matthias

P.S.: Helps writing docs/what?s new is always welcome.




From takowl at gmail.com  Thu Jan 21 17:59:13 2016
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 21 Jan 2016 22:59:13 +0000
Subject: [IPython-dev] Trying to get IPython 4.1 through the door.
In-Reply-To: <4914E6D3-AAEF-430A-A6D4-595100CDFA0F@gmail.com>
References: <4914E6D3-AAEF-430A-A6D4-595100CDFA0F@gmail.com>
Message-ID: <CAOvn4qhNtxN8WVrRBf_rA+AD7DVVOJPU5X3FKsKzv2fA-wVagQ@mail.gmail.com>

+1 to working towards a release of 4.1 pretty soon.

On 21 January 2016 at 18:07, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> Hi all,
>
> I?m trying to push for an IPython 4.1 (minor release) soon-ish,
>
> I think there are a few annoying bugs it would be nice to get released,
> and a few
> minor new features that are useful.
>
> There are only a few issues left for 4.1, some we don?t even know how to
> fix/why the bug.
> Other we cannot reproduce. Any insight help on that would be appreciated.
>
> If you have a particular issue you think could be solve easily and would
> need to be in 4.1
> could you please manifest yourself on the issue so that we can try to
> tackle it soonish.
>
> Branch 4.x will likely be a bug fix only branch, master will slowly
> diverge to bring
> awesome new features for 5.0.
>
> Thanks,
> --
> Matthias
>
> P.S.: Helps writing docs/what?s new is always welcome.
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160121/a09b9cf6/attachment.html>

From bussonniermatthias at gmail.com  Tue Jan 26 15:11:28 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 26 Jan 2016 12:11:28 -0800
Subject: [IPython-dev] IPython 4.1.0rc1 out.
Message-ID: <F2E7CE92-CA52-4226-B006-8EC4922097C4@gmail.com>

Hi IPythonistas, 


Thanks to the hard work of Fernando this morning, IPython 4.1.0rc1  is now out:

https://pypi.python.org/pypi/ipython/4.1.0rc1 <https://pypi.python.org/pypi/ipython/4.1.0rc1>

As usual RC releases, you can install it with 

$ pip install ipython --pre --upgrade

A bunch of bug fixes, and a few new features (the what?s new have not be written yet, any help with that would be appreciated).
As usual, please test, we will likely release the final version in a few days.

Thanks to everyone for the hard work. 
-- 
Matthias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160126/e0e1a487/attachment.html>

From bussonniermatthias at gmail.com  Fri Jan 29 18:41:28 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 29 Jan 2016 15:41:28 -0800
Subject: [IPython-dev] Release Friday: IPython 4.1.0rc2 out.
Message-ID: <959589C5-C5BF-4050-99A1-7201BC4F78FB@gmail.com>

Hi all, 

I just push a version of IPython 4.1.0rc2 that fixed a few bugs, and correct a few mistakes I did in releasing the rc1.

Mainly:

> installing on python2 with rc1 might create an `ipython2` shortcut
> IPython.display.Image was broken.

These should now be fixed (which was making notebook test-suite to fail)

As usual if all goes well except change to documentation this should be the version that will become 4.1.0 final. 
You have a few days to send critical bug reports. 

Thanks, 
-- 
Mattthias

From jon.freder at gmail.com  Fri Jan 29 19:16:19 2016
From: jon.freder at gmail.com (Jonathan Frederic)
Date: Fri, 29 Jan 2016 16:16:19 -0800
Subject: [IPython-dev] Release Friday: IPython 4.1.0rc2 out.
In-Reply-To: <959589C5-C5BF-4050-99A1-7201BC4F78FB@gmail.com>
References: <959589C5-C5BF-4050-99A1-7201BC4F78FB@gmail.com>
Message-ID: <CAAoBLw0ODG7JCSOC7Pg2GaVD8C0gpQgMoSAhigDWULn-LeAcHw@mail.gmail.com>

Thanks Matthias!

On Fri, Jan 29, 2016 at 3:41 PM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> Hi all,
>
> I just push a version of IPython 4.1.0rc2 that fixed a few bugs, and
> correct a few mistakes I did in releasing the rc1.
>
> Mainly:
>
> > installing on python2 with rc1 might create an `ipython2` shortcut
> > IPython.display.Image was broken.
>
> These should now be fixed (which was making notebook test-suite to fail)
>
> As usual if all goes well except change to documentation this should be
> the version that will become 4.1.0 final.
> You have a few days to send critical bug reports.
>
> Thanks,
> --
> Mattthias
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20160129/0137908d/attachment.html>