From rjtskolasinski at gmail.com  Fri Jan  2 12:43:48 2015
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Fri, 2 Jan 2015 18:43:48 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com>
References: <CAAiurtfj6wrxUN2QWZN7_eX1bZ4Z3voqA+=z3Avmp7xSQ_5goQ@mail.gmail.com>
	<CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
	<CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
	<CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>
	<CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com>
	<CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com>
Message-ID: <CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com>

Hi guys,

My converter starts to take the final form. I am now working on handling
used figures in some clever way.

All of them that are generated in notebooks and are in output fields go in
smooth way.

I have some problem with figures that are put inside text in markdowns,
i.ex. as
![some_caption](files/dot.png)
which goes in html into <img src="dot.png" alt="some_caption" /><p
class="caption">caption</p>
(as a template very similar to basic one).

How can force converter to not skip 'files/' to have src="files/dot.png"?
Probably it lies somewhere in template but I couldn't localize it.

Cheers,
Rafa?



2014-12-19 17:57 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:

> Ok. I found it.
> Now the corresponding line should be
> exportHtml =
> HTMLExporter(config=Config({'HTMLExporter':{'template_file':'basic'}}))
> instead of
> exportHtml =
> HTMLExporter(config=Config({'HTMLExporter':{'default_template':'basic'}}))
>
>
> 2014-12-19 17:12 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>>
>> Hi guys,
>>
>> I got some kind of working version. It almost do what I want from it (at
>> least for first working version).
>> http://pastebin.com/bi0003LS
>>
>> Of course later there will come more functionality and polishing.
>>
>> Although I got strange problem. It looks like my converter ignores fact I
>> ask it for using 'basic' template.
>>
>> Does the syntax for it changed? I was basing on this old tutorial:
>>
>> http://nbviewer.ipython.org/github/ipython/ipython-in-depth/blob/master/examples/Notebook/Using%20nbconvert%20as%20a%20Library.ipynb
>>
>> Best,
>> Rafal
>>
>> 2014-12-18 22:38 GMT+01:00 Rafa? Skolasi?ski <rjtskolasinski at gmail.com>:
>>>
>>> I will ; - )
>>>
>>> 2014-12-18 22:36 GMT+01:00 Fernando Perez <fperez.net at gmail.com>:
>>>
>>>>
>>>> On Thu, Dec 18, 2014 at 9:06 AM, Rafa? Skolasi?ski <
>>>> rjtskolasinski at gmail.com> wrote:
>>>>>
>>>>> I will let you know if I will encounter any problems.
>>>>
>>>>
>>>> And please post here on the list once you have something up and
>>>> running. It's very important for the project to show how others use it not
>>>> only as an interactive environment, but also as infrastructure to build
>>>> upon.  That's the kind of clear value that helps us secure resources to
>>>> sustain its development.
>>>>
>>>> Best
>>>>
>>>> f
>>>>
>>>>
>>>> --
>>>> Fernando Perez (@fperez_org; http://fperez.org)
>>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>>>> fernando.perez-at-berkeley: contact me here for any direct mail
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150102/a8acb956/attachment.html>

From dsdale24 at gmail.com  Sat Jan  3 10:45:41 2015
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 03 Jan 2015 15:45:41 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
	<DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov>
Message-ID: <CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com>

Hi Ray,

On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> wrote:

>  I don?t really understand what you are trying to achieve, but the
> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that
> is used by the console, and I would have thought you would have to do some
> kind of injection to add other objects from within the application.
>

What I am trying to achieve is explicitly documented at
http://ipython.org/ipython-doc/dev/interactive/reference.html#embedding-ipython
:

---
 It is also possible to embed an IPython shell in a namespace in your
Python code. This allows you to evaluate dynamically the state of your
code, operate with your variables, analyze them, etc. Note however that any
changes you make to values while in the shell do not propagate back to the
running code, so it is safe to modify your values because you won?t break
your code in bizarre ways by doing so.

Note
At present, embedding IPython cannot be done from inside IPython. Run the
code samples below outside IPython.
[DD: I am not attempting to embed ipython from inside ipython]

This feature allows you to easily have a fully functional python
environment for doing object introspection anywhere in your code with a
simple function call. In some cases a simple print statement is enough, but
if you need to do more detailed analysis of a code fragment this feature
can be very valuable.

It can also be useful in scientific computing situations where it is common
to need to do some automatic, computationally intensive part and then stop
to look at data, plots, etc. Opening an IPython instance will give you full
access to your data and functions, and you can resume program execution
once you are done with the interactive part (perhaps to stop again later,
as many times as needed).
The following code snippet is the bare minimum you need to include in your
Python programs for this to work (detailed examples follow later):

   from IPython import embed
   embed() # this call anywhere in your program will start IPython

You can also embed an IPython kernel, for use with qtconsole, etc. via
IPython.embed_kernel(). This should function work the same way, but you can
connect an external frontend (ipython qtconsole or ipython console), rather
than interacting with it in the terminal.
---

This is impressively simple for the embed function:

---
C:\Users\darren> python
Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 15:12:11)
[MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> from IPython import embed
>>> a=1
>>> embed()
Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 15:12:11)
[MSC v.1500 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: a
Out[1]: 1
---

But I have not been able to achieve the same behavior with the qt
in-process console.

Darren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150103/9e2d2fd7/attachment.html>

From ra092767 at ime.unicamp.br  Sat Jan  3 21:27:02 2015
From: ra092767 at ime.unicamp.br (Raniere Silva)
Date: Sun, 4 Jan 2015 00:27:02 -0200
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com>
References: <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
	<CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
	<CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>
	<CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com>
	<CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com>
	<CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com>
Message-ID: <20150104022702.GM1209@buriti.rgaiacs.com>

Hi Rafa?,

> I have some problem with figures that are put inside text in markdowns,
> i.ex. as
> ![some_caption](files/dot.png)
> which goes in html into <img src="dot.png" alt="some_caption" /><p
> class="caption">caption</p>
> (as a template very similar to basic one).
> 
> How can force converter to not skip 'files/' to have src="files/dot.png"?
> Probably it lies somewhere in template but I couldn't localize it.

I couldn't reproduce your problem.

    $ cat sample.ipynb
    {
     "cells": [
      {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
        "![foo](folder/bar.jpg)"
       ]
      },
      {
       "cell_type": "code",
       "execution_count": null,
       "metadata": {
        "collapsed": true
       },
       "outputs": [],
       "source": []
      }
     ],
     "metadata": {
      "kernelspec": {
       "display_name": "IPython (Python 3)",
       "name": "python3"
      },
      "language_info": {
       "codemirror_mode": {
        "name": "ipython",
        "version": 3
       },
       "file_extension": ".py",
       "mimetype": "text/x-python",
       "name": "python",
       "nbconvert_exporter": "python",
       "pygments_lexer": "ipython3"
      },
      "signature": "sha256:5c8cb31d4083cbf815d3e1dff7a3cc229fa54b9c02cff82af4548cef72581573"
     },
     "nbformat": 4,
     "nbformat_minor": 0
    }
    $ ipython nbconvert --to html sample.ipynb
    [NbConvertApp] Using existing profile dir: '/home/raniere/.ipython/profile_default'
    [NbConvertApp] Converting notebook sample.ipynb to html
    [NbConvertApp] Support files will be in sample_files/
    [NbConvertApp] Loaded template full.tpl
    [NbConvertApp] Writing 223380 bytes to sample.html
    $ grep '<img' sample.html
    <p><img src="folder/bar.jpg" alt="foo"></p>

What version of IPython/Jupyter are you using? I tested with

    $ ipython --version
    3.0.0-dev

Cheers,
Raniere
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150104/78c6ae6b/attachment.sig>

From fperez.net at gmail.com  Sat Jan  3 22:29:50 2015
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 3 Jan 2015 19:29:50 -0800
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the core
	Jupyter/IPython team
Message-ID: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>

We'd like to welcome Kyle Kelley and Nick Bollweg to the core development
team. In Kyle's case this is a long-overdue announcement, as Kyle has been
a core dev for quite a while now, we just hadn't been very good about
communicating that on-list.  Kyle, thanks to Racksspace's generous support
for the project,

Trying to redress that lack of communication, in Nick's case we're not
waiting :)  Nick has been making great contributions to nbviewer, widgets,
and overall discussions in the project, and accepted to be a core developer
moving forward, as his other obligations allow.

Welcome (if belatedly) to both!

Cheers,

f

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150103/56f0ded7/attachment.html>

From rjtskolasinski at gmail.com  Sun Jan  4 08:19:01 2015
From: rjtskolasinski at gmail.com (=?UTF-8?Q?Rafa=C5=82_Skolasi=C5=84ski?=)
Date: Sun, 4 Jan 2015 14:19:01 +0100
Subject: [IPython-dev] Making custom converter/template with nbconvert
In-Reply-To: <20150104022702.GM1209@buriti.rgaiacs.com>
References: <CAOvn4qgMtDKh_1v6GUDVFEYZLCidSqZSvx8PETHQus-pZCFQng@mail.gmail.com>
	<CAAiurtcyVp2Pz_=yVUPz1KMUyoYa7hxZEBbxZxkOs4iAoRUYWw@mail.gmail.com>
	<17558727-CFFE-4B0F-87F9-B0C7B5C58799@gmail.com>
	<CAAiurtdSh-hp88PggDUnsrN6QwZCKdJREx=oio8DcE5T3dr0kA@mail.gmail.com>
	<CAAiurtfTgC7rA+WwQmmSkJSq2bU_BfS_T9AujWdA2HmR1WJyPg@mail.gmail.com>
	<CAHAreOpPmg2bVUmeRMEYPOFdRMvkU_A1nfvgTxz=kcQ=T9ghjg@mail.gmail.com>
	<CAAiurtc0hT5muWuupzym20nNDrZ3TswH2t7d0YAuXUeSN54ipw@mail.gmail.com>
	<CAAiurtcXF5FRcXd7GiO5Jsd3+mHEGHB81cKfME1ypezEYcT-7w@mail.gmail.com>
	<CAAiurtdW4k1Ne0CUvz54TkLpdi+iiJYJR=Yrp7ADemXfnEDzFQ@mail.gmail.com>
	<CAAiurtfQicdQJYXtAGLF-sjNTYHxqDnwS1k7=9hMV3KTgCh+3w@mail.gmail.com>
	<20150104022702.GM1209@buriti.rgaiacs.com>
Message-ID: <CAAiurtdmBLcDa6ygAZmxP61q=z1F=Eo=uif+0sSFxyqsPfUtSw@mail.gmail.com>

Hi Raniere,

The problem occurred when folder was named 'files'. When I renamed it to
'figures' it behaves as in your example. I must say that this very weird
behavior but there must be some reason for that probably.

Cheers,
Rafa?

2015-01-04 3:27 GMT+01:00 Raniere Silva <ra092767 at ime.unicamp.br>:

> Hi Rafa?,
>
> > I have some problem with figures that are put inside text in markdowns,
> > i.ex. as
> > ![some_caption](files/dot.png)
> > which goes in html into <img src="dot.png" alt="some_caption" /><p
> > class="caption">caption</p>
> > (as a template very similar to basic one).
> >
> > How can force converter to not skip 'files/' to have src="files/dot.png"?
> > Probably it lies somewhere in template but I couldn't localize it.
>
> I couldn't reproduce your problem.
>
>     $ cat sample.ipynb
>     {
>      "cells": [
>       {
>        "cell_type": "markdown",
>        "metadata": {},
>        "source": [
>         "![foo](folder/bar.jpg)"
>        ]
>       },
>       {
>        "cell_type": "code",
>        "execution_count": null,
>        "metadata": {
>         "collapsed": true
>        },
>        "outputs": [],
>        "source": []
>       }
>      ],
>      "metadata": {
>       "kernelspec": {
>        "display_name": "IPython (Python 3)",
>        "name": "python3"
>       },
>       "language_info": {
>        "codemirror_mode": {
>         "name": "ipython",
>         "version": 3
>        },
>        "file_extension": ".py",
>        "mimetype": "text/x-python",
>        "name": "python",
>        "nbconvert_exporter": "python",
>        "pygments_lexer": "ipython3"
>       },
>       "signature":
> "sha256:5c8cb31d4083cbf815d3e1dff7a3cc229fa54b9c02cff82af4548cef72581573"
>      },
>      "nbformat": 4,
>      "nbformat_minor": 0
>     }
>     $ ipython nbconvert --to html sample.ipynb
>     [NbConvertApp] Using existing profile dir:
> '/home/raniere/.ipython/profile_default'
>     [NbConvertApp] Converting notebook sample.ipynb to html
>     [NbConvertApp] Support files will be in sample_files/
>     [NbConvertApp] Loaded template full.tpl
>     [NbConvertApp] Writing 223380 bytes to sample.html
>     $ grep '<img' sample.html
>     <p><img src="folder/bar.jpg" alt="foo"></p>
>
> What version of IPython/Jupyter are you using? I tested with
>
>     $ ipython --version
>     3.0.0-dev
>
> Cheers,
> Raniere
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150104/94f8eab6/attachment.html>

From schueller at phimeca.com  Sun Jan  4 12:31:00 2015
From: schueller at phimeca.com (Julien Schueller)
Date: Sun, 4 Jan 2015 18:31:00 +0100 (CET)
Subject: [IPython-dev] graph displayed twice while overloading _repr_svg_
	for notebook
In-Reply-To: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com>
References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com>
Message-ID: <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com>




Hello,

I'm having trouble overloading _repr_svg_ for graph inlining in the notebook; my object is displayed twice if I call plt.plot just before.
I assumed a figure was left open or something, but even if I call plt.close('all') between the matplotlib calls, the result remains the same.

I did some imports first:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
from scipy.stats import norm
import sys
import io
print('matplotlib version %s' % matplotlib.__version__)
import IPython
print('IPython version %s' % IPython.__version__)


Here'my object:
class NormGraph(object):
    def __init__(self):
        super(NormGraph, self).__init__()
    def _repr_svg_(self):
        if sys.version_info[0] >= 3:
            output = io.StringIO()
        else:
            output = io.BytesIO()
        self._fig = plt.figure()
        axes_kwargs = {}
        plot_kwargs = {}
        self._ax = [self._fig.add_subplot(111, **axes_kwargs)]
        x = np.arange(-10, 10, 0.001)
        y = norm.pdf(x,0,2)
        self._ax[0].plot(x, y, **plot_kwargs)
        self._fig.savefig(output, format='svg')
       
        return output.getvalue()


Then a basic call to pyplot:
%matplotlib inline
plt.plot(0, 1)

Then if I try to inline my object it's displayed twice:
NormGraph()

I atached the complete notebook.

-- 
Julien Schueller
Phimeca Engineering
www.phimeca.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Untitled0.ipynb
Type: application/octet-stream
Size: 40341 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150104/27afca35/attachment.obj>

From ssanderson at quantopian.com  Sun Jan  4 13:01:24 2015
From: ssanderson at quantopian.com (ssanderson)
Date: Sun, 4 Jan 2015 10:01:24 -0800 (PST)
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
 core Jupyter/IPython team
In-Reply-To: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
Message-ID: <1420394484530-5082155.post@n6.nabble.com>

Congrats!



--
View this message in context: http://python.6.x6.nabble.com/Welcoming-Kyle-Kelley-and-Nick-Bollweg-to-the-core-Jupyter-IPython-team-tp5082131p5082155.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From takowl at gmail.com  Mon Jan  5 12:43:57 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 5 Jan 2015 17:43:57 +0000
Subject: [IPython-dev] graph displayed twice while overloading
 _repr_svg_ for notebook
In-Reply-To: <1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com>
References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com>
	<1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com>
Message-ID: <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com>

If you've done '%matplotlib inline' previously, when you create a figure
inside a cell, it will automatically be displayed at the end of that cell,
separately from the regular repr system. So your _repr_svg_ method is
getting called once, and the figure you create in that is being displayed
again. I'm not sure what the best way to deal with that is.

Thomas

On 4 January 2015 at 17:31, Julien Schueller <schueller at phimeca.com> wrote:

>
>
>
> Hello,
>
> I'm having trouble overloading _repr_svg_ for graph inlining in the
> notebook; my object is displayed twice if I call plt.plot just before.
> I assumed a figure was left open or something, but even if I call
> plt.close('all') between the matplotlib calls, the result remains the same.
>
> I did some imports first:
> import numpy as np
> import matplotlib.pyplot as plt
> import matplotlib
> from scipy.stats import norm
> import sys
> import io
> print('matplotlib version %s' % matplotlib.__version__)
> import IPython
> print('IPython version %s' % IPython.__version__)
>
>
> Here'my object:
> class NormGraph(object):
>     def __init__(self):
>         super(NormGraph, self).__init__()
>     def _repr_svg_(self):
>         if sys.version_info[0] >= 3:
>             output = io.StringIO()
>         else:
>             output = io.BytesIO()
>         self._fig = plt.figure()
>         axes_kwargs = {}
>         plot_kwargs = {}
>         self._ax = [self._fig.add_subplot(111, **axes_kwargs)]
>         x = np.arange(-10, 10, 0.001)
>         y = norm.pdf(x,0,2)
>         self._ax[0].plot(x, y, **plot_kwargs)
>         self._fig.savefig(output, format='svg')
>
>         return output.getvalue()
>
>
> Then a basic call to pyplot:
> %matplotlib inline
> plt.plot(0, 1)
>
> Then if I try to inline my object it's displayed twice:
> NormGraph()
>
> I atached the complete notebook.
>
> --
> Julien Schueller
> Phimeca Engineering
> www.phimeca.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/8238f256/attachment.html>

From tritemio at gmail.com  Mon Jan  5 14:06:50 2015
From: tritemio at gmail.com (Antonino Ingargiola)
Date: Mon, 5 Jan 2015 11:06:50 -0800
Subject: [IPython-dev] graph displayed twice while overloading
 _repr_svg_ for notebook
In-Reply-To: <CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com>
References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com>
	<1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com>
	<CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com>
Message-ID: <CANn2QUyL99k-z6at_NssGFVgNr33RbReOH6jKQdDX5U91A0uqw@mail.gmail.com>

Hi,

I don't know if it helps but with plain matplotlib you can close the figure
and call display(fig) on the closed figure. I use this trick to plot
several figures in a loop (closing them at each iteration) or for updating
plots via widgets.

Antonio

On Mon, Jan 5, 2015 at 9:43 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> If you've done '%matplotlib inline' previously, when you create a figure
> inside a cell, it will automatically be displayed at the end of that cell,
> separately from the regular repr system. So your _repr_svg_ method is
> getting called once, and the figure you create in that is being displayed
> again. I'm not sure what the best way to deal with that is.
>
> Thomas
>
> On 4 January 2015 at 17:31, Julien Schueller <schueller at phimeca.com>
> wrote:
>
>>
>>
>>
>> Hello,
>>
>> I'm having trouble overloading _repr_svg_ for graph inlining in the
>> notebook; my object is displayed twice if I call plt.plot just before.
>> I assumed a figure was left open or something, but even if I call
>> plt.close('all') between the matplotlib calls, the result remains the same.
>>
>> I did some imports first:
>> import numpy as np
>> import matplotlib.pyplot as plt
>> import matplotlib
>> from scipy.stats import norm
>> import sys
>> import io
>> print('matplotlib version %s' % matplotlib.__version__)
>> import IPython
>> print('IPython version %s' % IPython.__version__)
>>
>>
>> Here'my object:
>> class NormGraph(object):
>>     def __init__(self):
>>         super(NormGraph, self).__init__()
>>     def _repr_svg_(self):
>>         if sys.version_info[0] >= 3:
>>             output = io.StringIO()
>>         else:
>>             output = io.BytesIO()
>>         self._fig = plt.figure()
>>         axes_kwargs = {}
>>         plot_kwargs = {}
>>         self._ax = [self._fig.add_subplot(111, **axes_kwargs)]
>>         x = np.arange(-10, 10, 0.001)
>>         y = norm.pdf(x,0,2)
>>         self._ax[0].plot(x, y, **plot_kwargs)
>>         self._fig.savefig(output, format='svg')
>>
>>         return output.getvalue()
>>
>>
>> Then a basic call to pyplot:
>> %matplotlib inline
>> plt.plot(0, 1)
>>
>> Then if I try to inline my object it's displayed twice:
>> NormGraph()
>>
>> I atached the complete notebook.
>>
>> --
>> Julien Schueller
>> Phimeca Engineering
>> www.phimeca.com
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/1d446bfa/attachment.html>

From darcamo at gmail.com  Mon Jan  5 14:14:56 2015
From: darcamo at gmail.com (darcamo at gmail.com)
Date: Mon, 05 Jan 2015 19:14:56 +0000
Subject: [IPython-dev] graph displayed twice while overloading
 _repr_svg_ for notebook
References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com>
	<1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com>
	<CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com>
Message-ID: <CAGjSdXqi-Cv_GzOthhkRt_to-56p5hQTTE5F0XQ=HoQcX3igkQ@mail.gmail.com>

Call plt.close(self._fig) after savefig and the inline plot should not be
shown. Since you are using the figure only to convert to svg with savefig
and you need to close it to avoid the inline plot, then there is no reason
to keep it as an instance attribute.

It might also be a good idea to disable matplotlib interactive mode by
calling plt.ioff() before creating the figure object and reenabling it
after the figure is closed by calling plt.ion().

--
Darlan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/a74c1c65/attachment.html>

From ellisonbg at gmail.com  Mon Jan  5 17:01:53 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 5 Jan 2015 14:01:53 -0800
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
 core Jupyter/IPython team
In-Reply-To: <1420394484530-5082155.post@n6.nabble.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
	<1420394484530-5082155.post@n6.nabble.com>
Message-ID: <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>

Thanks so much for your work Kyle and Nick!

Cheers,

Brian

On Sun, Jan 4, 2015 at 10:01 AM, ssanderson <ssanderson at quantopian.com>
wrote:

> Congrats!
>
>
>
> --
> View this message in context:
> http://python.6.x6.nabble.com/Welcoming-Kyle-Kelley-and-Nick-Bollweg-to-the-core-Jupyter-IPython-team-tp5082131p5082155.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150105/dec7cbb7/attachment.html>

From schueller at phimeca.com  Tue Jan  6 04:20:52 2015
From: schueller at phimeca.com (Julien Schueller)
Date: Tue, 6 Jan 2015 10:20:52 +0100 (CET)
Subject: [IPython-dev] graph displayed twice while overloading
 _repr_svg_ for notebook
In-Reply-To: <CAGjSdXqi-Cv_GzOthhkRt_to-56p5hQTTE5F0XQ=HoQcX3igkQ@mail.gmail.com>
References: <2107085751.5438082.1418729306654.JavaMail.zimbra@phimeca.com>
	<1095968995.59454.1420392660687.JavaMail.zimbra@phimeca.com>
	<CAOvn4qgmon9DXa-CcWd7ZRv3wy=gJUsZPFAAB_p9OXZ_HW7dUQ@mail.gmail.com>
	<CAGjSdXqi-Cv_GzOthhkRt_to-56p5hQTTE5F0XQ=HoQcX3igkQ@mail.gmail.com>
Message-ID: <1463575469.219037.1420536052858.JavaMail.zimbra@phimeca.com>



----- Mail original -----
> De: darcamo at gmail.com
> ?: "IPython developers list" <ipython-dev at scipy.org>
> Envoy?: Lundi 5 Janvier 2015 20:14:56
> Objet: Re: [IPython-dev] graph displayed twice while overloading _repr_svg_ for notebook
> 
> Call plt.close(self._fig) after savefig and the inline plot should not be
> shown. Since you are using the figure only to convert to svg with savefig
> and you need to close it to avoid the inline plot, then there is no reason
> to keep it as an instance attribute.
> 
> It might also be a good idea to disable matplotlib interactive mode by
> calling plt.ioff() before creating the figure object and reenabling it after
> the figure is closed by calling plt.ion().
> 
> --
> Darlan
> 
That was it, thanks!


From ellisonbg at gmail.com  Tue Jan  6 13:53:06 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 6 Jan 2015 10:53:06 -0800
Subject: [IPython-dev] Change in weekly dev meeting times
Message-ID: <CAH4pYpRht0rCGse2wDx6qd+wBvDUUozw91naG2-oxn4vE0LZTw@mail.gmail.com>

Hi all!

In the past we have had two weekly dev meetings:

* Tuesday 9:30-11am PST = special topics as needed
* Thursday 9:30-11am PST = general topics, always happens

Because of some scheduling conflicts, we are going to swap those starting
this week:

* Tuesday 9:30-11am PST = general topics, always happens
* Thursday 9:30-11am PST = special topics as needed

Thus, this Th, we will not have a regular, general meeting. We may have a
special topics meeting if needed, but we aren't sure yet. Out next general
meeting will be next Tuesday 9:30am PST.

Se you all soon!

Cheers,

Brian

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150106/b09877c5/attachment.html>

From benjaminrk at gmail.com  Thu Jan  8 13:04:10 2015
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 8 Jan 2015 10:04:10 -0800
Subject: [IPython-dev] Notebook websocket change
Message-ID: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>

Alternative notebook frontend authors:

There is an upcoming change <https://github.com/ipython/ipython/pull/7389>
to websocket connections to the notebook server. Instead of three
websockets per notebook, one for each zmq channel, there is a single
websocket connection between the server and client. The channel associated
with each message is identified by a channel key in the message dict itself.

Relevant changes:

   - url is /channels instead of /shell, etc.
   - when sending a message, add a channel key with the channel name ('shell',
   'stdin')
   - when receiving a message, check the channel key for routing

-MinRK
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/3e25d78e/attachment.html>

From wstein at gmail.com  Thu Jan  8 13:10:10 2015
From: wstein at gmail.com (William Stein)
Date: Thu, 8 Jan 2015 10:10:10 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
Message-ID: <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>

On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
> Alternative notebook frontend authors:
>
> There is an upcoming change to websocket connections to the notebook server.
> Instead of three websockets per notebook, one for each zmq channel, there is
> a single websocket connection between the server and client. The channel
> associated with each message is identified by a channel key in the message
> dict itself.
>
> Relevant changes:
>
> url is /channels instead of /shell, etc.
> when sending a message, add a channel key with the channel name ('shell',
> 'stdin')
> when receiving a message, check the channel key for routing
>
> -MinRK

Thanks.

Of related interest, what the situation involving providing a fallback
to websockets?

I tried to only support websockets connections with SageMathCloud for
a few weeks, and immediately ran into many complaints from users who
couldn't connect as a result.   This is even with https and so on --
it's just a sad fact that in some corporate or otherwise constrained
environments that websockets don't work.   By far the best approach I
found was to use Primus + Engine.io, which uses websockets if
possible, but will fallback to polling.   So this is what SMC does
now, and everything works, even for people that don't have websockets
as an option... except for IPython-in-SMC of course, which doesn't
work.

Another unrelated issue is that I'm now hardcoding making this change
around line 171 of IPython/html/notebookapp.py, since I want to serve
the purely static html of IPython from a completely different web
server:

            #static_url_prefix = url_path_join(base_url,'/static/'),
            static_url_prefix = '/static/ipython/',

(Sorry for derailing the thread.)

 -- William

>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From ellisonbg at gmail.com  Thu Jan  8 13:19:19 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 8 Jan 2015 10:19:19 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
Message-ID: <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>

So far we have felt that it isn't worth the complexity cost for us to
support people that deliberately choose to break the internet. It is not
like WebSockets is some crazy, insecure, unsupported hack over a weird
port. It is just different bytes going over the same HTTP/HTTPS socket
connection and is essentially a formal standard (I know you know this -
this is our messaging to companies who complain).

Our situation is a bit different though than SMC because most of our
current users run the notebook on their own computers. This may be
something that we eventually want to rethink as more people run this in the
cloud. However, I would like to push back on the companies choosing to
break the internet really hard before giving in. I am guessing that in most
cases, WebSockets are broken in enterprise setting not because of some
important deliberate choice, but rather because people don't understand it
and are using default settings that disable them (I could be wrong
though)....

Cheers,

Brian



On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote:

> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
> > Alternative notebook frontend authors:
> >
> > There is an upcoming change to websocket connections to the notebook
> server.
> > Instead of three websockets per notebook, one for each zmq channel,
> there is
> > a single websocket connection between the server and client. The channel
> > associated with each message is identified by a channel key in the
> message
> > dict itself.
> >
> > Relevant changes:
> >
> > url is /channels instead of /shell, etc.
> > when sending a message, add a channel key with the channel name ('shell',
> > 'stdin')
> > when receiving a message, check the channel key for routing
> >
> > -MinRK
>
> Thanks.
>
> Of related interest, what the situation involving providing a fallback
> to websockets?
>
> I tried to only support websockets connections with SageMathCloud for
> a few weeks, and immediately ran into many complaints from users who
> couldn't connect as a result.   This is even with https and so on --
> it's just a sad fact that in some corporate or otherwise constrained
> environments that websockets don't work.   By far the best approach I
> found was to use Primus + Engine.io, which uses websockets if
> possible, but will fallback to polling.   So this is what SMC does
> now, and everything works, even for people that don't have websockets
> as an option... except for IPython-in-SMC of course, which doesn't
> work.
>
> Another unrelated issue is that I'm now hardcoding making this change
> around line 171 of IPython/html/notebookapp.py, since I want to serve
> the purely static html of IPython from a completely different web
> server:
>
>             #static_url_prefix = url_path_join(base_url,'/static/'),
>             static_url_prefix = '/static/ipython/',
>
> (Sorry for derailing the thread.)
>
>  -- William
>
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/4eb68b6f/attachment.html>

From wstein at gmail.com  Thu Jan  8 13:55:53 2015
From: wstein at gmail.com (William Stein)
Date: Thu, 8 Jan 2015 10:55:53 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
Message-ID: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>

On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> So far we have felt that it isn't worth the complexity cost for us to
> support people that deliberately choose to break the internet. It is not
> like WebSockets is some crazy, insecure, unsupported hack over a weird port.
> It is just different bytes going over the same HTTP/HTTPS socket connection
> and is essentially a formal standard (I know you know this - this is our
> messaging to companies who complain).

I know -- that's why I switched SMC to pure websockets for a while.
Then I received complaints at a rate of about 2 very desperate users
per week, and these were the people who cared enough to complain
repeatedly.

> Our situation is a bit different though than SMC because most of our current
> users run the notebook on their own computers. This may be something that we
> eventually want to rethink as more people run this in the cloud. However, I
> would like to push back on the companies choosing to break the internet
> really hard before giving in. I am guessing that in most cases, WebSockets
> are broken in enterprise setting not because of some important deliberate
> choice, but rather because people don't understand it and are using default
> settings that disable them (I could be wrong though)....

In my experience, for some people that's exactly the problem.  For one
person I helped it turned out their specific problem was the Windows
install they had access to had websockets explicitly disabled --
perhaps that was an IT policy.  For other people the problem is
further along the stack (e.g., possibly involving caching).

Anyway, this change so that "there is a single websocket connection
between the server and client. " makes implementing an alternative to
WebSocket as a fallback dramatically easier, which I greatly
appreciate.

> Our situation is a bit different though than SMC because most of our current
> users run the notebook on their own computers. This may be something that we

Just curious -- how do you know that most people using IPython aren't
already using it through SMC?

William

>
> Cheers,
>
> Brian
>
>
>
> On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote:
>>
>> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
>> > Alternative notebook frontend authors:
>> >
>> > There is an upcoming change to websocket connections to the notebook
>> > server.
>> > Instead of three websockets per notebook, one for each zmq channel,
>> > there is
>> > a single websocket connection between the server and client. The channel
>> > associated with each message is identified by a channel key in the
>> > message
>> > dict itself.
>> >
>> > Relevant changes:
>> >
>> > url is /channels instead of /shell, etc.
>> > when sending a message, add a channel key with the channel name
>> > ('shell',
>> > 'stdin')
>> > when receiving a message, check the channel key for routing
>> >
>> > -MinRK
>>
>> Thanks.
>>
>> Of related interest, what the situation involving providing a fallback
>> to websockets?
>>
>> I tried to only support websockets connections with SageMathCloud for
>> a few weeks, and immediately ran into many complaints from users who
>> couldn't connect as a result.   This is even with https and so on --
>> it's just a sad fact that in some corporate or otherwise constrained
>> environments that websockets don't work.   By far the best approach I
>> found was to use Primus + Engine.io, which uses websockets if
>> possible, but will fallback to polling.   So this is what SMC does
>> now, and everything works, even for people that don't have websockets
>> as an option... except for IPython-in-SMC of course, which doesn't
>> work.
>>
>> Another unrelated issue is that I'm now hardcoding making this change
>> around line 171 of IPython/html/notebookapp.py, since I want to serve
>> the purely static html of IPython from a completely different web
>> server:
>>
>>             #static_url_prefix = url_path_join(base_url,'/static/'),
>>             static_url_prefix = '/static/ipython/',
>>
>> (Sorry for derailing the thread.)
>>
>>  -- William
>>
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From benjaminrk at gmail.com  Thu Jan  8 14:18:43 2015
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 8 Jan 2015 11:18:43 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
Message-ID: <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com>

Some time ago, I made a PR to IPython switching to socket.io, which
required the same basic changes since it only allowed a single connection
per page, so I know it's not too hard. We may start to see more pressure as
hosted instances, and reconsider the fallback in the future, but at this
point, we don't feel that pressure. Wind proper websockets to be
sufficiently preferable that we will wait as long as we can before
appeasing old, broken, or misconfigured network environments that prevent
websockets from working.

The majority of notebook users seem to be running as a desktop app via
localhost, where websockets are rarely a problem (The nightmare of Sophos
AV/firewall on Windows excluded).

-MinRK

On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote:

> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com>
> wrote:
> > So far we have felt that it isn't worth the complexity cost for us to
> > support people that deliberately choose to break the internet. It is not
> > like WebSockets is some crazy, insecure, unsupported hack over a weird
> port.
> > It is just different bytes going over the same HTTP/HTTPS socket
> connection
> > and is essentially a formal standard (I know you know this - this is our
> > messaging to companies who complain).
>
> I know -- that's why I switched SMC to pure websockets for a while.
> Then I received complaints at a rate of about 2 very desperate users
> per week, and these were the people who cared enough to complain
> repeatedly.
>
> > Our situation is a bit different though than SMC because most of our
> current
> > users run the notebook on their own computers. This may be something
> that we
> > eventually want to rethink as more people run this in the cloud.
> However, I
> > would like to push back on the companies choosing to break the internet
> > really hard before giving in. I am guessing that in most cases,
> WebSockets
> > are broken in enterprise setting not because of some important deliberate
> > choice, but rather because people don't understand it and are using
> default
> > settings that disable them (I could be wrong though)....
>
> In my experience, for some people that's exactly the problem.  For one
> person I helped it turned out their specific problem was the Windows
> install they had access to had websockets explicitly disabled --
> perhaps that was an IT policy.  For other people the problem is
> further along the stack (e.g., possibly involving caching).
>
> Anyway, this change so that "there is a single websocket connection
> between the server and client. " makes implementing an alternative to
> WebSocket as a fallback dramatically easier, which I greatly
> appreciate.
>
> > Our situation is a bit different though than SMC because most of our
> current
> > users run the notebook on their own computers. This may be something
> that we
>
> Just curious -- how do you know that most people using IPython aren't
> already using it through SMC?
>
> William
>
> >
> > Cheers,
> >
> > Brian
> >
> >
> >
> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote:
> >>
> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
> >> > Alternative notebook frontend authors:
> >> >
> >> > There is an upcoming change to websocket connections to the notebook
> >> > server.
> >> > Instead of three websockets per notebook, one for each zmq channel,
> >> > there is
> >> > a single websocket connection between the server and client. The
> channel
> >> > associated with each message is identified by a channel key in the
> >> > message
> >> > dict itself.
> >> >
> >> > Relevant changes:
> >> >
> >> > url is /channels instead of /shell, etc.
> >> > when sending a message, add a channel key with the channel name
> >> > ('shell',
> >> > 'stdin')
> >> > when receiving a message, check the channel key for routing
> >> >
> >> > -MinRK
> >>
> >> Thanks.
> >>
> >> Of related interest, what the situation involving providing a fallback
> >> to websockets?
> >>
> >> I tried to only support websockets connections with SageMathCloud for
> >> a few weeks, and immediately ran into many complaints from users who
> >> couldn't connect as a result.   This is even with https and so on --
> >> it's just a sad fact that in some corporate or otherwise constrained
> >> environments that websockets don't work.   By far the best approach I
> >> found was to use Primus + Engine.io, which uses websockets if
> >> possible, but will fallback to polling.   So this is what SMC does
> >> now, and everything works, even for people that don't have websockets
> >> as an option... except for IPython-in-SMC of course, which doesn't
> >> work.
> >>
> >> Another unrelated issue is that I'm now hardcoding making this change
> >> around line 171 of IPython/html/notebookapp.py, since I want to serve
> >> the purely static html of IPython from a completely different web
> >> server:
> >>
> >>             #static_url_prefix = url_path_join(base_url,'/static/'),
> >>             static_url_prefix = '/static/ipython/',
> >>
> >> (Sorry for derailing the thread.)
> >>
> >>  -- William
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > IPython-dev mailing list
> >> > IPython-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >
> >>
> >>
> >>
> >> --
> >> William Stein
> >> Professor of Mathematics
> >> University of Washington
> >> http://wstein.org
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> >
> > --
> > Brian E. Granger
> > Cal Poly State University, San Luis Obispo
> > @ellisonbg on Twitter and GitHub
> > bgranger at calpoly.edu and ellisonbg at gmail.com
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/c688ec6e/attachment.html>

From rgbkrk at gmail.com  Thu Jan  8 14:33:52 2015
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Thu, 8 Jan 2015 13:33:52 -0600
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
Message-ID: <CA+tbMaV2tUVnr5HOiXUPddoRmanb1YbqBO99eHJRUrtwtaqC4Q@mail.gmail.com>

You're right Brian, it's about them buying some
$FIREWALL_IPS_SUPER_PRICEY_DEVICE that doesn't proxy websockets
appropriately and the person purchasing having no clue other than the fact
that $VENDOR gave them a neat trinket at a conference.

Firewalls and proxies that do handle websockets appropriately do exist. I'd
be tempted to make a list of ones that do it well and display them
prominently, if only for the benefit of our own users.

I have witnessed these flavors of IPython in the wild (not including our
own recent stunts):

* Corporate entities using an Anaconda server (websockets not a problem
internally but were externally)
* People of all backgrounds using Anaconda's IPython notebook bundling
* Vanilla installations from package managers or pip
* Hosted instance of their own flavor
* DIT4C

Most users that we hear from are using a local installation and there's
probably a large amount of confirmation bias. It would be great if we had
actual shared metrics between us but we only know:

* Unique viewers/users of the Nature demo (> 14,000 total, according to GA)
* Viewers on the notebook viewer (> 200k per month, also according to GA)
* Downloads on PyPI (skewed, not unique)
* Clones from github (also skewed)

We could actually be tracking who has websocket support of those hitting
the notebook viewer for a bigger sense of potential users (since viewers
are not necessarily IPython/Jupyter users).



On Thu, Jan 8, 2015 at 12:55 PM, William Stein <wstein at gmail.com> wrote:

> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com>
> wrote:
> > So far we have felt that it isn't worth the complexity cost for us to
> > support people that deliberately choose to break the internet. It is not
> > like WebSockets is some crazy, insecure, unsupported hack over a weird
> port.
> > It is just different bytes going over the same HTTP/HTTPS socket
> connection
> > and is essentially a formal standard (I know you know this - this is our
> > messaging to companies who complain).
>
> I know -- that's why I switched SMC to pure websockets for a while.
> Then I received complaints at a rate of about 2 very desperate users
> per week, and these were the people who cared enough to complain
> repeatedly.
>
> > Our situation is a bit different though than SMC because most of our
> current
> > users run the notebook on their own computers. This may be something
> that we
> > eventually want to rethink as more people run this in the cloud.
> However, I
> > would like to push back on the companies choosing to break the internet
> > really hard before giving in. I am guessing that in most cases,
> WebSockets
> > are broken in enterprise setting not because of some important deliberate
> > choice, but rather because people don't understand it and are using
> default
> > settings that disable them (I could be wrong though)....
>
> In my experience, for some people that's exactly the problem.  For one
> person I helped it turned out their specific problem was the Windows
> install they had access to had websockets explicitly disabled --
> perhaps that was an IT policy.  For other people the problem is
> further along the stack (e.g., possibly involving caching).
>
> Anyway, this change so that "there is a single websocket connection
> between the server and client. " makes implementing an alternative to
> WebSocket as a fallback dramatically easier, which I greatly
> appreciate.
>
> > Our situation is a bit different though than SMC because most of our
> current
> > users run the notebook on their own computers. This may be something
> that we
>
> Just curious -- how do you know that most people using IPython aren't
> already using it through SMC?
>
> William
>
> >
> > Cheers,
> >
> > Brian
> >
> >
> >
> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote:
> >>
> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
> >> > Alternative notebook frontend authors:
> >> >
> >> > There is an upcoming change to websocket connections to the notebook
> >> > server.
> >> > Instead of three websockets per notebook, one for each zmq channel,
> >> > there is
> >> > a single websocket connection between the server and client. The
> channel
> >> > associated with each message is identified by a channel key in the
> >> > message
> >> > dict itself.
> >> >
> >> > Relevant changes:
> >> >
> >> > url is /channels instead of /shell, etc.
> >> > when sending a message, add a channel key with the channel name
> >> > ('shell',
> >> > 'stdin')
> >> > when receiving a message, check the channel key for routing
> >> >
> >> > -MinRK
> >>
> >> Thanks.
> >>
> >> Of related interest, what the situation involving providing a fallback
> >> to websockets?
> >>
> >> I tried to only support websockets connections with SageMathCloud for
> >> a few weeks, and immediately ran into many complaints from users who
> >> couldn't connect as a result.   This is even with https and so on --
> >> it's just a sad fact that in some corporate or otherwise constrained
> >> environments that websockets don't work.   By far the best approach I
> >> found was to use Primus + Engine.io, which uses websockets if
> >> possible, but will fallback to polling.   So this is what SMC does
> >> now, and everything works, even for people that don't have websockets
> >> as an option... except for IPython-in-SMC of course, which doesn't
> >> work.
> >>
> >> Another unrelated issue is that I'm now hardcoding making this change
> >> around line 171 of IPython/html/notebookapp.py, since I want to serve
> >> the purely static html of IPython from a completely different web
> >> server:
> >>
> >>             #static_url_prefix = url_path_join(base_url,'/static/'),
> >>             static_url_prefix = '/static/ipython/',
> >>
> >> (Sorry for derailing the thread.)
> >>
> >>  -- William
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > IPython-dev mailing list
> >> > IPython-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >
> >>
> >>
> >>
> >> --
> >> William Stein
> >> Professor of Mathematics
> >> University of Washington
> >> http://wstein.org
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> >
> > --
> > Brian E. Granger
> > Cal Poly State University, San Luis Obispo
> > @ellisonbg on Twitter and GitHub
> > bgranger at calpoly.edu and ellisonbg at gmail.com
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/4320a011/attachment.html>

From ellisonbg at gmail.com  Thu Jan  8 17:08:54 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 8 Jan 2015 14:08:54 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
Message-ID: <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>

>
>
> Just curious -- how do you know that most people using IPython aren't
> already using it through SMC?
>

Great question!

As with all such things, if you are asking for a precise data driven
answers, then we don't know.

But here is the constellation of evidence that would lead me to this
conclusion (that SMC represents a minority fraction of the overall number
of ipython users):

* We estimate that the total ipython user base is in the 0.5-1.5 million
range. However, this is likely a very conservative estimate. Fernando has
the best summary of where we get these estimates (I don't remember where
the most recent information is - probably in a private email to someone).
If SMC has this number of users, then my assertion is clearly to be
questioned.
* We spend a significant amount of time out meeting with ipython users an
conferences, universities, research labs, companies. Over this sample, I
know of very few people using SMC (less than 1%). I will be the first to
admit that this is both anecdotal and subject my the sampling bias of my
particular travel schedule...
* Users get and use ipython through a large number of avenues: anaconda,
pip, github, SMC, Canopy, apt-get, yum, etc. Our overall usage numbers come
from trying to estimate the install base of each of these things and adding
them up - we then double check with things like nbviewer usage, etc.
Anecdotally, it appears that anaconda is the most common way that people
get and use ipython, but the most reliable numbers we have come from
apt-get and yum.
* Commercial usage of the notebook (in private with private data) is
quickly overtaking the more open academic and educational uses of the
notebook. The difficulty here is that companies tend to keep their usage
private. However, each time we get a glimpse at commercial usage (which is
happening more and more) we are blown away by the scale of it. My
assumption is that industry usage with private code+data is not the main
audience for SMC.

I want to emphasize that I don't in any way mean to minimize the impact or
importance of SMC. We all think it is fantastic what you are doing and I
personally recommend it to people all the time.

ps - are you willing to share SMC usage stats with us? It would really help
us to reduce the error bars on our usage numbers, which is very helpful in
talking to funding agencies...

Cheers, Brian




-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/0fc1855d/attachment.html>

From wstein at gmail.com  Thu Jan  8 17:33:50 2015
From: wstein at gmail.com (William Stein)
Date: Thu, 8 Jan 2015 14:33:50 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>
Message-ID: <CACLE5GC1w3+anLwAFK6dwg_7OmzR6ULdrpH=rZFVcY6Ee4vqdw@mail.gmail.com>

On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> Just curious -- how do you know that most people using IPython aren't
>> already using it through SMC?
>
>
> Great question!
>
> As with all such things, if you are asking for a precise data driven
> answers, then we don't know.
>
> But here is the constellation of evidence that would lead me to this
> conclusion (that SMC represents a minority fraction of the overall number of
> ipython users):
>
> * We estimate that the total ipython user base is in the 0.5-1.5 million
> range. However, this is likely a very conservative estimate. Fernando has
> the best summary of where we get these estimates (I don't remember where the
> most recent information is - probably in a private email to someone). If SMC
> has this number of users, then my assertion is clearly to be questioned.
> * We spend a significant amount of time out meeting with ipython users an
> conferences, universities, research labs, companies. Over this sample, I
> know of very few people using SMC (less than 1%). I will be the first to
> admit that this is both anecdotal and subject my the sampling bias of my
> particular travel schedule...
> * Users get and use ipython through a large number of avenues: anaconda,
> pip, github, SMC, Canopy, apt-get, yum, etc. Our overall usage numbers come
> from trying to estimate the install base of each of these things and adding
> them up - we then double check with things like nbviewer usage, etc.
> Anecdotally, it appears that anaconda is the most common way that people get
> and use ipython, but the most reliable numbers we have come from apt-get and
> yum.
> * Commercial usage of the notebook (in private with private data) is quickly
> overtaking the more open academic and educational uses of the notebook. The
> difficulty here is that companies tend to keep their usage private. However,
> each time we get a glimpse at commercial usage (which is happening more and
> more) we are blown away by the scale of it. My assumption is that industry
> usage with private code+data is not the main audience for SMC.
>
> I want to emphasize that I don't in any way mean to minimize the impact or
> importance of SMC. We all think it is fantastic what you are doing and I
> personally recommend it to people all the time.
>
> ps - are you willing to share SMC usage stats with us? It would really help
> us to reduce the error bars on our usage numbers, which is very helpful in
> talking to funding agencies...

Thanks for explaining how you measure usage, which I knew nothing
about.  Regarding my SMC usage stats, it's all here:

   http://boxen.math.washington.edu/home/schilly/salvus/stats/stats.html

There is in theory more precise data, but it's not been mined yet.

Usage of SMC appears to be miniscule compared to any of the numbers
you list above.

William

>
> Cheers, Brian
>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From ellisonbg at gmail.com  Thu Jan  8 17:42:52 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 8 Jan 2015 14:42:52 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CACLE5GC1w3+anLwAFK6dwg_7OmzR6ULdrpH=rZFVcY6Ee4vqdw@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>
	<CACLE5GC1w3+anLwAFK6dwg_7OmzR6ULdrpH=rZFVcY6Ee4vqdw@mail.gmail.com>
Message-ID: <CAH4pYpSetKLCzT7--uHEns7mbDonvBc3L+nyT7LoXrmu11XiMQ@mail.gmail.com>

Thanks for sharing this data! This will be really helpful for us!

Cheers, Brian

On Thu, Jan 8, 2015 at 2:33 PM, William Stein <wstein at gmail.com> wrote:

> On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> >>
> >> Just curious -- how do you know that most people using IPython aren't
> >> already using it through SMC?
> >
> >
> > Great question!
> >
> > As with all such things, if you are asking for a precise data driven
> > answers, then we don't know.
> >
> > But here is the constellation of evidence that would lead me to this
> > conclusion (that SMC represents a minority fraction of the overall
> number of
> > ipython users):
> >
> > * We estimate that the total ipython user base is in the 0.5-1.5 million
> > range. However, this is likely a very conservative estimate. Fernando has
> > the best summary of where we get these estimates (I don't remember where
> the
> > most recent information is - probably in a private email to someone). If
> SMC
> > has this number of users, then my assertion is clearly to be questioned.
> > * We spend a significant amount of time out meeting with ipython users an
> > conferences, universities, research labs, companies. Over this sample, I
> > know of very few people using SMC (less than 1%). I will be the first to
> > admit that this is both anecdotal and subject my the sampling bias of my
> > particular travel schedule...
> > * Users get and use ipython through a large number of avenues: anaconda,
> > pip, github, SMC, Canopy, apt-get, yum, etc. Our overall usage numbers
> come
> > from trying to estimate the install base of each of these things and
> adding
> > them up - we then double check with things like nbviewer usage, etc.
> > Anecdotally, it appears that anaconda is the most common way that people
> get
> > and use ipython, but the most reliable numbers we have come from apt-get
> and
> > yum.
> > * Commercial usage of the notebook (in private with private data) is
> quickly
> > overtaking the more open academic and educational uses of the notebook.
> The
> > difficulty here is that companies tend to keep their usage private.
> However,
> > each time we get a glimpse at commercial usage (which is happening more
> and
> > more) we are blown away by the scale of it. My assumption is that
> industry
> > usage with private code+data is not the main audience for SMC.
> >
> > I want to emphasize that I don't in any way mean to minimize the impact
> or
> > importance of SMC. We all think it is fantastic what you are doing and I
> > personally recommend it to people all the time.
> >
> > ps - are you willing to share SMC usage stats with us? It would really
> help
> > us to reduce the error bars on our usage numbers, which is very helpful
> in
> > talking to funding agencies...
>
> Thanks for explaining how you measure usage, which I knew nothing
> about.  Regarding my SMC usage stats, it's all here:
>
>    http://boxen.math.washington.edu/home/schilly/salvus/stats/stats.html
>
> There is in theory more precise data, but it's not been mined yet.
>
> Usage of SMC appears to be miniscule compared to any of the numbers
> you list above.
>
> William
>
> >
> > Cheers, Brian
> >
> >
> >
> >
> > --
> > Brian E. Granger
> > Cal Poly State University, San Luis Obispo
> > @ellisonbg on Twitter and GitHub
> > bgranger at calpoly.edu and ellisonbg at gmail.com
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/b91b47dc/attachment.html>

From fperez.net at gmail.com  Thu Jan  8 18:20:03 2015
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 8 Jan 2015 15:20:03 -0800
Subject: [IPython-dev] [jupyter] Re:  Notebook websocket change
In-Reply-To: <CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>
Message-ID: <CAHAreOrQ+WvU2V2uxKYrho=XGYqSS6F1BrGmZth88kKU0s4Uaw@mail.gmail.com>

On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> * We estimate that the total ipython user base is in the 0.5-1.5 million
> range. However, this is likely a very conservative estimate. Fernando has
> the best summary of where we get these estimates (I don't remember where
> the most recent information is - probably in a private email to someone).


This is how I most recently answered the question "what is the impact and
user base of IPython", indeed in the context of a private email as part of
the EU Sage-math grant proposal:


Getting metrics for usage is very tough, but we have some information. You
can read the report we wrote early this year for Sloan, that provides a
bunch of info (notebook and PDF format):

http://nbviewer.ipython.org/github/ipython/sloan-2013-reports/blob/master/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.ipynb

http://archive.ipython.org/sloan-2013-reports/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.pdf

As you can see, rather than a single blurb of download stats, we paint a
more complex (but I think richer) picture of broad usage and impact.  A few
more things since:

- Recently highlighted in the cover of Nature:

http://www.nature.com/news/interactive-notebooks-sharing-the-code-1.16261

and that article links to a live, interactive demo here:

http://www.nature.com/news/ipython-interactive-demo-7.21492

This demo broke readership records for Nature, to the best of our
understanding (we blew past the estimates the editors gave us in terms of
highest numbers of simultaneous readers to expect).

- The NBviewer site has very significant usage, these are the Google
Analytics numbers for the period Nov 8, 2014-Dec 8, 2014:


Sessions
423,334
Users
215,546
Pageviews
844,523
That's almost 1 million page views for people sharing content as IPython
notebooks (that's the only purpose of that site).

- Our Gallery of notebooks is a good showcase of the breadth of usage and
impact of the notebook across domains:

https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks

- This list shows the adoption of our kernel architecture across languages:
23 independent implementations and counting, meaning this system is
becoming the lingua franca of interactive computing:

https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages

- Here is a list of other projects (in academia and industry) using IPython:

https://github.com/ipython/ipython/wiki/Projects-using-IPython

- It's very hard to get accurate user counts, but we *estimate* at least ~2
Million users.  This is a rough estimate, but I think realistic (and if
anything, a conservative undercount). The estimate goes as follows:

a) Estimates of linux users (not datacenter servers) range from 20M for
Ubuntu to ~70M across distros:

http://en.wikipedia.org/wiki/Ubuntu_(operating_system)#Installed_base
http://en.wikipedia.org/wiki/Linux_adoption#Measuring_desktop_adoption

b) The Debian 'popcon' statistics:

https://qa.debian.org/popcon.php?package=ipython

show IPython to be installed on ~ 5% of Debian installs.

If we use that as a baseline, and estimate total linux user counts at say
~50M (rough average of the above two numbers), we get about 2.5M installs
of IPython on linux. This doesn't count source installs, pip installs, nor
any of the Windows and Mac OSX users who get it via Anaconda, Enthought
Canopy, etc.  Nor does it count the increasing number of server-side hosted
deployments we see more and more of.

But at least it gives a credible way to justify that IPython probably has
*at least* 2M users.





-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/f319b925/attachment.html>

From ellisonbg at gmail.com  Thu Jan  8 18:23:32 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 8 Jan 2015 15:23:32 -0800
Subject: [IPython-dev] [jupyter] Re:  Notebook websocket change
In-Reply-To: <CAHAreOrQ+WvU2V2uxKYrho=XGYqSS6F1BrGmZth88kKU0s4Uaw@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAH4pYpSBZxUR4U9AjoD8LD0FRU2Zn2wAj15dADjLEH6MbGVgBQ@mail.gmail.com>
	<CAHAreOrQ+WvU2V2uxKYrho=XGYqSS6F1BrGmZth88kKU0s4Uaw@mail.gmail.com>
Message-ID: <CAH4pYpSGxeXE3RiHog-mWi6KZhrfixrNLZQ997+QK=syj9TkGA@mail.gmail.com>

Thanks Fernando, that was it exactly!

On Thu, Jan 8, 2015 at 3:20 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> On Thu, Jan 8, 2015 at 2:08 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> * We estimate that the total ipython user base is in the 0.5-1.5 million
>> range. However, this is likely a very conservative estimate. Fernando has
>> the best summary of where we get these estimates (I don't remember where
>> the most recent information is - probably in a private email to someone).
>
>
> This is how I most recently answered the question "what is the impact and
> user base of IPython", indeed in the context of a private email as part of
> the EU Sage-math grant proposal:
>
>
> Getting metrics for usage is very tough, but we have some information. You
> can read the report we wrote early this year for Sloan, that provides a
> bunch of info (notebook and PDF format):
>
>
> http://nbviewer.ipython.org/github/ipython/sloan-2013-reports/blob/master/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.ipynb
>
>
> http://archive.ipython.org/sloan-2013-reports/IPython%202013%20Progress%20Report%20-%20Sloan%20Foundation.pdf
>
> As you can see, rather than a single blurb of download stats, we paint a
> more complex (but I think richer) picture of broad usage and impact.  A few
> more things since:
>
> - Recently highlighted in the cover of Nature:
>
> http://www.nature.com/news/interactive-notebooks-sharing-the-code-1.16261
>
> and that article links to a live, interactive demo here:
>
> http://www.nature.com/news/ipython-interactive-demo-7.21492
>
> This demo broke readership records for Nature, to the best of our
> understanding (we blew past the estimates the editors gave us in terms of
> highest numbers of simultaneous readers to expect).
>
> - The NBviewer site has very significant usage, these are the Google
> Analytics numbers for the period Nov 8, 2014-Dec 8, 2014:
>
>
> Sessions
> 423,334
> Users
> 215,546
> Pageviews
> 844,523
> That's almost 1 million page views for people sharing content as IPython
> notebooks (that's the only purpose of that site).
>
> - Our Gallery of notebooks is a good showcase of the breadth of usage and
> impact of the notebook across domains:
>
>
> https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks
>
> - This list shows the adoption of our kernel architecture across
> languages: 23 independent implementations and counting, meaning this system
> is becoming the lingua franca of interactive computing:
>
> https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages
>
> - Here is a list of other projects (in academia and industry) using
> IPython:
>
> https://github.com/ipython/ipython/wiki/Projects-using-IPython
>
> - It's very hard to get accurate user counts, but we *estimate* at least
> ~2 Million users.  This is a rough estimate, but I think realistic (and if
> anything, a conservative undercount). The estimate goes as follows:
>
> a) Estimates of linux users (not datacenter servers) range from 20M for
> Ubuntu to ~70M across distros:
>
> http://en.wikipedia.org/wiki/Ubuntu_(operating_system)#Installed_base
> http://en.wikipedia.org/wiki/Linux_adoption#Measuring_desktop_adoption
>
> b) The Debian 'popcon' statistics:
>
> https://qa.debian.org/popcon.php?package=ipython
>
> show IPython to be installed on ~ 5% of Debian installs.
>
> If we use that as a baseline, and estimate total linux user counts at say
> ~50M (rough average of the above two numbers), we get about 2.5M installs
> of IPython on linux. This doesn't count source installs, pip installs, nor
> any of the Windows and Mac OSX users who get it via Anaconda, Enthought
> Canopy, etc.  Nor does it count the increasing number of server-side hosted
> deployments we see more and more of.
>
> But at least it gives a credible way to justify that IPython probably has
> *at least* 2M users.
>
>
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscribe at googlegroups.com.
> To post to this group, send email to jupyter at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/CAHAreOrQ%2BWvU2V2uxKYrho%3DXGYqSS6F1BrGmZth88kKU0s4Uaw%40mail.gmail.com
> <https://groups.google.com/d/msgid/jupyter/CAHAreOrQ%2BWvU2V2uxKYrho%3DXGYqSS6F1BrGmZth88kKU0s4Uaw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/6144b025/attachment.html>

From jackson_loper at brown.edu  Thu Jan  8 22:28:03 2015
From: jackson_loper at brown.edu (Loper, Jackson)
Date: Thu, 8 Jan 2015 22:28:03 -0500
Subject: [IPython-dev] threadsafe widgets
Message-ID: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>

It is my understanding that the traitlets in most ipython notebook widgets
aren't threadsafe.  If you're not sure what I mean, I attached a small
example at the bottom of this note.

What's the best way for me to make a threadsafe widget?  I could inherit
from DOMWidget and override a whole bunch of methods with a wrapper that
puts things inside a threading.Lock, but it seems kinda like overkill.   I
suppose theoretically you could lock all the traitlets & comms calls, but
that also seems like overkill.  Or maybe not...I have no sense of
proportion for these things.

The application is straightforward, maybe there's an easier way to do it.

I'd like to be able to edit files from jupyterhub's host directly inside
the notebook; CodeMirror is much more responsive than vim over ssh.  It was
easy to make a text-editing widget based on codemirror, but I'd like to pop
up a CodeMirror.dialog if somebody has changed the file being edited.
Can't figure out how to do that in a threadsafe way.

Happy Thursday!

Jackson Loper
Pattern Theory Group
Division of Applied Math
Brown University

====== threadsome danger =====

Running the latest github clone of jupyterhub, code like this is a very bad
idea:

import IPython.display
from IPython.html import widgets, nbextensions
import threading
from numpy.random import randint
isw=widgets.IntSlider()
IPython.display.display(isw)
def randoset(N):
    for i in range(N):
        isw.value=randint(0,50)
t1=threading.Thread(target=randoset,args=[100])
t2=threading.Thread(target=randoset,args=[100])
t1.start()
t2.start()

The jupyterhub logs errors such as

Malformed message: [b'comm-....', b'<IDS|MSG>', b'comm-...', b'<IDS|MSG>',
... more stuff.. ]

Presumably two attempts at sending data over comms got mixeduptogether
here, which is why we ended up with two <IDS_MSG> in one message!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/e7edef5e/attachment.html>

From fperez.net at gmail.com  Thu Jan  8 23:25:51 2015
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 8 Jan 2015 20:25:51 -0800
Subject: [IPython-dev] threadsafe widgets
In-Reply-To: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
Message-ID: <CAHAreOoLp3tqbbOcC76Xt79vdS+2k9DBuns=HyX_FQ3Eb=ESew@mail.gmail.com>

On Thu, Jan 8, 2015 at 7:28 PM, Loper, Jackson <jackson_loper at brown.edu>
wrote:

> The application is straightforward, maybe there's an easier way to do it.
>
> I'd like to be able to edit files from jupyterhub's host directly inside
> the notebook;
>

Gotta run, so skipping thinking about threads right now :)

But indeed, there's an easier way to do the above: git pull.

In master of IPython, you can create/edit full files, just try it out at
tmpnb.org and let us know how it goes.

Cheers

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150108/059efb28/attachment.html>

From dsdale24 at gmail.com  Fri Jan  9 11:38:29 2015
From: dsdale24 at gmail.com (Darren Dale)
Date: Fri, 09 Jan 2015 16:38:29 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
	<DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov>
	<CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com>
Message-ID: <CAK6O52kd_uAKQ4U8Hs5FS03b9VJ4iFv-8NRPM23pjgYOPtoZvg@mail.gmail.com>

On Sat Jan 03 2015 at 10:45:42 AM Darren Dale <dsdale24 at gmail.com> wrote:

> Hi Ray,
>
> On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> wrote:
>
>>  I don?t really understand what you are trying to achieve, but the
>> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that
>> is used by the console, and I would have thought you would have to do some
>> kind of injection to add other objects from within the application.
>>
>
> What I am trying to achieve is explicitly documented at
> http://ipython.org/ipython-doc/dev/interactive/reference.html#embedding-ipython
> :
>
> ---
>  It is also possible to embed an IPython shell in a namespace in your
> Python code. This allows you to evaluate dynamically the state of your
> code, operate with your variables, analyze them, etc. Note however that any
> changes you make to values while in the shell do not propagate back to the
> running code, so it is safe to modify your values because you won?t break
> your code in bizarre ways by doing so.
>
> Note
> At present, embedding IPython cannot be done from inside IPython. Run the
> code samples below outside IPython.
> [DD: I am not attempting to embed ipython from inside ipython]
>
> This feature allows you to easily have a fully functional python
> environment for doing object introspection anywhere in your code with a
> simple function call. In some cases a simple print statement is enough, but
> if you need to do more detailed analysis of a code fragment this feature
> can be very valuable.
>
> It can also be useful in scientific computing situations where it is
> common to need to do some automatic, computationally intensive part and
> then stop to look at data, plots, etc. Opening an IPython instance will
> give you full access to your data and functions, and you can resume program
> execution once you are done with the interactive part (perhaps to stop
> again later, as many times as needed).
> The following code snippet is the bare minimum you need to include in your
> Python programs for this to work (detailed examples follow later):
>
>    from IPython import embed
>    embed() # this call anywhere in your program will start IPython
>
> You can also embed an IPython kernel, for use with qtconsole, etc. via
> IPython.embed_kernel(). This should function work the same way, but you can
> connect an external frontend (ipython qtconsole or ipython console), rather
> than interacting with it in the terminal.
> ---
>
> This is impressively simple for the embed function:
>
> ---
> C:\Users\darren> python
> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 15:12:11)
> [MSC v.1500 64 bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> Anaconda is brought to you by Continuum Analytics.
> Please check out: http://continuum.io/thanks and https://binstar.org
> >>> from IPython import embed
> >>> a=1
> >>> embed()
> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 15:12:11)
> [MSC v.1500 64 bit (AMD64)]
> Type "copyright", "credits" or "license" for more information.
>
> IPython 2.3.1 -- An enhanced Interactive Python.
> Anaconda is brought to you by Continuum Analytics.
> Please check out: http://continuum.io/thanks and https://binstar.org
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object', use 'object??' for extra details.
>
> In [1]: a
> Out[1]: 1
> ---
>
> But I have not been able to achieve the same behavior with the qt
> in-process console.
>

If this is not possible, perhaps the documentation should be changed?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/6b5208e5/attachment.html>

From jackson_loper at brown.edu  Fri Jan  9 17:59:39 2015
From: jackson_loper at brown.edu (Loper, Jackson)
Date: Fri, 9 Jan 2015 17:59:39 -0500
Subject: [IPython-dev] threadsafe widgets
In-Reply-To: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
Message-ID: <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com>

Ah, the threading situation seems to be more diresome than I thought.  I
believe the following code is unsafe to run.

=== Problem code ===

import IPython.display
from IPython.html import widgets
import threading
from numpy.random import randint

isw1=widgets.IntSlider()
IPython.display.display(isw1)

isw2=widgets.IntSlider()
IPython.display.display(isw2)

def randoset(wid,N):
    for i in range(N):
        wid.value=randint(0,50)

t1=threading.Thread(target=randoset,args=[isw1,100])
t2=threading.Thread(target=randoset,args=[isw2,100])
t1.start()
t2.start()

This code produces malformed message errors for me in the jupyterhub
terminal output.

I believe this is because all widgets seem to share the same iopub_socket
and write to it whenever the user changes a traitlet.  And of course zmq
sockets aren't threadsafe.

=== A few potential solutions ===

1) "Quick" -- Wrap iopub_socket in a threadsafe wrapper.  Surround every
important function call in a mutex.
2) "Hintjens-approved" -- Give every single widget its own inproc PUB
socket.  We'd have to spin up an extra thread (or a greenlet in some
pre-extant thread?  I don't really understand what the different IPython
threads do...but I could find out...) with a SUB socket to collect them all
together and forward them all out on iopub_socket.
3) "Dirty" -- Wrap IPython.kernel.zmq.session.Session.send inside a mutex.
Are there use-cases where anybody besides Session actually does anything
with iopub_socket?

=== Motivation ===

I think there could be valid use-cases for widgets which display results
from polling external resources, e.g. widgets which keep track of EC2
instances, widgets which monitor IPython.parallel workerbees, etc.

=== Conclusion ===

I'd be happy to attack this problem, if you guys think it would be useful.
If so, let me know what would be a good general direction, and I'll put
together a pull request.

Keep warm!

Jackson Loper
Pattern Theory Group
Division of Applied Math
Brown University

PS. I love the new text editor feature in the ipython master branch!  My
days of vim-over-ssh aren't over yet (NERDTree and push-notification of
filechange detection are still too useful to me when logging on from
multiple machines) but I can see the end in sight!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/1084d54e/attachment.html>

From ellisonbg at gmail.com  Fri Jan  9 18:34:43 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 9 Jan 2015 15:34:43 -0800
Subject: [IPython-dev] threadsafe widgets
In-Reply-To: <CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com>
References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
	<CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com>
Message-ID: <CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com>

Yuck :( but I am not surprised at this...

>
>
> I believe this is because all widgets seem to share the same iopub_socket
> and write to it whenever the user changes a traitlet.  And of course zmq
> sockets aren't threadsafe.
>
>
Yep, they definitely share a single iopub socket.


> === A few potential solutions ===
>
> 1) "Quick" -- Wrap iopub_socket in a threadsafe wrapper.  Surround every
> important function call in a mutex.
>

Probably my second choice.


> 2) "Hintjens-approved" -- Give every single widget its own inproc PUB
> socket.  We'd have to spin up an extra thread (or a greenlet in some
> pre-extant thread?  I don't really understand what the different IPython
> threads do...but I could find out...) with a SUB socket to collect them all
> together and forward them all out on iopub_socket.
>

Definitely not going to happen. One of the issues we are running into,
especially in multiuser deployments of the notebook is that file
descriptors get consumed. If anything we are wanting to *reduce* the number
of sockets used by ipython.


> 3) "Dirty" -- Wrap IPython.kernel.zmq.session.Session.send inside a
> mutex.  Are there use-cases where anybody besides Session actually does
> anything with iopub_socket?
>
>
There shouldn't be any other way that send gets called, so this is probably
my top choice.



> === Motivation ===
>
> I think there could be valid use-cases for widgets which display results
> from polling external resources, e.g. widgets which keep track of EC2
> instances, widgets which monitor IPython.parallel workerbees, etc.
>
>
I have already started to use widgets with threads, but in my case I didn't
quite run into issue you are seeing - almost though. This is definitely
going to show up more.


> === Conclusion ===
>
> I'd be happy to attack this problem, if you guys think it would be
> useful.  If so, let me know what would be a good general direction, and
> I'll put together a pull request.
>

@minrk is our expert on the zeromq stuff. Can you open an issue on the
ipython/ipython repo and mention @minrk and @ellisonbg and @jdfreder

Thanks!

Brian


>
> Keep warm!
>
> Jackson Loper
> Pattern Theory Group
> Division of Applied Math
> Brown University
>
> PS. I love the new text editor feature in the ipython master branch!  My
> days of vim-over-ssh aren't over yet (NERDTree and push-notification of
> filechange detection are still too useful to me when logging on from
> multiple machines) but I can see the end in sight!
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/23b886e3/attachment.html>

From fperez.net at gmail.com  Fri Jan  9 18:57:17 2015
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 9 Jan 2015 15:57:17 -0800
Subject: [IPython-dev] threadsafe widgets
In-Reply-To: <CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com>
References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
	<CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com>
	<CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com>
Message-ID: <CAHAreOqFtmaT=sj0obNPBQWVz3wZzcbyjmsUPTnjH+T=0VsJXg@mail.gmail.com>

On Fri, Jan 9, 2015 at 3:34 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> @minrk is our expert on the zeromq stuff. Can you open an issue on the
> ipython/ipython repo and mention @minrk and @ellisonbg and @jdfreder


Add me @fperez as well please, I need to keep an eye on where this goes
too...


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/423951a6/attachment.html>

From benjaminrk at gmail.com  Fri Jan  9 19:25:15 2015
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 9 Jan 2015 16:25:15 -0800
Subject: [IPython-dev] threadsafe widgets
In-Reply-To: <CAHAreOqFtmaT=sj0obNPBQWVz3wZzcbyjmsUPTnjH+T=0VsJXg@mail.gmail.com>
References: <CAP8T2VLH-X8xJ8KF4M-J6R0Ymqo_42rr-fZSAE8Fd_qspVwfPw@mail.gmail.com>
	<CAP8T2VKxJ0S7bMMJ8au+343FCYF7s__nVZOph5BZows2y86u5g@mail.gmail.com>
	<CAH4pYpSxUOxKrzwWw1ptMAFKL_gapFN4XGVHFECoosBsuZTrGA@mail.gmail.com>
	<CAHAreOqFtmaT=sj0obNPBQWVz3wZzcbyjmsUPTnjH+T=0VsJXg@mail.gmail.com>
Message-ID: <CAHNn8BXJNFEsL40EZtF8GvNxGfQ2GOPB4sH2HqiwNeLxXfUVVg@mail.gmail.com>

Since there?s already a single-threaded eventloop running (tornado), the
easiest way to ensure that actions in threads are properly serialized is to
hand them off via IOLoop.add_callback. This has the advantage over the
mutexes-everywhere approach that there isn?t anything to do in the vanilla
single-threaded case.

Where to put this serialization is a bit of an open question. The simplest
place to do it is on Session itself, since it would be always correct, but
Session isn?t always used in the context of an IOLoop, so to do it at that
level would require the most general, and thus most costly, approach (mutex
on every send/recv). We could add a send method on Kernel that does any
threading checks outside of Session, assuming that tornado is involved,
which is always True when there?s a Kernel, unlike for Session.

A mockup of threadsafe send in a tornado context:

def threadsafe_send(*args, **kwargs):
    if not in_main_thread:
        # can't touch zmq sockets except from main thread
        # call me again ASAP from the main thread
        io_loop.add_callback(threadsafe_send, *args, **kwargs)

        return
    actually_send()

The advantage of this over a mutex is that there?s no lock to grab in the
99% case where it is actually called from the main thread.

The issue of mixed-up messages is actually a slightly separate from libzmq
sockets themselves not being threadsafe. This is additional thread-unsafety
provided by pyzmq?s send/recv_multipart, which was allowed because the
underlying C socket is also not threadsafe. Making pyzmq itself threadsafe
was considered (implemented, even), but it?s easy enough to do this at the
application level that pyzmq doesn?t provided it for performance reasons.

-MinRK
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150109/ea36ec3a/attachment.html>

From dsdale24 at gmail.com  Sat Jan 10 09:03:45 2015
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 10 Jan 2015 14:03:45 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
	<DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov>
	<CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com>
	<CAK6O52kd_uAKQ4U8Hs5FS03b9VJ4iFv-8NRPM23pjgYOPtoZvg@mail.gmail.com>
Message-ID: <CAK6O52=JCJFbGcMjTm0bNEPwE_U995KUi04G-QkvRct+G0OF=A@mail.gmail.com>

I found the start of a solution by digging into the IPython library,
especially
https://github.com/ipython/ipython/blob/master/IPython/kernel/zmq/ipkernel.py#L33
. After creating the in-process kernel, it is possible to reinitialize the
console namespace with, for example, `kernel.user_ns = globals()`. Now its
just a matter of wiring up the gui so the user_ns gets updated whenever the
console widget receives focus.
On Fri Jan 09 2015 at 11:38:33 AM Darren Dale <dsdale24 at gmail.com> wrote:

> On Sat Jan 03 2015 at 10:45:42 AM Darren Dale <dsdale24 at gmail.com> wrote:
>
>> Hi Ray,
>>
>> On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov> wrote:
>>
>>>  I don?t really understand what you are trying to achieve, but the
>>> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that
>>> is used by the console, and I would have thought you would have to do some
>>> kind of injection to add other objects from within the application.
>>>
>>
>> What I am trying to achieve is explicitly documented at
>> http://ipython.org/ipython-doc/dev/interactive/reference.
>> html#embedding-ipython :
>>
>> ---
>>  It is also possible to embed an IPython shell in a namespace in your
>> Python code. This allows you to evaluate dynamically the state of your
>> code, operate with your variables, analyze them, etc. Note however that any
>> changes you make to values while in the shell do not propagate back to the
>> running code, so it is safe to modify your values because you won?t break
>> your code in bizarre ways by doing so.
>>
>> Note
>> At present, embedding IPython cannot be done from inside IPython. Run the
>> code samples below outside IPython.
>> [DD: I am not attempting to embed ipython from inside ipython]
>>
>> This feature allows you to easily have a fully functional python
>> environment for doing object introspection anywhere in your code with a
>> simple function call. In some cases a simple print statement is enough, but
>> if you need to do more detailed analysis of a code fragment this feature
>> can be very valuable.
>>
>> It can also be useful in scientific computing situations where it is
>> common to need to do some automatic, computationally intensive part and
>> then stop to look at data, plots, etc. Opening an IPython instance will
>> give you full access to your data and functions, and you can resume program
>> execution once you are done with the interactive part (perhaps to stop
>> again later, as many times as needed).
>> The following code snippet is the bare minimum you need to include in
>> your Python programs for this to work (detailed examples follow later):
>>
>>    from IPython import embed
>>    embed() # this call anywhere in your program will start IPython
>>
>> You can also embed an IPython kernel, for use with qtconsole, etc. via
>> IPython.embed_kernel(). This should function work the same way, but you can
>> connect an external frontend (ipython qtconsole or ipython console), rather
>> than interacting with it in the terminal.
>> ---
>>
>> This is impressively simple for the embed function:
>>
>> ---
>> C:\Users\darren> python
>> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 15:12:11)
>> [MSC v.1500 64 bit (AMD64)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>> Anaconda is brought to you by Continuum Analytics.
>> Please check out: http://continuum.io/thanks and https://binstar.org
>> >>> from IPython import embed
>> >>> a=1
>> >>> embed()
>> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014, 15:12:11)
>> [MSC v.1500 64 bit (AMD64)]
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 2.3.1 -- An enhanced Interactive Python.
>> Anaconda is brought to you by Continuum Analytics.
>> Please check out: http://continuum.io/thanks and https://binstar.org
>> ?         -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help      -> Python's own help system.
>> object?   -> Details about 'object', use 'object??' for extra details.
>>
>> In [1]: a
>> Out[1]: 1
>> ---
>>
>> But I have not been able to achieve the same behavior with the qt
>> in-process console.
>>
>
> If this is not possible, perhaps the documentation should be changed?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150110/48f3a163/attachment.html>

From dsdale24 at gmail.com  Sat Jan 10 10:30:30 2015
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 10 Jan 2015 15:30:30 +0000
Subject: [IPython-dev] embedding ipython, namespace question
References: <CAK6O52=9ek5c3R4hFaSjx3+BOCVMcQ41ZhTJOmidmcB6oqxY4w@mail.gmail.com>
	<AA9919BA-8597-4AAC-A5FC-DF6C87DDCC5A@anl.gov>
	<CAK6O52k-Wr6VFTra=hb20cKaqd_4AP4yw8Zqs6w-4-EchwEcEQ@mail.gmail.com>
	<FABE7266-DBCA-4538-8E23-F4950AE8965C@gmail.com>
	<CAK6O52mFdzqxh5+xSUoet5JPmTerpJfXw3gk0ER1kk4S1T-SWg@mail.gmail.com>
	<DD2429D4-4BE1-49FC-A891-BF153FBA942F@anl.gov>
	<CAK6O52ns=WiaMSGRista9wE_VHdgFap1Wt65h1yDwEECawAULA@mail.gmail.com>
	<CAK6O52kd_uAKQ4U8Hs5FS03b9VJ4iFv-8NRPM23pjgYOPtoZvg@mail.gmail.com>
	<CAK6O52=JCJFbGcMjTm0bNEPwE_U995KUi04G-QkvRct+G0OF=A@mail.gmail.com>
Message-ID: <CAK6O52ma0DKMSUmgVf5EdnPv6xsGX5gMj1BJTScr3W7767uYhQ@mail.gmail.com>

In case anyone else may find it useful, attached is a small modification of
the qt in-process example that implements the behavior I was looking for.
The namespace gets updated each time the mouse enters the widget, so every
time you use the console, the environment appears to be the same as that of
the GUI.

On Sat Jan 10 2015 at 9:03:48 AM Darren Dale <dsdale24 at gmail.com> wrote:

> I found the start of a solution by digging into the IPython library,
> especially https://github.com/ipython/ipython/blob/master/IPython/
> kernel/zmq/ipkernel.py#L33 . After creating the in-process kernel, it is
> possible to reinitialize the console namespace with, for example,
> `kernel.user_ns = globals()`. Now its just a matter of wiring up the gui so
> the user_ns gets updated whenever the console widget receives focus.
> On Fri Jan 09 2015 at 11:38:33 AM Darren Dale <dsdale24 at gmail.com> wrote:
>
>> On Sat Jan 03 2015 at 10:45:42 AM Darren Dale <dsdale24 at gmail.com> wrote:
>>
>>> Hi Ray,
>>>
>>> On Sat Dec 27 2014 at 8:48:20 PM Osborn, Raymond <rosborn at anl.gov>
>>> wrote:
>>>
>>>>  I don?t really understand what you are trying to achieve, but the
>>>> ?user_ns? dictionary isn?t an isolated namespace - it?s the namespace that
>>>> is used by the console, and I would have thought you would have to do some
>>>> kind of injection to add other objects from within the application.
>>>>
>>>
>>> What I am trying to achieve is explicitly documented at
>>> http://ipython.org/ipython-doc/dev/interactive/reference.htm
>>> l#embedding-ipython :
>>>
>>> ---
>>>  It is also possible to embed an IPython shell in a namespace in your
>>> Python code. This allows you to evaluate dynamically the state of your
>>> code, operate with your variables, analyze them, etc. Note however that any
>>> changes you make to values while in the shell do not propagate back to the
>>> running code, so it is safe to modify your values because you won?t break
>>> your code in bizarre ways by doing so.
>>>
>>> Note
>>> At present, embedding IPython cannot be done from inside IPython. Run
>>> the code samples below outside IPython.
>>> [DD: I am not attempting to embed ipython from inside ipython]
>>>
>>> This feature allows you to easily have a fully functional python
>>> environment for doing object introspection anywhere in your code with a
>>> simple function call. In some cases a simple print statement is enough, but
>>> if you need to do more detailed analysis of a code fragment this feature
>>> can be very valuable.
>>>
>>> It can also be useful in scientific computing situations where it is
>>> common to need to do some automatic, computationally intensive part and
>>> then stop to look at data, plots, etc. Opening an IPython instance will
>>> give you full access to your data and functions, and you can resume program
>>> execution once you are done with the interactive part (perhaps to stop
>>> again later, as many times as needed).
>>> The following code snippet is the bare minimum you need to include in
>>> your Python programs for this to work (detailed examples follow later):
>>>
>>>    from IPython import embed
>>>    embed() # this call anywhere in your program will start IPython
>>>
>>> You can also embed an IPython kernel, for use with qtconsole, etc. via
>>> IPython.embed_kernel(). This should function work the same way, but you can
>>> connect an external frontend (ipython qtconsole or ipython console), rather
>>> than interacting with it in the terminal.
>>> ---
>>>
>>> This is impressively simple for the embed function:
>>>
>>> ---
>>> C:\Users\darren> python
>>> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014,
>>> 15:12:11) [MSC v.1500 64 bit (AMD64)] on win32
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> Anaconda is brought to you by Continuum Analytics.
>>> Please check out: http://continuum.io/thanks and https://binstar.org
>>> >>> from IPython import embed
>>> >>> a=1
>>> >>> embed()
>>> Python 2.7.8 |Continuum Analytics, Inc.| (default, Jul  2 2014,
>>> 15:12:11) [MSC v.1500 64 bit (AMD64)]
>>> Type "copyright", "credits" or "license" for more information.
>>>
>>> IPython 2.3.1 -- An enhanced Interactive Python.
>>> Anaconda is brought to you by Continuum Analytics.
>>> Please check out: http://continuum.io/thanks and https://binstar.org
>>> ?         -> Introduction and overview of IPython's features.
>>> %quickref -> Quick reference.
>>> help      -> Python's own help system.
>>> object?   -> Details about 'object', use 'object??' for extra details.
>>>
>>> In [1]: a
>>> Out[1]: 1
>>> ---
>>>
>>> But I have not been able to achieve the same behavior with the qt
>>> in-process console.
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150110/699cf809/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: inprocess_qtconsole.py
Type: application/octet-stream
Size: 1457 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150110/699cf809/attachment.obj>

From dsdale24 at gmail.com  Sun Jan 11 10:37:22 2015
From: dsdale24 at gmail.com (Darren Dale)
Date: Sun, 11 Jan 2015 15:37:22 +0000
Subject: [IPython-dev] Incompatible subclass instances of
	InteractiveShellEmbed
Message-ID: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com>

Hello,

I'm working on an application with an in-process qt console, following the
example at
https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py
. I would also like to be able to use the `embed` function occassionally
for troubleshooting other parts of the application that are unrelated to
the in-process qt console. However, calling embed yields an error:
"IPython.config.configurable.MultipleInstanceError: Multiple incompatible
subclass instances of InteractiveShellEmbed are being created."  I can
reproduce this with the inprocess example by adding a call to `embed()`
just before the call to `guisupport.start_event_loop_qt4(app)`. Any ideas
how to make this work?

Thanks,
Darren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150111/ca47c719/attachment.html>

From rosborn at anl.gov  Sun Jan 11 17:49:08 2015
From: rosborn at anl.gov (Osborn, Raymond)
Date: Sun, 11 Jan 2015 22:49:08 +0000
Subject: [IPython-dev] Incompatible subclass instances
	of	InteractiveShellEmbed
In-Reply-To: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com>
References: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com>
Message-ID: <7191368A-9B31-4D73-BB7B-D10ACE44F31D@anl.gov>

Until someone more knowledgeable replies, this is presumably because InteractiveShellEmbed ultimately inherits from SingletonConfigurable, which only allows one instance to be created. There is some code in embed.py, which saves an existing instance, clears it, and allows another to be created:

    saved_shell_instance = InteractiveShell._instance
    if saved_shell_instance is not None:
        cls = type(saved_shell_instance)
        cls.clear_instance()
    shell = InteractiveShellEmbed.instance(**kwargs)

If something like that is not in your current code, perhaps you need to add it.

Ray

On Jan 11, 2015, at 9:37 AM, Darren Dale <dsdale24 at gmail.com<mailto:dsdale24 at gmail.com>> wrote:

Hello,

I'm working on an application with an in-process qt console, following the example at https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py . I would also like to be able to use the `embed` function occassionally for troubleshooting other parts of the application that are unrelated to the in-process qt console. However, calling embed yields an error: "IPython.config.configurable.MultipleInstanceError: Multiple incompatible subclass instances of InteractiveShellEmbed are being created."  I can reproduce this with the inprocess example by adding a call to `embed()` just before the call to `guisupport.start_event_loop_qt4(app)`. Any ideas how to make this work?

Thanks,
Darren
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org<mailto:IPython-dev at scipy.org>
http://mail.scipy.org/mailman/listinfo/ipython-dev

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150111/803c3fe8/attachment.html>

From fperez.net at gmail.com  Sun Jan 11 22:47:22 2015
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 11 Jan 2015 19:47:22 -0800
Subject: [IPython-dev] IPython 3.0, the homestretch. We need your help!
Message-ID: <CAHAreOppTKvawbCj+4k+qd7-Sg3cLKTLpUFpcGbnbMn7NzuaVA@mail.gmail.com>

Hi all,

we're behind schedule, but there is light at the end of the 3.0 tunnel.
Now is the time where we need the help from our community. This is the list
of currently open issues tagged for 3.0:

https://github.com/ipython/ipython/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0

If you see anything there for which you can contribute in any way a fix,
patch, or even constructive feedback, we'd be very grateful.

Eventually some of those will inevitably have to be retargetted for
3.1/4.0, since we won't be able to get to all of them. But there's a reason
we'd labeled them 3.0, so it would be great to actually close as many of
them as possible (or at least make partial progress on them, enough to
address the key problem and perhaps leave an easier but potentially longer
tail for 3.1/4).

So please jump in!

f

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150111/54c7a75c/attachment.html>

From dsdale24 at gmail.com  Mon Jan 12 12:28:34 2015
From: dsdale24 at gmail.com (Darren Dale)
Date: Mon, 12 Jan 2015 17:28:34 +0000
Subject: [IPython-dev] Incompatible subclass instances of
	InteractiveShellEmbed
References: <CAK6O52=wGAMhamdKToM7WLoxDDjS5hEHV6mL_m_H-KF2uVyu1w@mail.gmail.com>
	<7191368A-9B31-4D73-BB7B-D10ACE44F31D@anl.gov>
Message-ID: <CAK6O52=uTzgpGgRLDJJxkHrp_eihB9pA934Cz3_zkhwuBd_Pzw@mail.gmail.com>

The following seems to work for me: in my main hexrd/__init__.py, I do
`from IPython import embed as debug`. Then, in my gui module, I monkeypatch
hexrd.debug so it uses the inprocess qtconsole:

        class Debug(object):
            def __init__(self, shell):
                self.shell = shell
            def __call__(self):
                import inspect
                frame = inspect.currentframe()
                try:
                    # prepare the debug namespace
                    temp = frame.f_back.f_globals
                    temp.update(frame.f_back.f_locals)
                finally:
                    del frame
                # clear and repopulate the qtconsole namespace
                self.shell.run_line_magic('reset', '-f -s')
                self.shell.push(temp)

        hexrd.debug = Debug(kernel_manager.kernel.shell)

Darren

On Sun Jan 11 2015 at 5:49:16 PM Osborn, Raymond <rosborn at anl.gov> wrote:

>  Until someone more knowledgeable replies, this is presumably because
> InteractiveShellEmbed ultimately inherits from SingletonConfigurable, which
> only allows one instance to be created. There is some code in embed.py,
> which saves an existing instance, clears it, and allows another to be
> created:
>
>      saved_shell_instance = InteractiveShell._instance
>     if saved_shell_instance is not None:
>         cls = type(saved_shell_instance)
>         cls.clear_instance()
>     shell = InteractiveShellEmbed.instance(**kwargs)
>
>  If something like that is not in your current code, perhaps you need to
> add it.
>
>  Ray
>
>  On Jan 11, 2015, at 9:37 AM, Darren Dale <dsdale24 at gmail.com> wrote:
>
> Hello,
>
>  I'm working on an application with an in-process qt console, following
> the example at
> https://github.com/ipython/ipython/blob/master/examples/Embedding/inprocess_qtconsole.py
> . I would also like to be able to use the `embed` function occassionally
> for troubleshooting other parts of the application that are unrelated to
> the in-process qt console. However, calling embed yields an error:
> "IPython.config.configurable.MultipleInstanceError: Multiple incompatible
> subclass instances of InteractiveShellEmbed are being created."  I can
> reproduce this with the inprocess example by adding a call to `embed()`
> just before the call to `guisupport.start_event_loop_qt4(app)`. Any ideas
> how to make this work?
>
>  Thanks,
> Darren
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> --
> Ray Osborn, Senior Scientist
> Materials Science Division
> Argonne National Laboratory
> Argonne, IL 60439, USA
> Phone: +1 (630) 252-9011
> Email: ROsborn at anl.gov
>
>
>   _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150112/7d46578d/attachment.html>

From wstein at gmail.com  Wed Jan 14 19:12:50 2015
From: wstein at gmail.com (William Stein)
Date: Wed, 14 Jan 2015 16:12:50 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com>
Message-ID: <CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com>

On Thu, Jan 8, 2015 at 11:18 AM, MinRK <benjaminrk at gmail.com> wrote:
> Some time ago, I made a PR to IPython switching to socket.io, which required
> the same basic changes since it only allowed a single connection per page,
> so I know it's not too hard. We may start to see more pressure as hosted
> instances, and reconsider the fallback in the future, but at this point, we
> don't feel that pressure. Wind proper websockets to be sufficiently
> preferable that we will wait as long as we can before appeasing old, broken,
> or misconfigured network environments that prevent websockets from working.
>
> The majority of notebook users seem to be running as a desktop app via
> localhost, where websockets are rarely a problem (The nightmare of Sophos
> AV/firewall on Windows excluded).

Do you have a pointer to the above pull request?

In my experience battling with websockets and the cloud over the last
few years, it's not just an issue of websockets versus fallbacks --
there's also issues with the robustness of the websocket connection,
how it's maintained,  automatically reconnecting if the connection has
problems, heartbeats, etc.   If you only use IPython locally then none
of this is a problem.  However, if you use IPython over a complicated
network that drops packets sometimes, flakes out, etc., it can be.   I
get fairly regular emails from people having trouble specifically with
the robustness of the iPython connection inside of SMC.    Libraries
like socket.io and engine.io [1] seem to have pretty good debuged and
thought out code for dealing with these situations (and for fallbacks
in case websockets doesn't work).

Since this use case is only a tiny part of the total IPython notebook
usage, it makes sense for me to take this on myself.   I will
definitely highly prioritize trying to hack on IPython in order to get
it to use engine.io+primus, since I have to deal with this websocket
robustness situation to improve my user's experience.   For example,
I'm getting fairly regular complaints from Randy Leveque and his
colleagues at UW.     So I'd greatly appreciate a link to that pull
request, and of course I'll share anything I do here.

[1] https://github.com/Automattic/engine.io

>
> -MinRK
>
> On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote:
>>
>> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com>
>> wrote:
>> > So far we have felt that it isn't worth the complexity cost for us to
>> > support people that deliberately choose to break the internet. It is not
>> > like WebSockets is some crazy, insecure, unsupported hack over a weird
>> > port.
>> > It is just different bytes going over the same HTTP/HTTPS socket
>> > connection
>> > and is essentially a formal standard (I know you know this - this is our
>> > messaging to companies who complain).
>>
>> I know -- that's why I switched SMC to pure websockets for a while.
>> Then I received complaints at a rate of about 2 very desperate users
>> per week, and these were the people who cared enough to complain
>> repeatedly.
>>
>> > Our situation is a bit different though than SMC because most of our
>> > current
>> > users run the notebook on their own computers. This may be something
>> > that we
>> > eventually want to rethink as more people run this in the cloud.
>> > However, I
>> > would like to push back on the companies choosing to break the internet
>> > really hard before giving in. I am guessing that in most cases,
>> > WebSockets
>> > are broken in enterprise setting not because of some important
>> > deliberate
>> > choice, but rather because people don't understand it and are using
>> > default
>> > settings that disable them (I could be wrong though)....
>>
>> In my experience, for some people that's exactly the problem.  For one
>> person I helped it turned out their specific problem was the Windows
>> install they had access to had websockets explicitly disabled --
>> perhaps that was an IT policy.  For other people the problem is
>> further along the stack (e.g., possibly involving caching).
>>
>> Anyway, this change so that "there is a single websocket connection
>> between the server and client. " makes implementing an alternative to
>> WebSocket as a fallback dramatically easier, which I greatly
>> appreciate.
>>
>> > Our situation is a bit different though than SMC because most of our
>> > current
>> > users run the notebook on their own computers. This may be something
>> > that we
>>
>> Just curious -- how do you know that most people using IPython aren't
>> already using it through SMC?
>>
>> William
>>
>> >
>> > Cheers,
>> >
>> > Brian
>> >
>> >
>> >
>> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com> wrote:
>> >>
>> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
>> >> > Alternative notebook frontend authors:
>> >> >
>> >> > There is an upcoming change to websocket connections to the notebook
>> >> > server.
>> >> > Instead of three websockets per notebook, one for each zmq channel,
>> >> > there is
>> >> > a single websocket connection between the server and client. The
>> >> > channel
>> >> > associated with each message is identified by a channel key in the
>> >> > message
>> >> > dict itself.
>> >> >
>> >> > Relevant changes:
>> >> >
>> >> > url is /channels instead of /shell, etc.
>> >> > when sending a message, add a channel key with the channel name
>> >> > ('shell',
>> >> > 'stdin')
>> >> > when receiving a message, check the channel key for routing
>> >> >
>> >> > -MinRK
>> >>
>> >> Thanks.
>> >>
>> >> Of related interest, what the situation involving providing a fallback
>> >> to websockets?
>> >>
>> >> I tried to only support websockets connections with SageMathCloud for
>> >> a few weeks, and immediately ran into many complaints from users who
>> >> couldn't connect as a result.   This is even with https and so on --
>> >> it's just a sad fact that in some corporate or otherwise constrained
>> >> environments that websockets don't work.   By far the best approach I
>> >> found was to use Primus + Engine.io, which uses websockets if
>> >> possible, but will fallback to polling.   So this is what SMC does
>> >> now, and everything works, even for people that don't have websockets
>> >> as an option... except for IPython-in-SMC of course, which doesn't
>> >> work.
>> >>
>> >> Another unrelated issue is that I'm now hardcoding making this change
>> >> around line 171 of IPython/html/notebookapp.py, since I want to serve
>> >> the purely static html of IPython from a completely different web
>> >> server:
>> >>
>> >>             #static_url_prefix = url_path_join(base_url,'/static/'),
>> >>             static_url_prefix = '/static/ipython/',
>> >>
>> >> (Sorry for derailing the thread.)
>> >>
>> >>  -- William
>> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > IPython-dev mailing list
>> >> > IPython-dev at scipy.org
>> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> William Stein
>> >> Professor of Mathematics
>> >> University of Washington
>> >> http://wstein.org
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> >
>> > --
>> > Brian E. Granger
>> > Cal Poly State University, San Luis Obispo
>> > @ellisonbg on Twitter and GitHub
>> > bgranger at calpoly.edu and ellisonbg at gmail.com
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From benjaminrk at gmail.com  Wed Jan 14 19:50:46 2015
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 14 Jan 2015 16:50:46 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com>
	<CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com>
Message-ID: <CAHNn8BVvt+wFwt5u1OvQ-6D2fiGenscaAdFaQd729gp3WXVQRw@mail.gmail.com>

The PR is 2321 <https://github.com/ipython/ipython/pull/2321>, switching
IPython to use SockJS instead of WebSockets. I?m not sure how informative
it is, since things have moved around *a lot* since then, and it is from a
time when we were shipping more things in external, rather than accepting
them as dependencies. I think a similar patch today would be a great deal
simpler, especially now that it doesn?t need to include merging sockets. I
also had to include a hack to workaround the fact that SockJS didn?t allow
empty messages, and we were sending empty messages as part of the initial
handshake (also not true anymore).

-MinRK
?

On Wed, Jan 14, 2015 at 4:12 PM, William Stein <wstein at gmail.com> wrote:

> On Thu, Jan 8, 2015 at 11:18 AM, MinRK <benjaminrk at gmail.com> wrote:
> > Some time ago, I made a PR to IPython switching to socket.io, which
> required
> > the same basic changes since it only allowed a single connection per
> page,
> > so I know it's not too hard. We may start to see more pressure as hosted
> > instances, and reconsider the fallback in the future, but at this point,
> we
> > don't feel that pressure. Wind proper websockets to be sufficiently
> > preferable that we will wait as long as we can before appeasing old,
> broken,
> > or misconfigured network environments that prevent websockets from
> working.
> >
> > The majority of notebook users seem to be running as a desktop app via
> > localhost, where websockets are rarely a problem (The nightmare of Sophos
> > AV/firewall on Windows excluded).
>
> Do you have a pointer to the above pull request?
>
> In my experience battling with websockets and the cloud over the last
> few years, it's not just an issue of websockets versus fallbacks --
> there's also issues with the robustness of the websocket connection,
> how it's maintained,  automatically reconnecting if the connection has
> problems, heartbeats, etc.   If you only use IPython locally then none
> of this is a problem.  However, if you use IPython over a complicated
> network that drops packets sometimes, flakes out, etc., it can be.   I
> get fairly regular emails from people having trouble specifically with
> the robustness of the iPython connection inside of SMC.    Libraries
> like socket.io and engine.io [1] seem to have pretty good debuged and
> thought out code for dealing with these situations (and for fallbacks
> in case websockets doesn't work).
>
> Since this use case is only a tiny part of the total IPython notebook
> usage, it makes sense for me to take this on myself.   I will
> definitely highly prioritize trying to hack on IPython in order to get
> it to use engine.io+primus, since I have to deal with this websocket
> robustness situation to improve my user's experience.   For example,
> I'm getting fairly regular complaints from Randy Leveque and his
> colleagues at UW.     So I'd greatly appreciate a link to that pull
> request, and of course I'll share anything I do here.
>
> [1] https://github.com/Automattic/engine.io
>
> >
> > -MinRK
> >
> > On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote:
> >>
> >> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com>
> >> wrote:
> >> > So far we have felt that it isn't worth the complexity cost for us to
> >> > support people that deliberately choose to break the internet. It is
> not
> >> > like WebSockets is some crazy, insecure, unsupported hack over a weird
> >> > port.
> >> > It is just different bytes going over the same HTTP/HTTPS socket
> >> > connection
> >> > and is essentially a formal standard (I know you know this - this is
> our
> >> > messaging to companies who complain).
> >>
> >> I know -- that's why I switched SMC to pure websockets for a while.
> >> Then I received complaints at a rate of about 2 very desperate users
> >> per week, and these were the people who cared enough to complain
> >> repeatedly.
> >>
> >> > Our situation is a bit different though than SMC because most of our
> >> > current
> >> > users run the notebook on their own computers. This may be something
> >> > that we
> >> > eventually want to rethink as more people run this in the cloud.
> >> > However, I
> >> > would like to push back on the companies choosing to break the
> internet
> >> > really hard before giving in. I am guessing that in most cases,
> >> > WebSockets
> >> > are broken in enterprise setting not because of some important
> >> > deliberate
> >> > choice, but rather because people don't understand it and are using
> >> > default
> >> > settings that disable them (I could be wrong though)....
> >>
> >> In my experience, for some people that's exactly the problem.  For one
> >> person I helped it turned out their specific problem was the Windows
> >> install they had access to had websockets explicitly disabled --
> >> perhaps that was an IT policy.  For other people the problem is
> >> further along the stack (e.g., possibly involving caching).
> >>
> >> Anyway, this change so that "there is a single websocket connection
> >> between the server and client. " makes implementing an alternative to
> >> WebSocket as a fallback dramatically easier, which I greatly
> >> appreciate.
> >>
> >> > Our situation is a bit different though than SMC because most of our
> >> > current
> >> > users run the notebook on their own computers. This may be something
> >> > that we
> >>
> >> Just curious -- how do you know that most people using IPython aren't
> >> already using it through SMC?
> >>
> >> William
> >>
> >> >
> >> > Cheers,
> >> >
> >> > Brian
> >> >
> >> >
> >> >
> >> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com>
> wrote:
> >> >>
> >> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
> >> >> > Alternative notebook frontend authors:
> >> >> >
> >> >> > There is an upcoming change to websocket connections to the
> notebook
> >> >> > server.
> >> >> > Instead of three websockets per notebook, one for each zmq channel,
> >> >> > there is
> >> >> > a single websocket connection between the server and client. The
> >> >> > channel
> >> >> > associated with each message is identified by a channel key in the
> >> >> > message
> >> >> > dict itself.
> >> >> >
> >> >> > Relevant changes:
> >> >> >
> >> >> > url is /channels instead of /shell, etc.
> >> >> > when sending a message, add a channel key with the channel name
> >> >> > ('shell',
> >> >> > 'stdin')
> >> >> > when receiving a message, check the channel key for routing
> >> >> >
> >> >> > -MinRK
> >> >>
> >> >> Thanks.
> >> >>
> >> >> Of related interest, what the situation involving providing a
> fallback
> >> >> to websockets?
> >> >>
> >> >> I tried to only support websockets connections with SageMathCloud for
> >> >> a few weeks, and immediately ran into many complaints from users who
> >> >> couldn't connect as a result.   This is even with https and so on --
> >> >> it's just a sad fact that in some corporate or otherwise constrained
> >> >> environments that websockets don't work.   By far the best approach I
> >> >> found was to use Primus + Engine.io, which uses websockets if
> >> >> possible, but will fallback to polling.   So this is what SMC does
> >> >> now, and everything works, even for people that don't have websockets
> >> >> as an option... except for IPython-in-SMC of course, which doesn't
> >> >> work.
> >> >>
> >> >> Another unrelated issue is that I'm now hardcoding making this change
> >> >> around line 171 of IPython/html/notebookapp.py, since I want to serve
> >> >> the purely static html of IPython from a completely different web
> >> >> server:
> >> >>
> >> >>             #static_url_prefix = url_path_join(base_url,'/static/'),
> >> >>             static_url_prefix = '/static/ipython/',
> >> >>
> >> >> (Sorry for derailing the thread.)
> >> >>
> >> >>  -- William
> >> >>
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > IPython-dev mailing list
> >> >> > IPython-dev at scipy.org
> >> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> William Stein
> >> >> Professor of Mathematics
> >> >> University of Washington
> >> >> http://wstein.org
> >> >> _______________________________________________
> >> >> IPython-dev mailing list
> >> >> IPython-dev at scipy.org
> >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Brian E. Granger
> >> > Cal Poly State University, San Luis Obispo
> >> > @ellisonbg on Twitter and GitHub
> >> > bgranger at calpoly.edu and ellisonbg at gmail.com
> >> >
> >> > _______________________________________________
> >> > IPython-dev mailing list
> >> > IPython-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >
> >>
> >>
> >>
> >> --
> >> William Stein
> >> Professor of Mathematics
> >> University of Washington
> >> http://wstein.org
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150114/4a3f1ac5/attachment.html>

From wstein at gmail.com  Wed Jan 14 19:55:21 2015
From: wstein at gmail.com (William Stein)
Date: Wed, 14 Jan 2015 16:55:21 -0800
Subject: [IPython-dev] Notebook websocket change
In-Reply-To: <CAHNn8BVvt+wFwt5u1OvQ-6D2fiGenscaAdFaQd729gp3WXVQRw@mail.gmail.com>
References: <CAHNn8BVQ89k4DikAqYitjUQMAArZzmpPfQHVWayqA-7v+b+SHg@mail.gmail.com>
	<CACLE5GDm+rvTc2J3j6m5YFYE0dMCG1rcX8pCFW11sMMqEtk2yg@mail.gmail.com>
	<CAH4pYpTh00tPTQ2hfv5mC2FftfZnt1SbJ5ZmrqUwoViKOqLgdQ@mail.gmail.com>
	<CACLE5GA8-TPwou9fO-jpfMTPv1aCckwpVaPwexiqFgJ-P69tgw@mail.gmail.com>
	<CAHNn8BUSrDb8kVO94-dv9FsNJtZu=u=dW2QMdLgEn5_B3v4CRQ@mail.gmail.com>
	<CACLE5GA4G74y5V_WCurA6tYS76yWWgVwnMJ+eZTVENHqUNzODg@mail.gmail.com>
	<CAHNn8BVvt+wFwt5u1OvQ-6D2fiGenscaAdFaQd729gp3WXVQRw@mail.gmail.com>
Message-ID: <CACLE5GAaNC_n9q-Nr+y+CTyz9T8u71iSZNe4LTT-bsjWV=zyHw@mail.gmail.com>

On Wed, Jan 14, 2015 at 4:50 PM, MinRK <benjaminrk at gmail.com> wrote:
> The PR is 2321, switching IPython to use SockJS instead of WebSockets. I'm
> not sure how informative it is, since things have moved around a lot since
> then, and it is from a time when we were shipping more things in external,
> rather than accepting them as dependencies. I think a similar patch today
> would be a great deal simpler, especially now that it doesn't need to
> include merging sockets. I also had to include a hack to workaround the fact
> that SockJS didn't allow empty messages, and we were sending empty messages
> as part of the initial handshake (also not true anymore).
>
> -MinRK

Thanks!  I'll carefully think through a range of options.  Another
might be to just plug into your code to send/receive messages, and
just send/receive them over my existing websocket connection, so there
is just a single websocket connection, shared across ipython and smc.

 -- William

>
>
> On Wed, Jan 14, 2015 at 4:12 PM, William Stein <wstein at gmail.com> wrote:
>>
>> On Thu, Jan 8, 2015 at 11:18 AM, MinRK <benjaminrk at gmail.com> wrote:
>> > Some time ago, I made a PR to IPython switching to socket.io, which
>> > required
>> > the same basic changes since it only allowed a single connection per
>> > page,
>> > so I know it's not too hard. We may start to see more pressure as hosted
>> > instances, and reconsider the fallback in the future, but at this point,
>> > we
>> > don't feel that pressure. Wind proper websockets to be sufficiently
>> > preferable that we will wait as long as we can before appeasing old,
>> > broken,
>> > or misconfigured network environments that prevent websockets from
>> > working.
>> >
>> > The majority of notebook users seem to be running as a desktop app via
>> > localhost, where websockets are rarely a problem (The nightmare of
>> > Sophos
>> > AV/firewall on Windows excluded).
>>
>> Do you have a pointer to the above pull request?
>>
>> In my experience battling with websockets and the cloud over the last
>> few years, it's not just an issue of websockets versus fallbacks --
>> there's also issues with the robustness of the websocket connection,
>> how it's maintained,  automatically reconnecting if the connection has
>> problems, heartbeats, etc.   If you only use IPython locally then none
>> of this is a problem.  However, if you use IPython over a complicated
>> network that drops packets sometimes, flakes out, etc., it can be.   I
>> get fairly regular emails from people having trouble specifically with
>> the robustness of the iPython connection inside of SMC.    Libraries
>> like socket.io and engine.io [1] seem to have pretty good debuged and
>> thought out code for dealing with these situations (and for fallbacks
>> in case websockets doesn't work).
>>
>> Since this use case is only a tiny part of the total IPython notebook
>> usage, it makes sense for me to take this on myself.   I will
>> definitely highly prioritize trying to hack on IPython in order to get
>> it to use engine.io+primus, since I have to deal with this websocket
>> robustness situation to improve my user's experience.   For example,
>> I'm getting fairly regular complaints from Randy Leveque and his
>> colleagues at UW.     So I'd greatly appreciate a link to that pull
>> request, and of course I'll share anything I do here.
>>
>> [1] https://github.com/Automattic/engine.io
>>
>> >
>> > -MinRK
>> >
>> > On Thu, Jan 8, 2015 at 10:55 AM, William Stein <wstein at gmail.com> wrote:
>> >>
>> >> On Thu, Jan 8, 2015 at 10:19 AM, Brian Granger <ellisonbg at gmail.com>
>> >> wrote:
>> >> > So far we have felt that it isn't worth the complexity cost for us to
>> >> > support people that deliberately choose to break the internet. It is
>> >> > not
>> >> > like WebSockets is some crazy, insecure, unsupported hack over a
>> >> > weird
>> >> > port.
>> >> > It is just different bytes going over the same HTTP/HTTPS socket
>> >> > connection
>> >> > and is essentially a formal standard (I know you know this - this is
>> >> > our
>> >> > messaging to companies who complain).
>> >>
>> >> I know -- that's why I switched SMC to pure websockets for a while.
>> >> Then I received complaints at a rate of about 2 very desperate users
>> >> per week, and these were the people who cared enough to complain
>> >> repeatedly.
>> >>
>> >> > Our situation is a bit different though than SMC because most of our
>> >> > current
>> >> > users run the notebook on their own computers. This may be something
>> >> > that we
>> >> > eventually want to rethink as more people run this in the cloud.
>> >> > However, I
>> >> > would like to push back on the companies choosing to break the
>> >> > internet
>> >> > really hard before giving in. I am guessing that in most cases,
>> >> > WebSockets
>> >> > are broken in enterprise setting not because of some important
>> >> > deliberate
>> >> > choice, but rather because people don't understand it and are using
>> >> > default
>> >> > settings that disable them (I could be wrong though)....
>> >>
>> >> In my experience, for some people that's exactly the problem.  For one
>> >> person I helped it turned out their specific problem was the Windows
>> >> install they had access to had websockets explicitly disabled --
>> >> perhaps that was an IT policy.  For other people the problem is
>> >> further along the stack (e.g., possibly involving caching).
>> >>
>> >> Anyway, this change so that "there is a single websocket connection
>> >> between the server and client. " makes implementing an alternative to
>> >> WebSocket as a fallback dramatically easier, which I greatly
>> >> appreciate.
>> >>
>> >> > Our situation is a bit different though than SMC because most of our
>> >> > current
>> >> > users run the notebook on their own computers. This may be something
>> >> > that we
>> >>
>> >> Just curious -- how do you know that most people using IPython aren't
>> >> already using it through SMC?
>> >>
>> >> William
>> >>
>> >> >
>> >> > Cheers,
>> >> >
>> >> > Brian
>> >> >
>> >> >
>> >> >
>> >> > On Thu, Jan 8, 2015 at 10:10 AM, William Stein <wstein at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On Thu, Jan 8, 2015 at 10:04 AM, MinRK <benjaminrk at gmail.com> wrote:
>> >> >> > Alternative notebook frontend authors:
>> >> >> >
>> >> >> > There is an upcoming change to websocket connections to the
>> >> >> > notebook
>> >> >> > server.
>> >> >> > Instead of three websockets per notebook, one for each zmq
>> >> >> > channel,
>> >> >> > there is
>> >> >> > a single websocket connection between the server and client. The
>> >> >> > channel
>> >> >> > associated with each message is identified by a channel key in the
>> >> >> > message
>> >> >> > dict itself.
>> >> >> >
>> >> >> > Relevant changes:
>> >> >> >
>> >> >> > url is /channels instead of /shell, etc.
>> >> >> > when sending a message, add a channel key with the channel name
>> >> >> > ('shell',
>> >> >> > 'stdin')
>> >> >> > when receiving a message, check the channel key for routing
>> >> >> >
>> >> >> > -MinRK
>> >> >>
>> >> >> Thanks.
>> >> >>
>> >> >> Of related interest, what the situation involving providing a
>> >> >> fallback
>> >> >> to websockets?
>> >> >>
>> >> >> I tried to only support websockets connections with SageMathCloud
>> >> >> for
>> >> >> a few weeks, and immediately ran into many complaints from users who
>> >> >> couldn't connect as a result.   This is even with https and so on --
>> >> >> it's just a sad fact that in some corporate or otherwise constrained
>> >> >> environments that websockets don't work.   By far the best approach
>> >> >> I
>> >> >> found was to use Primus + Engine.io, which uses websockets if
>> >> >> possible, but will fallback to polling.   So this is what SMC does
>> >> >> now, and everything works, even for people that don't have
>> >> >> websockets
>> >> >> as an option... except for IPython-in-SMC of course, which doesn't
>> >> >> work.
>> >> >>
>> >> >> Another unrelated issue is that I'm now hardcoding making this
>> >> >> change
>> >> >> around line 171 of IPython/html/notebookapp.py, since I want to
>> >> >> serve
>> >> >> the purely static html of IPython from a completely different web
>> >> >> server:
>> >> >>
>> >> >>             #static_url_prefix = url_path_join(base_url,'/static/'),
>> >> >>             static_url_prefix = '/static/ipython/',
>> >> >>
>> >> >> (Sorry for derailing the thread.)
>> >> >>
>> >> >>  -- William
>> >> >>
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > IPython-dev mailing list
>> >> >> > IPython-dev at scipy.org
>> >> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> William Stein
>> >> >> Professor of Mathematics
>> >> >> University of Washington
>> >> >> http://wstein.org
>> >> >> _______________________________________________
>> >> >> IPython-dev mailing list
>> >> >> IPython-dev at scipy.org
>> >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Brian E. Granger
>> >> > Cal Poly State University, San Luis Obispo
>> >> > @ellisonbg on Twitter and GitHub
>> >> > bgranger at calpoly.edu and ellisonbg at gmail.com
>> >> >
>> >> > _______________________________________________
>> >> > IPython-dev mailing list
>> >> > IPython-dev at scipy.org
>> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> William Stein
>> >> Professor of Mathematics
>> >> University of Washington
>> >> http://wstein.org
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From liam.gretton at leicester.ac.uk  Thu Jan 15 04:28:03 2015
From: liam.gretton at leicester.ac.uk (Liam Gretton)
Date: Thu, 15 Jan 2015 09:28:03 +0000
Subject: [IPython-dev] %load hangs in qtconsole
Message-ID: <54B78823.1050506@leicester.ac.uk>

I've just built Python 2.7.9, iPython 2.3.1, PyQT 4.11.3 (against Qt
4.8.6) with matplotlib 1.4.2 etc on a Scientific Linux 6.4 system (64 bit).

What I'm finding is that any attempt to %load a Python script from
within the qtconsole hangs after displaying the final line of the
script. The script neither executes nor returns the command prompt.

>From within the command line ipython console there's no such problem.
Manually typing or pasting in contents to the qtconsole works and the
script executes as expected, it just hangs if %load is used.

Any ideas what might be causing this?

-- 
Liam Gretton                                    liam.gretton at le.ac.uk
Systems Specialist                           http://www.le.ac.uk/its/
IT Services                                   Tel: +44 (0)116 2522254
University Of Leicester, University Road
Leicestershire LE1 7RH, United Kingdom


From benjaminrk at gmail.com  Thu Jan 15 13:13:45 2015
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 15 Jan 2015 10:13:45 -0800
Subject: [IPython-dev] %load hangs in qtconsole
In-Reply-To: <54B78823.1050506@leicester.ac.uk>
References: <54B78823.1050506@leicester.ac.uk>
Message-ID: <CAHNn8BWYMdROY65PSR94XFAM5OmOYgBHqzxGRJktpNhO2MdQmw@mail.gmail.com>

It looks like it's putting the cursor in the wrong place, confusing the UI.
If you place the cursor in the input area, it should allow you to execute
the input.

-MinRK

On Thu, Jan 15, 2015 at 1:28 AM, Liam Gretton <liam.gretton at leicester.ac.uk>
wrote:

> I've just built Python 2.7.9, iPython 2.3.1, PyQT 4.11.3 (against Qt
> 4.8.6) with matplotlib 1.4.2 etc on a Scientific Linux 6.4 system (64 bit).
>
> What I'm finding is that any attempt to %load a Python script from
> within the qtconsole hangs after displaying the final line of the
> script. The script neither executes nor returns the command prompt.
>
> >From within the command line ipython console there's no such problem.
> Manually typing or pasting in contents to the qtconsole works and the
> script executes as expected, it just hangs if %load is used.
>
> Any ideas what might be causing this?
>
> --
> Liam Gretton                                    liam.gretton at le.ac.uk
> Systems Specialist                           http://www.le.ac.uk/its/
> IT Services                                   Tel: +44 (0)116 2522254
> University Of Leicester, University Road
> Leicestershire LE1 7RH, United Kingdom
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150115/35908ca9/attachment.html>

From soumith at fb.com  Thu Jan 15 22:26:58 2015
From: soumith at fb.com (Soumith Chintala)
Date: Fri, 16 Jan 2015 03:26:58 +0000
Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux
Message-ID: <D0DDC4FF.3884%soumith@fb.com>

Hey everyone,

As some of you might be aware, we released an ipython kernel for Lua (works only with LuaJIT because we use FFI for non-blocking reads). We also integrated visualization of images/video/audio and plotting using Bokeh.
https://github.com/facebook/iTorch

We use a two-process kernel, one of them handles stdout from the other process and sends it over to ipython. To pipe the processes, we use mkfifo, which is a posix utility.
http://pubs.opengroup.org/onlinepubs/009695399/functions/mkfifo.html

One problem we oncovered is that on certain (or possibly all linux distros) ipython thinks our kernel died, even though it is alive and well.
This only happen if we use mkfifo to redirect the output stream from one process to another. If we use a standard file like with this commit:
https://github.com/soumith/iTorch/commit/96e8a26a91a90d47583761a55b308364d391585c
we do not see any issues.

I am trying to understand Ipython semantics for thinking that a kernel died, as the heartbeat is still ticking.
I initially thought a SIGPIPE might be triggering Ipython to think that the kernel died, but I am not sure of that. I tried to set a trap here: https://github.com/soumith/iTorch/blob/master/itorch#L7 for SIGPIPE so that before the launch of ipython, SIGPIPE is trapped completely. However that did not help the ?kernel died? message.

The "kernel died? message appears when the first execute_request is processed, not before that.

Do let me know if you have any pointers.

Thanks,
Soumith


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/ca6d322d/attachment.html>

From liam.gretton at leicester.ac.uk  Fri Jan 16 05:34:53 2015
From: liam.gretton at leicester.ac.uk (Liam Gretton)
Date: Fri, 16 Jan 2015 10:34:53 +0000
Subject: [IPython-dev] %load hangs in qtconsole
In-Reply-To: <CAHNn8BWYMdROY65PSR94XFAM5OmOYgBHqzxGRJktpNhO2MdQmw@mail.gmail.com>
References: <54B78823.1050506@leicester.ac.uk>
	<CAHNn8BWYMdROY65PSR94XFAM5OmOYgBHqzxGRJktpNhO2MdQmw@mail.gmail.com>
Message-ID: <54B8E94D.8080706@leicester.ac.uk>

On 15/01/2015 18:13, MinRK wrote:
> It looks like it's putting the cursor in the wrong place, confusing the UI.
> If you place the cursor in the input area, it should allow you to execute
> the input.

Yes! That is indeed the problem, thanks.

I'll report this in GitHub and I'll take a look at the code myself to
see if I can fix it.

-- 
Liam Gretton                                    liam.gretton at le.ac.uk
Systems Specialist                           http://www.le.ac.uk/its/
IT Services                                   Tel: +44 (0)116 2522254
University Of Leicester, University Road
Leicestershire LE1 7RH, United Kingdom


From takowl at gmail.com  Fri Jan 16 13:16:48 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 16 Jan 2015 10:16:48 -0800
Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux
In-Reply-To: <D0DDC4FF.3884%soumith@fb.com>
References: <D0DDC4FF.3884%soumith@fb.com>
Message-ID: <CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com>

On 15 January 2015 at 19:26, Soumith Chintala <soumith at fb.com> wrote:

> One problem we oncovered is that on certain (or possibly all linux
> distros) ipython thinks our kernel died, even though it is alive and well.


FIFOs have some bizarre properties which might be related - for instance,
opening a FIFO for reading will block until something else opens it for
writing (the open() call itself blocks, not just attempts to read). Could
it be that an unexpectedly blocking system call is stopping the heartbeat
and causing IPython to think that your kernel is dead?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/9ac086cb/attachment.html>

From soumith at fb.com  Fri Jan 16 14:05:44 2015
From: soumith at fb.com (Soumith Chintala)
Date: Fri, 16 Jan 2015 19:05:44 +0000
Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux
In-Reply-To: <CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com>
References: <D0DDC4FF.3884%soumith@fb.com>
	<CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com>
Message-ID: <D0DEA0AE.3CAE%soumith@fb.com>

This is not a problem of blocking reads, or the heartbeat stopping, I have verified both those parts.
But I?ll try investigating more into if it?s a system-call/signaling weirdness.
Thanks,
Soumith

From: Thomas Kluyver <takowl at gmail.com<mailto:takowl at gmail.com>>
Reply-To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>>
Date: Friday, January 16, 2015 at 10:16 AM
To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>>
Subject: Re: [IPython-dev] iTorch - Weird issues with mkfifo on linux

On 15 January 2015 at 19:26, Soumith Chintala <soumith at fb.com<mailto:soumith at fb.com>> wrote:
One problem we oncovered is that on certain (or possibly all linux distros) ipython thinks our kernel died, even though it is alive and well.

FIFOs have some bizarre properties which might be related - for instance, opening a FIFO for reading will block until something else opens it for writing (the open() call itself blocks, not just attempts to read). Could it be that an unexpectedly blocking system call is stopping the heartbeat and causing IPython to think that your kernel is dead?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/37b89aa3/attachment.html>

From benjaminrk at gmail.com  Fri Jan 16 23:44:38 2015
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 16 Jan 2015 20:44:38 -0800
Subject: [IPython-dev] iTorch - Weird issues with mkfifo on linux
In-Reply-To: <D0DEA0AE.3CAE%soumith@fb.com>
References: <D0DDC4FF.3884%soumith@fb.com>
	<CAOvn4qiqGhQ1bTiBUKbSs29gv-0G_FHGYtxtemiPY02PVrty2Q@mail.gmail.com>
	<D0DEA0AE.3CAE%soumith@fb.com>
Message-ID: <CAHNn8BVaNsRe2=W6NL09SOpsb2yj-d40c8HUo-4BfwR75d8Jkw@mail.gmail.com>

Can you run the server with `--debug`, and post the logs around when the
failure occurs?

On Fri, Jan 16, 2015 at 11:05 AM, Soumith Chintala <soumith at fb.com> wrote:

>  This is not a problem of blocking reads, or the heartbeat stopping, I
> have verified both those parts.
> But I?ll try investigating more into if it?s a system-call/signaling
> weirdness.
> Thanks,
> Soumith
>
>   From: Thomas Kluyver <takowl at gmail.com>
> Reply-To: IPython developers list <ipython-dev at scipy.org>
> Date: Friday, January 16, 2015 at 10:16 AM
> To: IPython developers list <ipython-dev at scipy.org>
> Subject: Re: [IPython-dev] iTorch - Weird issues with mkfifo on linux
>
>    On 15 January 2015 at 19:26, Soumith Chintala <soumith at fb.com> wrote:
>
>> One problem we oncovered is that on certain (or possibly all linux
>> distros) ipython thinks our kernel died, even though it is alive and well.
>
>
>  FIFOs have some bizarre properties which might be related - for
> instance, opening a FIFO for reading will block until something else opens
> it for writing (the open() call itself blocks, not just attempts to read).
> Could it be that an unexpectedly blocking system call is stopping the
> heartbeat and causing IPython to think that your kernel is dead?
>
>  Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150116/8fd48d15/attachment.html>

From rnelsonchem at gmail.com  Sat Jan 17 14:14:12 2015
From: rnelsonchem at gmail.com (Ryan Nelson)
Date: Sat, 17 Jan 2015 14:14:12 -0500
Subject: [IPython-dev] Run IPython Parallel from custom module?
Message-ID: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com>

Hello everyone,

I'm developing a module for processing some data files. There are several
steps to this process, which can be time consuming, so I would like to
write a helper function in my module that uses IPython.parallel to spread
out the work over several processors. Unfortunately, I'm having trouble
getting things to work properly. Note: I can get something similar to work
when it is not written as a module...

I tried my best to make a simple example that fails with the same error
that I'm seeing in the larger case (below); however, it is still a little
complicated.

After running the test script, I get a "NameError" on the cluster nodes
(traceback below). If I check the nodes manually, the objects are defined.
I can also define new functions from the IPython terminal that act on those
objects, and those work as expected.

Any help you can provide is most appreciated. Sorry if this is a duplicate,
but I couldn't find it anywhere else.

Python 3.4 and 2.7 and IPython 2.3.1

Ryan

________________________________________

Here's the approximate structure of my module:

mymod/
    __init__.py
    functs.py : helper functions with Parallel support
    objects.py : contains my custom processing objects

# functs.py #
-------------
from IPython.parallel import Client

import mymod.objects as objects

def process(files):
    client = Client()
    dview = client[:]
    dview.block = True

    # Do this to make module path known
    # However, my mod will eventually be installed via pip, so this
    # isn't necessary to reproduce error in that case
    with dview.sync_imports():
        import sys
    sys.path.append(['/home/nelson/code/testing/',])

    dview['sys.path'] = sys.path
    File = objects.File1
    dview['File'] = File
    result = dview.map_sync(_do_this, files)
    return result

def _do_this(fname):
    f = File(fname)
    f.extra = 'hello'
    return f
-------------

# objects.py #
-------------
class File1(object):
    def __init__(self, fname):
        self.fname = fname
-------------

I also have a test script ("test.py") with the following:
-------------
from mymod.functs import process
result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
[print(i.fname, i.extra) for i in result]
-------------

When I run this script, I get the following errors:

importing sys on engine(s)
Traceback (most recent call last):
  File "test2.py", line 3, in <module>
    result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
  File "/home/nelson/code/testing/mymod/functs.py", line 16, in process
    result = dview.map_sync(_do_this, files)
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
366, in map_sync
    return self.map(f,*sequences,**kwargs)
  File "<string>", line 2, in map
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
66, in sync_results
    ret = f(self, *args, **kwargs)
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
624, in map
    return pf.map(*sequences)
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
line 271, in map
    ret = self(*sequences)
  File "<string>", line 2, in __call__
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
line 78, in sync_view_results
    return f(self, *args, **kwargs)
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
line 254, in __call__
    return r.get()
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
line 118, in get
    raise self._exception
  File
"/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
line 153, in wait
    results = error.collect_exceptions(results, self._fname)
  File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", line
233, in collect_exceptions
    raise e
  File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py", line
231, in collect_exceptions
    raise CompositeError(msg, elist)
IPython.parallel.error.CompositeError: one or more exceptions from call to
method: _do_this
[0:apply]: NameError: name 'File' is not defined
[1:apply]: NameError: name 'File' is not defined
[2:apply]: NameError: name 'File' is not defined
[3:apply]: NameError: name 'File' is not defined

However, if I jump into an interactive Python terminal, these objects are
defined on the cluster nodes.
In [1]: from IPython.parallel import Client

In [2]: client = Client()

In [3]: dview = client[:]

In [4]: dview['File']


Out[4]:
[mymod.objects.File1,
 mymod.objects.File1,
 mymod.objects.File1,
 mymod.objects.File1]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/e3dc9b1a/attachment.html>

From benjaminrk at gmail.com  Sat Jan 17 15:15:47 2015
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 17 Jan 2015 12:15:47 -0800
Subject: [IPython-dev] Run IPython Parallel from custom module?
In-Reply-To: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com>
References: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com>
Message-ID: <CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com>

This is one of the most common sources of confusion with IPython.parallel.
When you call a function, it?s globals are resolved to its own module. So
when you call _do_this, it is looking for File in mymod, not in the
interactive namespace. If you had defined _do_this interactively (in
__main__), it would have found File. Similarly, if File were defined in the
same module as _do_this, it would also be found.

IPython provides a decorator that lets you define functions in modules that
will be treated as if they were defined in __main__, and will thus have
access to the interactive namespace.

from IPython.parallel import interactive
@interactivedef _do_this(fname):
    f = File(fname)
    ...

-MinRK
?

On Sat, Jan 17, 2015 at 11:14 AM, Ryan Nelson <rnelsonchem at gmail.com> wrote:

> Hello everyone,
>
> I'm developing a module for processing some data files. There are several
> steps to this process, which can be time consuming, so I would like to
> write a helper function in my module that uses IPython.parallel to spread
> out the work over several processors. Unfortunately, I'm having trouble
> getting things to work properly. Note: I can get something similar to work
> when it is not written as a module...
>
> I tried my best to make a simple example that fails with the same error
> that I'm seeing in the larger case (below); however, it is still a little
> complicated.
>
> After running the test script, I get a "NameError" on the cluster nodes
> (traceback below). If I check the nodes manually, the objects are defined.
> I can also define new functions from the IPython terminal that act on those
> objects, and those work as expected.
>
> Any help you can provide is most appreciated. Sorry if this is a
> duplicate, but I couldn't find it anywhere else.
>
> Python 3.4 and 2.7 and IPython 2.3.1
>
> Ryan
>
> ________________________________________
>
> Here's the approximate structure of my module:
>
> mymod/
>     __init__.py
>     functs.py : helper functions with Parallel support
>     objects.py : contains my custom processing objects
>
> # functs.py #
> -------------
> from IPython.parallel import Client
>
> import mymod.objects as objects
>
> def process(files):
>     client = Client()
>     dview = client[:]
>     dview.block = True
>
>     # Do this to make module path known
>     # However, my mod will eventually be installed via pip, so this
>     # isn't necessary to reproduce error in that case
>     with dview.sync_imports():
>         import sys
>     sys.path.append(['/home/nelson/code/testing/',])
>
>     dview['sys.path'] = sys.path
>     File = objects.File1
>     dview['File'] = File
>     result = dview.map_sync(_do_this, files)
>     return result
>
> def _do_this(fname):
>     f = File(fname)
>     f.extra = 'hello'
>     return f
> -------------
>
> # objects.py #
> -------------
> class File1(object):
>     def __init__(self, fname):
>         self.fname = fname
> -------------
>
> I also have a test script ("test.py") with the following:
> -------------
> from mymod.functs import process
> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
> [print(i.fname, i.extra) for i in result]
> -------------
>
> When I run this script, I get the following errors:
>
> importing sys on engine(s)
> Traceback (most recent call last):
>   File "test2.py", line 3, in <module>
>     result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
>   File "/home/nelson/code/testing/mymod/functs.py", line 16, in process
>     result = dview.map_sync(_do_this, files)
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
> 366, in map_sync
>     return self.map(f,*sequences,**kwargs)
>   File "<string>", line 2, in map
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
> 66, in sync_results
>     ret = f(self, *args, **kwargs)
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
> 624, in map
>     return pf.map(*sequences)
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
> line 271, in map
>     ret = self(*sequences)
>   File "<string>", line 2, in __call__
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
> line 78, in sync_view_results
>     return f(self, *args, **kwargs)
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
> line 254, in __call__
>     return r.get()
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
> line 118, in get
>     raise self._exception
>   File
> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
> line 153, in wait
>     results = error.collect_exceptions(results, self._fname)
>   File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py",
> line 233, in collect_exceptions
>     raise e
>   File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py",
> line 231, in collect_exceptions
>     raise CompositeError(msg, elist)
> IPython.parallel.error.CompositeError: one or more exceptions from call to
> method: _do_this
> [0:apply]: NameError: name 'File' is not defined
> [1:apply]: NameError: name 'File' is not defined
> [2:apply]: NameError: name 'File' is not defined
> [3:apply]: NameError: name 'File' is not defined
>
> However, if I jump into an interactive Python terminal, these objects are
> defined on the cluster nodes.
> In [1]: from IPython.parallel import Client
>
> In [2]: client = Client()
>
> In [3]: dview = client[:]
>
> In [4]: dview['File']
>
>
> Out[4]:
> [mymod.objects.File1,
>  mymod.objects.File1,
>  mymod.objects.File1,
>  mymod.objects.File1]
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/23cef824/attachment.html>

From rnelsonchem at gmail.com  Sat Jan 17 16:17:07 2015
From: rnelsonchem at gmail.com (Ryan Nelson)
Date: Sat, 17 Jan 2015 16:17:07 -0500
Subject: [IPython-dev] Run IPython Parallel from custom module?
In-Reply-To: <CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com>
References: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com>
	<CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com>
Message-ID: <CAFbt6dcu3u55JO6P2+GqOXS9gMr8rdj-hy4vOddDas56sZBfjA@mail.gmail.com>

Thanks!

Searching for that decorator brings up a bunch of search hits. However, it
looks like it isn't really documented in the main document yet (?).

I'll try this out.

Ryan



On Sat, Jan 17, 2015 at 3:15 PM, MinRK <benjaminrk at gmail.com> wrote:

> This is one of the most common sources of confusion with IPython.parallel.
> When you call a function, it?s globals are resolved to its own module. So
> when you call _do_this, it is looking for File in mymod, not in the
> interactive namespace. If you had defined _do_this interactively (in
> __main__), it would have found File. Similarly, if File were defined in
> the same module as _do_this, it would also be found.
>
> IPython provides a decorator that lets you define functions in modules
> that will be treated as if they were defined in __main__, and will thus
> have access to the interactive namespace.
>
> from IPython.parallel import interactive
> @interactivedef _do_this(fname):
>     f = File(fname)
>     ...
>
> -MinRK
> ?
>
> On Sat, Jan 17, 2015 at 11:14 AM, Ryan Nelson <rnelsonchem at gmail.com>
> wrote:
>
>> Hello everyone,
>>
>> I'm developing a module for processing some data files. There are several
>> steps to this process, which can be time consuming, so I would like to
>> write a helper function in my module that uses IPython.parallel to spread
>> out the work over several processors. Unfortunately, I'm having trouble
>> getting things to work properly. Note: I can get something similar to work
>> when it is not written as a module...
>>
>> I tried my best to make a simple example that fails with the same error
>> that I'm seeing in the larger case (below); however, it is still a little
>> complicated.
>>
>> After running the test script, I get a "NameError" on the cluster nodes
>> (traceback below). If I check the nodes manually, the objects are defined.
>> I can also define new functions from the IPython terminal that act on those
>> objects, and those work as expected.
>>
>> Any help you can provide is most appreciated. Sorry if this is a
>> duplicate, but I couldn't find it anywhere else.
>>
>> Python 3.4 and 2.7 and IPython 2.3.1
>>
>> Ryan
>>
>> ________________________________________
>>
>> Here's the approximate structure of my module:
>>
>> mymod/
>>     __init__.py
>>     functs.py : helper functions with Parallel support
>>     objects.py : contains my custom processing objects
>>
>> # functs.py #
>> -------------
>> from IPython.parallel import Client
>>
>> import mymod.objects as objects
>>
>> def process(files):
>>     client = Client()
>>     dview = client[:]
>>     dview.block = True
>>
>>     # Do this to make module path known
>>     # However, my mod will eventually be installed via pip, so this
>>     # isn't necessary to reproduce error in that case
>>     with dview.sync_imports():
>>         import sys
>>     sys.path.append(['/home/nelson/code/testing/',])
>>
>>     dview['sys.path'] = sys.path
>>     File = objects.File1
>>     dview['File'] = File
>>     result = dview.map_sync(_do_this, files)
>>     return result
>>
>> def _do_this(fname):
>>     f = File(fname)
>>     f.extra = 'hello'
>>     return f
>> -------------
>>
>> # objects.py #
>> -------------
>> class File1(object):
>>     def __init__(self, fname):
>>         self.fname = fname
>> -------------
>>
>> I also have a test script ("test.py") with the following:
>> -------------
>> from mymod.functs import process
>> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
>> [print(i.fname, i.extra) for i in result]
>> -------------
>>
>> When I run this script, I get the following errors:
>>
>> importing sys on engine(s)
>> Traceback (most recent call last):
>>   File "test2.py", line 3, in <module>
>>     result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
>>   File "/home/nelson/code/testing/mymod/functs.py", line 16, in process
>>     result = dview.map_sync(_do_this, files)
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
>> 366, in map_sync
>>     return self.map(f,*sequences,**kwargs)
>>   File "<string>", line 2, in map
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
>> 66, in sync_results
>>     ret = f(self, *args, **kwargs)
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
>> 624, in map
>>     return pf.map(*sequences)
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
>> line 271, in map
>>     ret = self(*sequences)
>>   File "<string>", line 2, in __call__
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
>> line 78, in sync_view_results
>>     return f(self, *args, **kwargs)
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
>> line 254, in __call__
>>     return r.get()
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
>> line 118, in get
>>     raise self._exception
>>   File
>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
>> line 153, in wait
>>     results = error.collect_exceptions(results, self._fname)
>>   File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py",
>> line 233, in collect_exceptions
>>     raise e
>>   File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py",
>> line 231, in collect_exceptions
>>     raise CompositeError(msg, elist)
>> IPython.parallel.error.CompositeError: one or more exceptions from call
>> to method: _do_this
>> [0:apply]: NameError: name 'File' is not defined
>> [1:apply]: NameError: name 'File' is not defined
>> [2:apply]: NameError: name 'File' is not defined
>> [3:apply]: NameError: name 'File' is not defined
>>
>> However, if I jump into an interactive Python terminal, these objects are
>> defined on the cluster nodes.
>> In [1]: from IPython.parallel import Client
>>
>> In [2]: client = Client()
>>
>> In [3]: dview = client[:]
>>
>> In [4]: dview['File']
>>
>>
>> Out[4]:
>> [mymod.objects.File1,
>>  mymod.objects.File1,
>>  mymod.objects.File1,
>>  mymod.objects.File1]
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/415828c1/attachment.html>

From rnelsonchem at gmail.com  Sat Jan 17 20:23:07 2015
From: rnelsonchem at gmail.com (Ryan Nelson)
Date: Sat, 17 Jan 2015 20:23:07 -0500
Subject: [IPython-dev] Run IPython Parallel from custom module?
In-Reply-To: <CAFbt6dcu3u55JO6P2+GqOXS9gMr8rdj-hy4vOddDas56sZBfjA@mail.gmail.com>
References: <CAFbt6dcnqqPvQ3aS4-sCLV9r_fS9HfRkxoN-z3LZsHjAMDF4AA@mail.gmail.com>
	<CAHNn8BUcN7KQ4bpnAqSUd75SF_6QtsXS2w4DrkcK88sPui4j7Q@mail.gmail.com>
	<CAFbt6dcu3u55JO6P2+GqOXS9gMr8rdj-hy4vOddDas56sZBfjA@mail.gmail.com>
Message-ID: <CAFbt6dfE-U-CbfNJZ66cpx1V=Bp6iTo0yjzvDbfDc7pGFaKiqg@mail.gmail.com>

This worked well for the example that I posted. Thanks again.

However, it brings up another question. What are the restrictions on using
this on methods inside a custom object? For example, if I change
"functs.py" into an object with the following methods:

    @interactive
    def _do_this(self, fname):
        f = File(fname)
        f.extra = 'hello'
        self._do_something_else(f)
        return f

    @interactive
    def _do_something_else(self, f):
        f.junk = 'World'

I don't get any errors, but "_do_something_else" doesn't seem to be called,
but "_do_this" works fine. (Actually, in my larger code, I'm getting a
pickling error... But that's tough to reproduce.)

Ryan

On Sat, Jan 17, 2015 at 4:17 PM, Ryan Nelson <rnelsonchem at gmail.com> wrote:

> Thanks!
>
> Searching for that decorator brings up a bunch of search hits. However, it
> looks like it isn't really documented in the main document yet (?).
>
> I'll try this out.
>
> Ryan
>
>
>
> On Sat, Jan 17, 2015 at 3:15 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>> This is one of the most common sources of confusion with
>> IPython.parallel. When you call a function, it?s globals are resolved to
>> its own module. So when you call _do_this, it is looking for File in
>> mymod, not in the interactive namespace. If you had defined _do_this
>> interactively (in __main__), it would have found File. Similarly, if File
>> were defined in the same module as _do_this, it would also be found.
>>
>> IPython provides a decorator that lets you define functions in modules
>> that will be treated as if they were defined in __main__, and will thus
>> have access to the interactive namespace.
>>
>> from IPython.parallel import interactive
>> @interactivedef _do_this(fname):
>>     f = File(fname)
>>     ...
>>
>> -MinRK
>> ?
>>
>> On Sat, Jan 17, 2015 at 11:14 AM, Ryan Nelson <rnelsonchem at gmail.com>
>> wrote:
>>
>>> Hello everyone,
>>>
>>> I'm developing a module for processing some data files. There are
>>> several steps to this process, which can be time consuming, so I would like
>>> to write a helper function in my module that uses IPython.parallel to
>>> spread out the work over several processors. Unfortunately, I'm having
>>> trouble getting things to work properly. Note: I can get something similar
>>> to work when it is not written as a module...
>>>
>>> I tried my best to make a simple example that fails with the same error
>>> that I'm seeing in the larger case (below); however, it is still a little
>>> complicated.
>>>
>>> After running the test script, I get a "NameError" on the cluster nodes
>>> (traceback below). If I check the nodes manually, the objects are defined.
>>> I can also define new functions from the IPython terminal that act on those
>>> objects, and those work as expected.
>>>
>>> Any help you can provide is most appreciated. Sorry if this is a
>>> duplicate, but I couldn't find it anywhere else.
>>>
>>> Python 3.4 and 2.7 and IPython 2.3.1
>>>
>>> Ryan
>>>
>>> ________________________________________
>>>
>>> Here's the approximate structure of my module:
>>>
>>> mymod/
>>>     __init__.py
>>>     functs.py : helper functions with Parallel support
>>>     objects.py : contains my custom processing objects
>>>
>>> # functs.py #
>>> -------------
>>> from IPython.parallel import Client
>>>
>>> import mymod.objects as objects
>>>
>>> def process(files):
>>>     client = Client()
>>>     dview = client[:]
>>>     dview.block = True
>>>
>>>     # Do this to make module path known
>>>     # However, my mod will eventually be installed via pip, so this
>>>     # isn't necessary to reproduce error in that case
>>>     with dview.sync_imports():
>>>         import sys
>>>     sys.path.append(['/home/nelson/code/testing/',])
>>>
>>>     dview['sys.path'] = sys.path
>>>     File = objects.File1
>>>     dview['File'] = File
>>>     result = dview.map_sync(_do_this, files)
>>>     return result
>>>
>>> def _do_this(fname):
>>>     f = File(fname)
>>>     f.extra = 'hello'
>>>     return f
>>> -------------
>>>
>>> # objects.py #
>>> -------------
>>> class File1(object):
>>>     def __init__(self, fname):
>>>         self.fname = fname
>>> -------------
>>>
>>> I also have a test script ("test.py") with the following:
>>> -------------
>>> from mymod.functs import process
>>> result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
>>> [print(i.fname, i.extra) for i in result]
>>> -------------
>>>
>>> When I run this script, I get the following errors:
>>>
>>> importing sys on engine(s)
>>> Traceback (most recent call last):
>>>   File "test2.py", line 3, in <module>
>>>     result = process(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
>>>   File "/home/nelson/code/testing/mymod/functs.py", line 16, in process
>>>     result = dview.map_sync(_do_this, files)
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
>>> 366, in map_sync
>>>     return self.map(f,*sequences,**kwargs)
>>>   File "<string>", line 2, in map
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
>>> 66, in sync_results
>>>     ret = f(self, *args, **kwargs)
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/view.py", line
>>> 624, in map
>>>     return pf.map(*sequences)
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
>>> line 271, in map
>>>     ret = self(*sequences)
>>>   File "<string>", line 2, in __call__
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
>>> line 78, in sync_view_results
>>>     return f(self, *args, **kwargs)
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/remotefunction.py",
>>> line 254, in __call__
>>>     return r.get()
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
>>> line 118, in get
>>>     raise self._exception
>>>   File
>>> "/usr/lib64/python3.4/site-packages/IPython/parallel/client/asyncresult.py",
>>> line 153, in wait
>>>     results = error.collect_exceptions(results, self._fname)
>>>   File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py",
>>> line 233, in collect_exceptions
>>>     raise e
>>>   File "/usr/lib64/python3.4/site-packages/IPython/parallel/error.py",
>>> line 231, in collect_exceptions
>>>     raise CompositeError(msg, elist)
>>> IPython.parallel.error.CompositeError: one or more exceptions from call
>>> to method: _do_this
>>> [0:apply]: NameError: name 'File' is not defined
>>> [1:apply]: NameError: name 'File' is not defined
>>> [2:apply]: NameError: name 'File' is not defined
>>> [3:apply]: NameError: name 'File' is not defined
>>>
>>> However, if I jump into an interactive Python terminal, these objects
>>> are defined on the cluster nodes.
>>> In [1]: from IPython.parallel import Client
>>>
>>> In [2]: client = Client()
>>>
>>> In [3]: dview = client[:]
>>>
>>> In [4]: dview['File']
>>>
>>>
>>> Out[4]:
>>> [mymod.objects.File1,
>>>  mymod.objects.File1,
>>>  mymod.objects.File1,
>>>  mymod.objects.File1]
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150117/7f178703/attachment.html>

From lev at columbia.edu  Sun Jan 18 08:28:55 2015
From: lev at columbia.edu (Lev Givon)
Date: Sun, 18 Jan 2015 08:28:55 -0500
Subject: [IPython-dev] question about KernelClient.is_alive()
Message-ID: <20150118132855.GA5685@avicenna>

I recently tried using IPython.kernel.KernelClient (in IPython 2.3.1 on Ubuntu
14.04.1) to connect to a locally running kernel (started from the command line
by running "ipython kernel") by passing it the connection file associated with
the latter. While doing so, I noticed that KernelClient.is_alive() returns True
even when KernelClient is instantiated with a nonexistent connection file, i.e.,

import IPython.kernel
kc = IPython.kernel.KernelClient(connection_file='/tmp/nonexistent_file')
print kc.is_alive()

Is this expected? The online documentation seems to imply that is_alive() should
return True only if the kernel to which KernelClient is connected is running.
-- 
Lev Givon
Bionet Group | Neurokernel Project
http://www.columbia.edu/~lev/
http://lebedov.github.io/
http://neurokernel.github.io/



From damianavila at gmail.com  Sun Jan 18 09:19:44 2015
From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=)
Date: Sun, 18 Jan 2015 11:19:44 -0300
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
 core Jupyter/IPython team
In-Reply-To: <CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
	<1420394484530-5082155.post@n6.nabble.com>
	<CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>
Message-ID: <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com>

(Belatedly) Congrats!
You both make an amazing work every day!

Cheers.

Damian

2015-01-05 19:01 GMT-03:00 Brian Granger <ellisonbg at gmail.com>:

> Thanks so much for your work Kyle and Nick!
>
> Cheers,
>
> Brian
>
> On Sun, Jan 4, 2015 at 10:01 AM, ssanderson <ssanderson at quantopian.com>
> wrote:
>
>> Congrats!
>>
>>
>>
>> --
>> View this message in context:
>> http://python.6.x6.nabble.com/Welcoming-Kyle-Kelley-and-Nick-Bollweg-to-the-core-Jupyter-IPython-team-tp5082131p5082155.html
>> Sent from the IPython - Development mailing list archive at Nabble.com.
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
*Dami?n*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/6f92c07d/attachment.html>

From lev at columbia.edu  Sun Jan 18 11:43:19 2015
From: lev at columbia.edu (Lev Givon)
Date: Sun, 18 Jan 2015 11:43:19 -0500
Subject: [IPython-dev] question about KernelClient.is_alive()
In-Reply-To: <20150118132855.GA5685@avicenna>
References: <20150118132855.GA5685@avicenna>
Message-ID: <20150118164319.GB7084@avicenna.ee.columbia.edu>

Received from Lev Givon on Sun, Jan 18, 2015 at 08:28:55AM EST:
> I recently tried using IPython.kernel.KernelClient (in IPython 2.3.1 on Ubuntu
> 14.04.1) to connect to a locally running kernel (started from the command line
> by running "ipython kernel") by passing it the connection file associated with
> the latter. While doing so, I noticed that KernelClient.is_alive() returns True
> even when KernelClient is instantiated with a nonexistent connection file, i.e.,
> 
> import IPython.kernel
> kc = IPython.kernel.KernelClient(connection_file='/tmp/nonexistent_file')
> print kc.is_alive()
> 
> Is this expected? The online documentation seems to imply that is_alive() should
> return True only if the kernel to which KernelClient is connected is running.

(Answering my own question for the benefit of others.)

The value of KernelClient.is_alive() evidently isn't indicative of actual kernel
status until the heartbeat channel is activated and unpaused to determine
whether the client is interacting with a live kernel.
-- 
Lev Givon
Bionet Group | Neurokernel Project
http://www.columbia.edu/~lev/
http://lebedov.github.io/
http://neurokernel.github.io/



From nick.bollweg at gmail.com  Sun Jan 18 13:30:40 2015
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Sun, 18 Jan 2015 13:30:40 -0500
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
 core Jupyter/IPython team
In-Reply-To: <CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
	<1420394484530-5082155.post@n6.nabble.com>
	<CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>
	<CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com>
Message-ID: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>

Thanks everybody!

Just as an update: we're back from the hospital, and our new little person
is doing great! We're all a bit sleepy, so take anything posted at strange
hours of the day with a grain of salt :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/45b3b005/attachment.html>

From ellisonbg at gmail.com  Sun Jan 18 13:45:09 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 18 Jan 2015 10:45:09 -0800
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
 core Jupyter/IPython team
In-Reply-To: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
	<1420394484530-5082155.post@n6.nabble.com>
	<CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>
	<CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com>
	<CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>
Message-ID: <CAH4pYpTu89Lqwz7oBBB9TBSxFAx+xzmBaA2xCcLcbMoCketd5A@mail.gmail.com>

Congrats!

On Sun, Jan 18, 2015 at 10:30 AM, Nicholas Bollweg <nick.bollweg at gmail.com>
wrote:

> Thanks everybody!
>
> Just as an update: we're back from the hospital, and our new little person
> is doing great! We're all a bit sleepy, so take anything posted at strange
> hours of the day with a grain of salt :)
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/f20594c8/attachment.html>

From rgbkrk at gmail.com  Sun Jan 18 14:31:07 2015
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Sun, 18 Jan 2015 13:31:07 -0600
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
 core Jupyter/IPython team
In-Reply-To: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
	<1420394484530-5082155.post@n6.nabble.com>
	<CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>
	<CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com>
	<CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>
Message-ID: <CA+tbMaVkdfXs3oHS6xWip22CM2c3jmNaT+Qt4OOxEACyULCBaA@mail.gmail.com>

Congratulations!

We have little ones too, so we understand. The team and community seem to
put up with my sleep deprived contributions. ;) Rest well and enjoy the new
addition. They're a lot of fun!

-- Kyle

On Sunday, January 18, 2015, Nicholas Bollweg <nick.bollweg at gmail.com>
wrote:

> Thanks everybody!
>
> Just as an update: we're back from the hospital, and our new little person
> is doing great! We're all a bit sleepy, so take anything posted at strange
> hours of the day with a grain of salt :)
>


-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150118/00c52a09/attachment.html>

From DavidAnthonyPowell+python at gmail.com  Mon Jan 19 01:47:21 2015
From: DavidAnthonyPowell+python at gmail.com (David Powell)
Date: Mon, 19 Jan 2015 17:47:21 +1100
Subject: [IPython-dev] WebGL example in IPython notebook based on three.js
Message-ID: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>

Hello all,

I recently managed to get 3D vector plots working in the IPython notebook,
by statically producing three.js output. I thought I'd share it here in
case this example is useful for anyone else, or if anyone has any
suggestions as to how I could do things better.

The advantage of the static approach is that the output survives the
nbconvert process, which you can see in this notebook here (which also
compares with a couple of other unsatisfactory ways of visualising 3D
vector plots).

http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb

Another example can be seen at:

http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb

If you are interested to see how this works, you can check out the python
source:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py

As well as the javascript:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js

and finally the HTML:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html

By the way, I am aware of the efforts to make a proper three.js IPython
widget (https://github.com/jasongrout/pythreejs/), and will probably switch
over to this once it is stable, and when the output can survive ipython's
nbconvert process. Is this something we can expect for widgets in IPython
3.0?

regards
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/d65e9f23/attachment.html>

From soumith at fb.com  Mon Jan 19 01:50:30 2015
From: soumith at fb.com (Soumith Chintala)
Date: Mon, 19 Jan 2015 06:50:30 +0000
Subject: [IPython-dev] WebGL example in IPython notebook based on
 three.js
In-Reply-To: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
Message-ID: <D0E1E8D1.3E2F%soumith@fb.com>

This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble.
I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch.
?
Soumith

From: David Powell <DavidAnthonyPowell+python at gmail.com<mailto:DavidAnthonyPowell+python at gmail.com>>
Reply-To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>>
Date: Sunday, January 18, 2015 at 10:47 PM
To: IPython developers list <ipython-dev at scipy.org<mailto:ipython-dev at scipy.org>>
Subject: [IPython-dev] WebGL example in IPython notebook based on three.js

Hello all,

I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better.

The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots).

http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb<https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%2520to%2520create%25203D%2520plots.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=aad5ead78a1b3391d5b9c2a10e1214ef17fd7df8d223ad6076c6b78f9ff1246c>

Another example can be seen at:

http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb<https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%2520and%2520creating%2520geometric%2520shapes.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=76948f7a6b29f01f0eccbe0899c7d91d427a1c1588d11ecbc4e568f992fddea7>

If you are interested to see how this works, you can check out the python source:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py

As well as the javascript:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js

and finally the HTML:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html

By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0?

regards
David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/1a95b887/attachment.html>

From bussonniermatthias at gmail.com  Mon Jan 19 06:08:52 2015
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 19 Jan 2015 12:08:52 +0100
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <D0E1E8D1.3E2F%soumith@fb.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<D0E1E8D1.3E2F%soumith@fb.com>
Message-ID: <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com>

Hi all, 
Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit :

> This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble.
> I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch.

We still haven't really figured it out how to share javascript in between packages of different languages. 
(well technically in 3.0 yo can install into $NBEXTENSION directory, but install process is not perfect) 
If you can figured that out, and that all kernels agree on a set of relatively common JS "plugins"  that
are often used we might be able to work on making  theses plugin available on nbviewer.

Long term planning of course, but I'd like people to keep that in mind. 

-- 
M

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/129dbaf2/attachment.html>

From bussonniermatthias at gmail.com  Mon Jan 19 06:19:02 2015
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 19 Jan 2015 12:19:02 +0100
Subject: [IPython-dev] Welcoming Kyle Kelley and Nick Bollweg to the
	core Jupyter/IPython team
In-Reply-To: <CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>
References: <CAHAreOqLEs4P+kwUGC532SvdNrg=mbOBqzo_GH0xr-P75=+pXw@mail.gmail.com>
	<1420394484530-5082155.post@n6.nabble.com>
	<CAH4pYpTLSQwcNycjT5BNebDgtP9hVGpQjEe0JZHvDAfJiBp2uQ@mail.gmail.com>
	<CAH+mRR1DE6s=B4tACY0BtdkuhNO-r=1y+oRp+fi-TCyp3g_Zkw@mail.gmail.com>
	<CACejjWwTijEAFE_eqX9VUDiV-84RH6SDYH9aSgagdrDTnoiiPQ@mail.gmail.com>
Message-ID: <9202D5B9-022F-4A95-B9D2-427E5B2A84A0@gmail.com>

Congratulation too, 

Looking forward to your nightly contributions ! 

Le 18 janv. 2015 ? 19:30, Nicholas Bollweg a ?crit :

> Thanks everybody!
> 
> Just as an update: we're back from the hospital, and our new little person is doing great! We're all a bit sleepy, so take anything posted at strange hours of the day with a grain of salt :)
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From zvoros at gmail.com  Mon Jan 19 06:26:00 2015
From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=)
Date: Mon, 19 Jan 2015 12:26:00 +0100
Subject: [IPython-dev] status of nbconvert
Message-ID: <54BCE9C8.6080304@gmail.com>

Hi all,

I have recently had problems with nbconvert, and I was wondering, 
whether I do something improper, or the code is really broken. I have 
pulled the latest from master, and nbconvert fails outright with a 
simple notebook. (This eventually leads to a 500 server error, whenever 
I try to download a notebook.)

Here is my traceback:

ipython nbconvert test.ipynb
Traceback (most recent call last):
   File "/usr/local/bin/ipython", line 5, in <module>
     start_ipython()
   File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", 
line 120, in start_ipython
     return launch_new_instance(argv=argv, **kwargs)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 573, in launch_instance
     app.initialize(argv)
   File "<string>", line 2, in initialize
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 75, in catch_config_error
     return method(app, *args, **kwargs)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 
321, in initialize
     super(TerminalIPythonApp, self).initialize(argv)
   File "<string>", line 2, in initialize
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 75, in catch_config_error
     return method(app, *args, **kwargs)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/core/application.py", 
line 369, in initialize
     self.parse_command_line(argv)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 
316, in parse_command_line
     return super(TerminalIPythonApp, self).parse_command_line(argv)
   File "<string>", line 2, in parse_command_line
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 75, in catch_config_error
     return method(app, *args, **kwargs)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 471, in parse_command_line
     return self.initialize_subcommand(subc, subargv)
   File "<string>", line 2, in initialize_subcommand
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 75, in catch_config_error
     return method(app, *args, **kwargs)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
line 402, in initialize_subcommand
     subapp = import_item(subapp)
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/utils/importstring.py", 
line 42, in import_item
     module = __import__(package, fromlist=[obj])
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/__init__.py", 
line 3, in <module>
     from .exporters import *
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/__init__.py", 
line 1, in <module>
     from .export import *
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/export.py", 
line 13, in <module>
     from .templateexporter import TemplateExporter
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/templateexporter.py", 
line 30, in <module>
     from IPython.nbconvert import filters
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/__init__.py", 
line 6, in <module>
     from .markdown import *
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", 
line 79, in <module>
     class MathBlockLexer(mistune.BlockLexer):
   File 
"/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", 
line 80, in MathBlockLexer
     default_rules = ['block_math', 'latex_environment'] + 
mistune.BlockLexer.default_rules
AttributeError: type object 'BlockLexer' has no attribute 'default_rules'



If, in markdown.py, I remove

+ mistune.BlockLexer.default_rules on line 80, and
+ mistune.InlineLexer.default_rules on line 108, then everything seems 
to work fine.

Could it be that the mistune version that I have (0.4) doesn't have the 
default_rules property, but the version with which ipython was tested on 
travis has? If so, shouldn't nbconvert bail out when the version 
requirement is not fulfilled, or shouldn't simply the default_rules 
property be defined, when the class is inherited in markdown.py? If this 
is the case, I could file a fix on github.

Somewhat related: if there is a server error, wouldn't it be more 
meaningful to return the traceback, instead of "500: internal server error"?

Cheers,
Zolt?n




From bussonniermatthias at gmail.com  Mon Jan 19 06:31:17 2015
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 19 Jan 2015 12:31:17 +0100
Subject: [IPython-dev] status of nbconvert
In-Reply-To: <54BCE9C8.6080304@gmail.com>
References: <54BCE9C8.6080304@gmail.com>
Message-ID: <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com>



There is definitively a mistune api change, 
we check dependency in setup.py :

https://github.com/ipython/ipython/blob/50799e6943174cbd24998c4fb018eff42aac5cfb/setup.py#L257

'mistune>=0.5'

But we definitively don't check at runtime. 
So the bug should be encounters only by people having installed
master some time ago and just pulling new versions.

Normal install should not suffer of this. 

You are right about the 500 though.

-- 
M



Le 19 janv. 2015 ? 12:26, Zolt?n V?r?s a ?crit :

> Hi all,
> 
> I have recently had problems with nbconvert, and I was wondering, 
> whether I do something improper, or the code is really broken. I have 
> pulled the latest from master, and nbconvert fails outright with a 
> simple notebook. (This eventually leads to a 500 server error, whenever 
> I try to download a notebook.)
> 
> Here is my traceback:
> 
> ipython nbconvert test.ipynb
> Traceback (most recent call last):
>   File "/usr/local/bin/ipython", line 5, in <module>
>     start_ipython()
>   File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", 
> line 120, in start_ipython
>     return launch_new_instance(argv=argv, **kwargs)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 573, in launch_instance
>     app.initialize(argv)
>   File "<string>", line 2, in initialize
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 75, in catch_config_error
>     return method(app, *args, **kwargs)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 
> 321, in initialize
>     super(TerminalIPythonApp, self).initialize(argv)
>   File "<string>", line 2, in initialize
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 75, in catch_config_error
>     return method(app, *args, **kwargs)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/core/application.py", 
> line 369, in initialize
>     self.parse_command_line(argv)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 
> 316, in parse_command_line
>     return super(TerminalIPythonApp, self).parse_command_line(argv)
>   File "<string>", line 2, in parse_command_line
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 75, in catch_config_error
>     return method(app, *args, **kwargs)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 471, in parse_command_line
>     return self.initialize_subcommand(subc, subargv)
>   File "<string>", line 2, in initialize_subcommand
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 75, in catch_config_error
>     return method(app, *args, **kwargs)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", 
> line 402, in initialize_subcommand
>     subapp = import_item(subapp)
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/utils/importstring.py", 
> line 42, in import_item
>     module = __import__(package, fromlist=[obj])
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/__init__.py", 
> line 3, in <module>
>     from .exporters import *
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/__init__.py", 
> line 1, in <module>
>     from .export import *
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/export.py", 
> line 13, in <module>
>     from .templateexporter import TemplateExporter
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/templateexporter.py", 
> line 30, in <module>
>     from IPython.nbconvert import filters
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/__init__.py", 
> line 6, in <module>
>     from .markdown import *
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", 
> line 79, in <module>
>     class MathBlockLexer(mistune.BlockLexer):
>   File 
> "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", 
> line 80, in MathBlockLexer
>     default_rules = ['block_math', 'latex_environment'] + 
> mistune.BlockLexer.default_rules
> AttributeError: type object 'BlockLexer' has no attribute 'default_rules'
> 
> 
> 
> If, in markdown.py, I remove
> 
> + mistune.BlockLexer.default_rules on line 80, and
> + mistune.InlineLexer.default_rules on line 108, then everything seems 
> to work fine.
> 
> Could it be that the mistune version that I have (0.4) doesn't have the 
> default_rules property, but the version with which ipython was tested on 
> travis has? If so, shouldn't nbconvert bail out when the version 
> requirement is not fulfilled, or shouldn't simply the default_rules 
> property be defined, when the class is inherited in markdown.py? If this 
> is the case, I could file a fix on github.
> 
> Somewhat related: if there is a server error, wouldn't it be more 
> meaningful to return the traceback, instead of "500: internal server error"?
> 
> Cheers,
> Zolt?n
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/4d0de64f/attachment.html>

From zvoros at gmail.com  Mon Jan 19 06:40:45 2015
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 19 Jan 2015 12:40:45 +0100
Subject: [IPython-dev] status of nbconvert
In-Reply-To: <467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com>
References: <54BCE9C8.6080304@gmail.com>
	<467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com>
Message-ID: <54BCED3D.8040705@gmail.com>

Hi Matthias,

On 01/19/2015 12:31 PM, Matthias BUSSONNIER wrote:
>
>
> There is definitively a mistune api change,
> we check dependency in setup.py :
>
> https://github.com/ipython/ipython/blob/50799e6943174cbd24998c4fb018eff42aac5cfb/setup.py#L257
>
> 'mistune>=0.5'
>
> But we definitively don't check at runtime.
> So the bug should be encounters only by people having installed
> master some time ago and just pulling new versions.

Yes, I think this is what has happened. What is a bit odd, though, is 
that both easy_install and pip want to install version 0.4. I guess, you 
installed it from source.

Cheers,
Zolt?n


From bussonniermatthias at gmail.com  Mon Jan 19 07:06:00 2015
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 19 Jan 2015 13:06:00 +0100
Subject: [IPython-dev] status of nbconvert
In-Reply-To: <54BCED3D.8040705@gmail.com>
References: <54BCE9C8.6080304@gmail.com>
	<467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com>
	<54BCED3D.8040705@gmail.com>
Message-ID: <76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com>


Le 19 janv. 2015 ? 12:40, Zolt?n V?r?s a ?crit :

> Hi Matthias,
> 
> On 01/19/2015 12:31 PM, Matthias BUSSONNIER wrote:
>> 
>> 
>> There is definitively a mistune api change,
>> we check dependency in setup.py :
>> 
>> https://github.com/ipython/ipython/blob/50799e6943174cbd24998c4fb018eff42aac5cfb/setup.py#L257
>> 
>> 'mistune>=0.5'
>> 
>> But we definitively don't check at runtime.
>> So the bug should be encounters only by people having installed
>> master some time ago and just pulling new versions.
> 
> Yes, I think this is what has happened. What is a bit odd, though, is 
> that both easy_install and pip want to install version 0.4. I guess, you 
> installed it from source.

Hum, that's weird. 

I'll try to have a look. 

Thanks !

-- 
M


> 
> Cheers,
> Zolt?n
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From zvoros at gmail.com  Mon Jan 19 07:33:11 2015
From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 19 Jan 2015 13:33:11 +0100
Subject: [IPython-dev] status of nbconvert
In-Reply-To: <76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com>
References: <54BCE9C8.6080304@gmail.com>	<467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com>	<54BCED3D.8040705@gmail.com>
	<76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com>
Message-ID: <54BCF987.3030501@gmail.com>

Hi Matthias,


On 01/19/2015 01:06 PM, Matthias BUSSONNIER wrote:
>> Yes, I think this is what has happened. What is a bit odd, though, is
>> that both easy_install and pip want to install version 0.4. I guess, you
>> installed it from source.
> Hum, that's weird.
>
> I'll try to have a look.
>
> Thanks !

I have installed the source, and I no longer have problems with 
nbconvert. Is there someone whom I could notify to fix pip/easy_install? 
I have absolutely no idea how that is managed. I have seen that Thomas 
Kluyver is on the development team. Thomas?

Cheers,

Zolt?n



From cyrille.rossant at gmail.com  Mon Jan 19 07:30:15 2015
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Mon, 19 Jan 2015 13:30:15 +0100
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<D0E1E8D1.3E2F%soumith@fb.com>
	<AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com>
Message-ID: <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com>

Thanks for sharing this!

I'd like to add that there's also work being done currently for
bringing WebGL visualization to the notebook via VisPy. We don't use
three.js but custom WebGL code generated by Python. The WebGL backend
currently requires a live Python server, but a standalone version is
also in the pipes.

2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <bussonniermatthias at gmail.com>:
> Hi all,
> Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit :
>
> This is fantastic! I was going to do some of this work myself over the next
> month for iTorch (https://github.com/facebook/iTorch), and you saved me some
> trouble.
> I will take a look at your code over the next couple of days, I am wondering
> if you would have any issues if I directly adapted your javascript side of
> things in iTorch rather than rewriting from scratch.
>
>
> We still haven't really figured it out how to share javascript in between
> packages of different languages.
> (well technically in 3.0 yo can install into $NBEXTENSION directory, but
> install process is not perfect)
> If you can figured that out, and that all kernels agree on a set of
> relatively common JS "plugins"  that
> are often used we might be able to work on making  theses plugin available
> on nbviewer.
>
> Long term planning of course, but I'd like people to keep that in mind.
>
> --
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From takowl at gmail.com  Mon Jan 19 12:54:54 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 19 Jan 2015 09:54:54 -0800
Subject: [IPython-dev] status of nbconvert
In-Reply-To: <54BCF987.3030501@gmail.com>
References: <54BCE9C8.6080304@gmail.com>
	<467EF986-3EB8-40C8-9C16-38B6DF6D84CF@gmail.com>
	<54BCED3D.8040705@gmail.com>
	<76A22C20-FFC4-4B59-A06D-F144FCF863F2@gmail.com>
	<54BCF987.3030501@gmail.com>
Message-ID: <CAOvn4qiyETucNaNiZ9M5hJGMmkax=cur9NaJtuccx-ck5ZE1Dw@mail.gmail.com>

On 19 January 2015 at 04:33, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> I have installed the source, and I no longer have problems with
> nbconvert. Is there someone whom I could notify to fix pip/easy_install?
> I have absolutely no idea how that is managed. I have seen that Thomas
> Kluyver is on the development team. Thomas?
>

No ideas, sorry. My pip found mistune 0.5 OK. I wouldn't use easy_install
for anything now.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/8d8a4741/attachment.html>

From andy at payne.org  Mon Jan 19 14:30:30 2015
From: andy at payne.org (Andrew Payne)
Date: Mon, 19 Jan 2015 14:30:30 -0500
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
Message-ID: <CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com>

> I recently managed to get 3D vector plots working in the IPython notebook,
> by statically producing three.js output. I thought I'd share it here in
> case this example is useful for anyone else, or if anyone has any
> suggestions as to how I could do things better.
>

Great work!  Thanks for sharing that.

Did you consider or explore using require.js to load the three.js library?
You can reference and load the library once, and then not have to inline it
within each cell.

Aside for folks fiddling with Javscript libraries & the Web notebook:   the
obvious approach of dropping a <script> reference in an HTML cell or result
isn't robust.  When the cell is reexecuted, the browser will load the
library again, overwriting the original library name space.  However, the
first instance of the library will likely still be there because of
lingering references.  So you'll have a variable "foo" of type X in the old
library instance, and you'll pass it into a function call (new library
instance) and things will fail in strange ways because all the type tests
in the new library instance fail.  And you will drive yourself crazy
because you'll be in the JS console and wonder why foo is clearly of type
"X", but "isinstance X" is giving False.

That's why David's JS code has this test:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175

if ( camera instanceof THREE.Camera === false || !
document.body.contains(three_container)) {
            console.log("Animation loop failed: stopping");
            return;
        }

>
>
> The advantage of the static approach is that the output survives the
> nbconvert process, which you can see in this notebook here (which also
> compares with a couple of other unsatisfactory ways of visualising 3D
> vector plots).
>
>
> http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb
>
> Another example can be seen at:
>
>
> http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb
>
> If you are interested to see how this works, you can check out the python
> source:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py
>
> As well as the javascript:
>
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js
>
> and finally the HTML:
>
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html
>
> By the way, I am aware of the efforts to make a proper three.js IPython
> widget (https://github.com/jasongrout/pythreejs/), and will probably
> switch over to this once it is stable, and when the output can survive
> ipython's nbconvert process. Is this something we can expect for widgets in
> IPython 3.0?
>
> regards
> David
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/09199558/attachment.html>

From andy at payne.org  Mon Jan 19 14:39:18 2015
From: andy at payne.org (Andrew Payne)
Date: Mon, 19 Jan 2015 14:39:18 -0500
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com>
Message-ID: <CABbuyuVNKj57qYahiMa9szNft3TQeTD9=gqs_xU+BKfr+McnsQ@mail.gmail.com>

[*clicked send accidentally, please ignore my previous incomplete email*]

I recently managed to get 3D vector plots working in the IPython notebook,
> by statically producing three.js output. I thought I'd share it here in
> case this example is useful for anyone else, or if anyone has any
> suggestions as to how I could do things better.
>

Great work!  Thanks for sharing that.

Did you consider or explore using require.js to load the three.js library?
You can reference and load the library once, and then not have to inline it
within each cell.

Aside for folks fiddling with Javscript libraries & the Web notebook:   the
obvious approach of dropping a <script> reference in an HTML cell or result
isn't robust.  When the cell is reexecuted, the browser will load the
library again, overwriting the original library name space.  However, the
first instance of the library will likely still be there because of
lingering references.  So you'll have a variable "foo" of type X in the old
library instance, and you'll pass it into a function call (new library
instance) and things will fail in strange ways because all the type tests
in the new library instance fail.  And you will drive yourself crazy
because you'll be in the JS console and wonder why foo is clearly of type
"X", but "isinstanceof X" is giving False.

That's why David's JS code has this test:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175

       if ( *camera instanceof THREE.Camera === false* || !
document.body.contains(three_container)) {
            console.log("Animation loop failed: stopping");
            return;
        }

See my example of using require.js for three.js, here:
http://nbviewer.ipython.org/github/payne92/notebooks/blob/master/00%20Javascript%20In%20Notebooks.ipynb


For the dev team, this raises the issue of:  what's the best idiom for JS
in a cell to access the output area?  (See SO question:
http://stackoverflow.com/questions/20019690/how-to-access-current-cell-output-area-with-javascript-in-ipython-web-notebook
)

On a "live" notebook, the Javascript *element *variable is fine.  But as
you can see with my notebook (above) vs David's, the element variable
approach doesn't survive nbconvert -- my Javascript examples break because
element doesn't exist in an nbconvert(ed) notebook.

David made things work by generating (effectively) a UID as part of his
HTML cell result that the corresponding Javascript can pick up.

As we see Javascript used more with the Web notebook, folks are going to
consistently run into this pattern.  I think the notebook could provide
some "standard" way to reference the output area for the cell, that
survives nbconvert.

What do you think?

-andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/a0284f19/attachment.html>

From antgonza at gmail.com  Mon Jan 19 16:49:31 2015
From: antgonza at gmail.com (=?UTF-8?Q?Antonio_Gonz=C3=A1lez_Pe=C3=B1a?=)
Date: Mon, 19 Jan 2015 14:49:31 -0700
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
Message-ID: <CAH4xwWpfzr14uUteNYmLg9G3ayy9MvkUqUN9ZXat5hfwdOE4CA@mail.gmail.com>

Hello,

Related to this topic. We maintain Emperor
(http://biocore.github.io/emperor/), a package that uses python to
create very simple THREE.js code that can be embedded as a web page to
visualize and manipulate scatter plots + their metadata. We have
successfully been able to include Emperor plots into IPython notebooks
using _html_repr_. Note that in this case resources are stored in a
server, but we should really be making use of the $NBEXTENSION.

Anyway, as a side note we encountered a small issue with
nbviewer+Emperor: https://github.com/jupyter/nbviewer/issues/316,
which you perhaps might encounter.

Cheers,


On Sun, Jan 18, 2015 at 11:47 PM, David Powell
<DavidAnthonyPowell+python at gmail.com> wrote:
> Hello all,
>
> I recently managed to get 3D vector plots working in the IPython notebook,
> by statically producing three.js output. I thought I'd share it here in case
> this example is useful for anyone else, or if anyone has any suggestions as
> to how I could do things better.
>
> The advantage of the static approach is that the output survives the
> nbconvert process, which you can see in this notebook here (which also
> compares with a couple of other unsatisfactory ways of visualising 3D vector
> plots).
>
> http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb
>
> Another example can be seen at:
>
> http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb
>
> If you are interested to see how this works, you can check out the python
> source:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py
>
> As well as the javascript:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js
>
> and finally the HTML:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html
>
> By the way, I am aware of the efforts to make a proper three.js IPython
> widget (https://github.com/jasongrout/pythreejs/), and will probably switch
> over to this once it is stable, and when the output can survive ipython's
> nbconvert process. Is this something we can expect for widgets in IPython
> 3.0?
>
> regards
> David
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Antonio


From DavidAnthonyPowell+python at gmail.com  Mon Jan 19 19:47:36 2015
From: DavidAnthonyPowell+python at gmail.com (David Powell)
Date: Tue, 20 Jan 2015 11:47:36 +1100
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <D0E1E8D1.3E2F%soumith@fb.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<D0E1E8D1.3E2F%soumith@fb.com>
Message-ID: <CAGXzvidyYpqu2wYL721jdDCFOnoLz_PDB=yfdJpV_9nph=7jeQ@mail.gmail.com>

Hello Soumith,
My code is GPL3, but if you tell me which files you want to use, I will
release them under a BSD license to be compatible with itorch.
Regards,
David
 This is fantastic! I was going to do some of this work myself over the
next month for iTorch (https://github.com/facebook/iTorch), and you saved
me some trouble.
I will take a look at your code over the next couple of days, I am
wondering if you would have any issues if I directly adapted your
javascript side of things in iTorch rather than rewriting from scratch.
?
Soumith

  From: David Powell <DavidAnthonyPowell+python at gmail.com>
Reply-To: IPython developers list <ipython-dev at scipy.org>
Date: Sunday, January 18, 2015 at 10:47 PM
To: IPython developers list <ipython-dev at scipy.org>
Subject: [IPython-dev] WebGL example in IPython notebook based on three.js

  Hello all,

 I recently managed to get 3D vector plots working in the IPython notebook,
by statically producing three.js output. I thought I'd share it here in
case this example is useful for anyone else, or if anyone has any
suggestions as to how I could do things better.

 The advantage of the static approach is that the output survives the
nbconvert process, which you can see in this notebook here (which also
compares with a couple of other unsatisfactory ways of visualising 3D
vector plots).


http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb
<https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%2520to%2520create%25203D%2520plots.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=aad5ead78a1b3391d5b9c2a10e1214ef17fd7df8d223ad6076c6b78f9ff1246c>

 Another example can be seen at:

http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb
<https://urldefense.proofpoint.com/v1/url?u=http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%2520and%2520creating%2520geometric%2520shapes.ipynb&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=5S5uukCkPxLR1dZ0syN0fw%3D%3D%0A&m=70WnYrYAO8zL773kKr3zku5qmdNjduxrZK8pOaiqCaE%3D%0A&s=76948f7a6b29f01f0eccbe0899c7d91d427a1c1588d11ecbc4e568f992fddea7>

 If you are interested to see how this works, you can check out the python
source:

 https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py

 As well as the javascript:


https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js

 and finally the HTML:


https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html

 By the way, I am aware of the efforts to make a proper three.js IPython
widget (https://github.com/jasongrout/pythreejs/), and will probably switch
over to this once it is stable, and when the output can survive ipython's
nbconvert process. Is this something we can expect for widgets in IPython
3.0?

 regards
David


_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150120/e96afd4a/attachment.html>

From DavidAnthonyPowell+python at gmail.com  Mon Jan 19 21:17:09 2015
From: DavidAnthonyPowell+python at gmail.com (David Powell)
Date: Tue, 20 Jan 2015 13:17:09 +1100
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CABbuyuVNKj57qYahiMa9szNft3TQeTD9=gqs_xU+BKfr+McnsQ@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<CABbuyuVmV3ymZkYFUcbKCdPh_KhpbAKJW5S8_oguhTyPfomUvQ@mail.gmail.com>
	<CABbuyuVNKj57qYahiMa9szNft3TQeTD9=gqs_xU+BKfr+McnsQ@mail.gmail.com>
Message-ID: <CAGXzvievJLrsGPXV33AqwEOF=BrkM9wOV_D_A3CBugMNy=U8pA@mail.gmail.com>

>
> Did you consider or explore using require.js to load the three.js
> library?  You can reference and load the library once, and then not have to
> inline it within each cell.
>
>
My reason for not using require is because I need parts of three.js (e.g
orbit controls) which are not part of the main library, but are instead
included as examples, and hence are not hosted on CDNs.
Also, my code for injecting javascript libraries is in a separate function
which only needs to be called once at the start of the notebook, so it
shouldn't need to be included more than once.



> Aside for folks fiddling with Javscript libraries & the Web notebook:
> the obvious approach of dropping a <script> reference in an HTML cell or
> result isn't robust.  When the cell is reexecuted, the browser will load
> the library again, overwriting the original library name space.  However,
> the first instance of the library will likely still be there because of
> lingering references.  So you'll have a variable "foo" of type X in the old
> library instance, and you'll pass it into a function call (new library
> instance) and things will fail in strange ways because all the type tests
> in the new library instance fail.  And you will drive yourself crazy
> because you'll be in the JS console and wonder why foo is clearly of type
> "X", but "isinstanceof X" is giving False.
>
> That's why David's JS code has this test:
>
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175
>
>        if ( *camera instanceof THREE.Camera === false* || !
> document.body.contains(three_container)) {
>             console.log("Animation loop failed: stopping");
>             return;
>         }
>
>
I agree that it's a horrible hack, I'm definitely looking for a more
elegant long-term solution. This hack here isn't strictly needed for the
code to work, but if it's missing, then the browser's javascript console
will spew out error messages 60 times per second, which makes debugging
javascript essentially impossible. I think I minimise some of the problems
you describe by injecting the javascript libraries once at the start,
rather than with every plot.


> See my example of using require.js for three.js, here:
> http://nbviewer.ipython.org/github/payne92/notebooks/blob/master/00%20Javascript%20In%20Notebooks.ipynb
>
>
 I did find your examples online, but had real trouble getting your
approach to work with the additional javascript from the three.js examples.
So in the end I just went back to this simpler approach of directly
injecting code.


>
> For the dev team, this raises the issue of:  what's the best idiom for JS
> in a cell to access the output area?  (See SO question:
> http://stackoverflow.com/questions/20019690/how-to-access-current-cell-output-area-with-javascript-in-ipython-web-notebook
> )
>
> On a "live" notebook, the Javascript *element *variable is fine.  But as
> you can see with my notebook (above) vs David's, the element variable
> approach doesn't survive nbconvert -- my Javascript examples break because
> element doesn't exist in an nbconvert(ed) notebook.
>
> David made things work by generating (effectively) a UID as part of his
> HTML cell result that the corresponding Javascript can pick up.
>
> As we see Javascript used more with the Web notebook, folks are going to
> consistently run into this pattern.  I think the notebook could provide
> some "standard" way to reference the output area for the cell, that
> survives nbconvert.
>

I agree, you've exactly described the problems I ran into, and the reason
why I generate html with a <script> tag rather than javascript output.


>
> What do you think?
>
> -andy
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150120/763e1e1e/attachment.html>

From DavidAnthonyPowell+python at gmail.com  Mon Jan 19 21:44:28 2015
From: DavidAnthonyPowell+python at gmail.com (David Powell)
Date: Tue, 20 Jan 2015 13:44:28 +1100
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<D0E1E8D1.3E2F%soumith@fb.com>
	<AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com>
	<CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com>
Message-ID: <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com>

Hello Cyrille,

Thanks for the info. In case it helps you, the reasons why I didn't go with
vispy are
- When I started this coding, the IPython notebook examples for vispy
weren't readily available (I vagely remember failing to find them with
google even though something was mentioned on the home page)
- The idea that you *have* to write low-level GLSL code seems a bit
intimidating (with three.js you can, but don't need to)
- three.js includes nice controls to pan, zoom etc. out of the box
- Because I'm trying to attract users to my software, I think it's
absolutely essential to be able to use the nbconverted output to show nice
interactive plots to potential users who haven't installed my program yet
(and who may not even be familiar with the scipy ecosystem), as a kind of
"marketing" tool.

However, I'll definitely keep an eye out for progress on vispy, it sounds
like some cool features are planned. The one thing I don't like about
three.js is that it is very much fixed around the idea of always animating
at 60 frames/sec for games etc. This means that it is chewing up
significant CPU, even if the figure if not updating.

regards
David

On 19 January 2015 at 23:30, Cyrille Rossant <cyrille.rossant at gmail.com>
wrote:

> Thanks for sharing this!
>
> I'd like to add that there's also work being done currently for
> bringing WebGL visualization to the notebook via VisPy. We don't use
> three.js but custom WebGL code generated by Python. The WebGL backend
> currently requires a live Python server, but a standalone version is
> also in the pipes.
>
> 2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <
> bussonniermatthias at gmail.com>:
> > Hi all,
> > Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit :
> >
> > This is fantastic! I was going to do some of this work myself over the
> next
> > month for iTorch (https://github.com/facebook/iTorch), and you saved me
> some
> > trouble.
> > I will take a look at your code over the next couple of days, I am
> wondering
> > if you would have any issues if I directly adapted your javascript side
> of
> > things in iTorch rather than rewriting from scratch.
> >
> >
> > We still haven't really figured it out how to share javascript in between
> > packages of different languages.
> > (well technically in 3.0 yo can install into $NBEXTENSION directory, but
> > install process is not perfect)
> > If you can figured that out, and that all kernels agree on a set of
> > relatively common JS "plugins"  that
> > are often used we might be able to work on making  theses plugin
> available
> > on nbviewer.
> >
> > Long term planning of course, but I'd like people to keep that in mind.
> >
> > --
> > M
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150120/8ee50390/attachment.html>

From sylvain.corlay at gmail.com  Mon Jan 19 22:05:12 2015
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Mon, 19 Jan 2015 22:05:12 -0500
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<D0E1E8D1.3E2F%soumith@fb.com>
	<AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com>
	<CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com>
	<CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com>
Message-ID: <CAK=Phk7ugWXRC13A4xXGjKVZTF8RsaC-XOCYLezhWo=+-kVPbg@mail.gmail.com>

@David, PyThreejs works pretty well with the current dev version of IPython
(3.0dev). Older versions should not be supported as it uses the new API
with the javascript promises. (Ping Jason Grout).

Sylvain

On Mon, Jan 19, 2015 at 9:44 PM, David Powell <
DavidAnthonyPowell+python at gmail.com> wrote:

> Hello Cyrille,
>
> Thanks for the info. In case it helps you, the reasons why I didn't go
> with vispy are
> - When I started this coding, the IPython notebook examples for vispy
> weren't readily available (I vagely remember failing to find them with
> google even though something was mentioned on the home page)
> - The idea that you *have* to write low-level GLSL code seems a bit
> intimidating (with three.js you can, but don't need to)
> - three.js includes nice controls to pan, zoom etc. out of the box
> - Because I'm trying to attract users to my software, I think it's
> absolutely essential to be able to use the nbconverted output to show nice
> interactive plots to potential users who haven't installed my program yet
> (and who may not even be familiar with the scipy ecosystem), as a kind of
> "marketing" tool.
>
> However, I'll definitely keep an eye out for progress on vispy, it sounds
> like some cool features are planned. The one thing I don't like about
> three.js is that it is very much fixed around the idea of always animating
> at 60 frames/sec for games etc. This means that it is chewing up
> significant CPU, even if the figure if not updating.
>
> regards
> David
>
> On 19 January 2015 at 23:30, Cyrille Rossant <cyrille.rossant at gmail.com>
> wrote:
>
>> Thanks for sharing this!
>>
>> I'd like to add that there's also work being done currently for
>> bringing WebGL visualization to the notebook via VisPy. We don't use
>> three.js but custom WebGL code generated by Python. The WebGL backend
>> currently requires a live Python server, but a standalone version is
>> also in the pipes.
>>
>> 2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <
>> bussonniermatthias at gmail.com>:
>> > Hi all,
>> > Le 19 janv. 2015 ? 07:50, Soumith Chintala a ?crit :
>> >
>> > This is fantastic! I was going to do some of this work myself over the
>> next
>> > month for iTorch (https://github.com/facebook/iTorch), and you saved
>> me some
>> > trouble.
>> > I will take a look at your code over the next couple of days, I am
>> wondering
>> > if you would have any issues if I directly adapted your javascript side
>> of
>> > things in iTorch rather than rewriting from scratch.
>> >
>> >
>> > We still haven't really figured it out how to share javascript in
>> between
>> > packages of different languages.
>> > (well technically in 3.0 yo can install into $NBEXTENSION directory, but
>> > install process is not perfect)
>> > If you can figured that out, and that all kernels agree on a set of
>> > relatively common JS "plugins"  that
>> > are often used we might be able to work on making  theses plugin
>> available
>> > on nbviewer.
>> >
>> > Long term planning of course, but I'd like people to keep that in mind.
>> >
>> > --
>> > M
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150119/35521b2f/attachment.html>

From cyrille.rossant at gmail.com  Tue Jan 20 04:09:08 2015
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Tue, 20 Jan 2015 10:09:08 +0100
Subject: [IPython-dev] WebGL example in IPython notebook based on
	three.js
In-Reply-To: <CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com>
References: <CAGXzvicZrGe8xbH_CFEzbsM8Rakfn+SGqX3B8AcJEcPv+E-PSg@mail.gmail.com>
	<D0E1E8D1.3E2F%soumith@fb.com>
	<AE913428-931D-42EC-BD29-5E062BA3A709@gmail.com>
	<CA+-1RQTj_=3qGri9n+HKtvjmFWgWOLtGDjxeQZSXsLVEkrme8g@mail.gmail.com>
	<CAGXzvienQNyrbSENJzKt2N2vzp=iW6SkAPL=8v3WSt7jUC5AOQ@mail.gmail.com>
Message-ID: <CA+-1RQSYZwpnFfe7JYzWue5hPhSkWAC33VHgFQgLDRb0faMCFg@mail.gmail.com>

Thank you for your feedback. VisPy/WebGL is definitely not yet ready
for production, I was essentially mentioning that there was work in
progress in that direction!

> - When I started this coding, the IPython notebook examples for vispy
> weren't readily available (I vagely remember failing to find them with
> google even though something was mentioned on the home page)

It's still experimental at this point, but a more robust version will
be available in the coming weeks.

> - The idea that you *have* to write low-level GLSL code seems a bit
> intimidating (with three.js you can, but don't need to)

That's actually not the case in the latest version, see for example
<http://vispy.org/examples/basics/scene/surface_plot.html>. We do have
high-level interfaces that don't require users to write GLSL. Only the
most advanced users will be interested in writing custom GLSL.

> - three.js includes nice controls to pan, zoom etc. out of the box

Idem, in VisPy we do have interactive "cameras" that provide controls
to pan, zoom, etc.

> - Because I'm trying to attract users to my software, I think it's
> absolutely essential to be able to use the nbconverted output to show nice
> interactive plots to potential users who haven't installed my program yet
> (and who may not even be familiar with the scipy ecosystem), as a kind of
> "marketing" tool.

That's a good point and that's definitely something on our radar.

> However, I'll definitely keep an eye out for progress on vispy, it sounds
> like some cool features are planned. The one thing I don't like about
> three.js is that it is very much fixed around the idea of always animating
> at 60 frames/sec for games etc. This means that it is chewing up significant
> CPU, even if the figure if not updating.

VisPy is indeed more efficient in that the scene is only updated when
needed (pan, zoom, etc.). But you can also have timer-based
animations.

Cyrille


From hughesadam87 at gmail.com  Wed Jan 21 17:07:55 2015
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 21 Jan 2015 17:07:55 -0500
Subject: [IPython-dev] Replacing PopUp?
Message-ID: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com>

Hi,

We have a widget GUI that was previously using Popup, and I noticed it's
been deleted from dev.  We're kind of stuck now in a weird point between
2.x and master, and it seems like everytime we update ipython, a new widget
component is changed.  So my question is, can we still get Popup behavior
in our widgets?  IE can I go through the code and make a simple fix to
retain the old functionality of the Popups?  And secondly, are there a lot
more major changes to the widget system planned?   If so, we'll probably
bundle our working version if IPython rather than try to maintain
compatibility until the official 3.x release.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150121/9bdb416d/attachment.html>

From ellisonbg at gmail.com  Wed Jan 21 17:19:16 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 21 Jan 2015 14:19:16 -0800
Subject: [IPython-dev] Replacing PopUp?
In-Reply-To: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com>
References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com>
Message-ID: <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com>

Adam,

We apologize for all of the instability. We consider the IPython widgets to
be research grade code for now and there will continue to be API and
architecture changes for a few releases to come. For 3.0 we have added a
warning upon import that clarifies this aspect of widgets.

Moving forward, there will eventually be something that provides popup-like
capabilities for widgets, but it was causing a lots of problems and the
implementation had usability/UI problems for users. But, you should be able
to take the popup widget out of ipython 2.0 and get it working with 3.0
when it comes out as a third party widget.

Cheers,

Brian

On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:

> Hi,
>
> We have a widget GUI that was previously using Popup, and I noticed it's
> been deleted from dev.  We're kind of stuck now in a weird point between
> 2.x and master, and it seems like everytime we update ipython, a new widget
> component is changed.  So my question is, can we still get Popup behavior
> in our widgets?  IE can I go through the code and make a simple fix to
> retain the old functionality of the Popups?  And secondly, are there a lot
> more major changes to the widget system planned?   If so, we'll probably
> bundle our working version if IPython rather than try to maintain
> compatibility until the official 3.x release.
>
> Thanks
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150121/887e2862/attachment.html>

From hughesadam87 at gmail.com  Wed Jan 21 20:21:27 2015
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Wed, 21 Jan 2015 20:21:27 -0500
Subject: [IPython-dev] Replacing PopUp?
In-Reply-To: <CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com>
References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com>
	<CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com>
Message-ID: <CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com>

Thanks Brian.  Turns out the Popup, while imported, was not even used by
our program.  The new notebook looks awesome by the way.

On Wed, Jan 21, 2015 at 5:19 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Adam,
>
> We apologize for all of the instability. We consider the IPython widgets
> to be research grade code for now and there will continue to be API and
> architecture changes for a few releases to come. For 3.0 we have added a
> warning upon import that clarifies this aspect of widgets.
>
> Moving forward, there will eventually be something that provides
> popup-like capabilities for widgets, but it was causing a lots of problems
> and the implementation had usability/UI problems for users. But, you should
> be able to take the popup widget out of ipython 2.0 and get it working with
> 3.0 when it comes out as a third party widget.
>
> Cheers,
>
> Brian
>
> On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com>
> wrote:
>
>> Hi,
>>
>> We have a widget GUI that was previously using Popup, and I noticed it's
>> been deleted from dev.  We're kind of stuck now in a weird point between
>> 2.x and master, and it seems like everytime we update ipython, a new widget
>> component is changed.  So my question is, can we still get Popup behavior
>> in our widgets?  IE can I go through the code and make a simple fix to
>> retain the old functionality of the Popups?  And secondly, are there a lot
>> more major changes to the widget system planned?   If so, we'll probably
>> bundle our working version if IPython rather than try to maintain
>> compatibility until the official 3.x release.
>>
>> Thanks
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150121/5ea46b21/attachment.html>

From bussonniermatthias at gmail.com  Thu Jan 22 04:20:07 2015
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 22 Jan 2015 10:20:07 +0100
Subject: [IPython-dev] Replacing PopUp?
In-Reply-To: <CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com>
References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com>
	<CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com>
	<CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com>
Message-ID: <3E532361-42C7-404A-BC2F-3DED7C18FEBD@gmail.com>


Le 22 janv. 2015 ? 02:21, Adam Hughes a ?crit :

> Thanks Brian.  Turns out the Popup, while imported, was not even used by our program.  The new notebook looks awesome by the way.

Thanks, 

I want to precise that widgets will become a standalone repo at some point in the future, so could get a release schedule decoupled from IPython.

-- 
M




> 
> On Wed, Jan 21, 2015 at 5:19 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Adam,
> 
> We apologize for all of the instability. We consider the IPython widgets to be research grade code for now and there will continue to be API and architecture changes for a few releases to come. For 3.0 we have added a warning upon import that clarifies this aspect of widgets.
> 
> Moving forward, there will eventually be something that provides popup-like capabilities for widgets, but it was causing a lots of problems and the implementation had usability/UI problems for users. But, you should be able to take the popup widget out of ipython 2.0 and get it working with 3.0 when it comes out as a third party widget.
> 
> Cheers,
> 
> Brian
> 
> On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com> wrote:
> Hi,
> 
> We have a widget GUI that was previously using Popup, and I noticed it's been deleted from dev.  We're kind of stuck now in a weird point between 2.x and master, and it seems like everytime we update ipython, a new widget component is changed.  So my question is, can we still get Popup behavior in our widgets?  IE can I go through the code and make a simple fix to retain the old functionality of the Popups?  And secondly, are there a lot more major changes to the widget system planned?   If so, we'll probably bundle our working version if IPython rather than try to maintain compatibility until the official 3.x release.
> 
> Thanks
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150122/d1f171a4/attachment.html>

From hughesadam87 at gmail.com  Thu Jan 22 14:06:28 2015
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Thu, 22 Jan 2015 14:06:28 -0500
Subject: [IPython-dev] Replacing PopUp?
In-Reply-To: <3E532361-42C7-404A-BC2F-3DED7C18FEBD@gmail.com>
References: <CAMHV+dCRVZbP554br_YBqDPc4DUexADotuZJDU+tX2FVBJd1gw@mail.gmail.com>
	<CAH4pYpShRHR4FTdDe0gQ9mU8w+STCu+K28LOKz85a84N76uSoA@mail.gmail.com>
	<CAMHV+dCz-0NF-PQBU9KcHRn8k=cxPmnHeykzFWeNKQzH15djaA@mail.gmail.com>
	<3E532361-42C7-404A-BC2F-3DED7C18FEBD@gmail.com>
Message-ID: <CAMHV+dB5EGbRctVXCDfYBf9V=jKhDEPYr3EA-H=43qi_i2KUDA@mail.gmail.com>

Cool, thanks for the heads up

On Thu, Jan 22, 2015 at 4:20 AM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

>
> Le 22 janv. 2015 ? 02:21, Adam Hughes a ?crit :
>
> Thanks Brian.  Turns out the Popup, while imported, was not even used by
> our program.  The new notebook looks awesome by the way.
>
>
> Thanks,
>
> I want to precise that widgets will become a standalone repo at some point
> in the future, so could get a release schedule decoupled from IPython.
>
> --
> M
>
>
>
>
>
> On Wed, Jan 21, 2015 at 5:19 PM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>
>> Adam,
>>
>> We apologize for all of the instability. We consider the IPython widgets
>> to be research grade code for now and there will continue to be API and
>> architecture changes for a few releases to come. For 3.0 we have added a
>> warning upon import that clarifies this aspect of widgets.
>>
>> Moving forward, there will eventually be something that provides
>> popup-like capabilities for widgets, but it was causing a lots of problems
>> and the implementation had usability/UI problems for users. But, you should
>> be able to take the popup widget out of ipython 2.0 and get it working with
>> 3.0 when it comes out as a third party widget.
>>
>> Cheers,
>>
>> Brian
>>
>> On Wed, Jan 21, 2015 at 2:07 PM, Adam Hughes <hughesadam87 at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> We have a widget GUI that was previously using Popup, and I noticed it's
>>> been deleted from dev.  We're kind of stuck now in a weird point between
>>> 2.x and master, and it seems like everytime we update ipython, a new widget
>>> component is changed.  So my question is, can we still get Popup behavior
>>> in our widgets?  IE can I go through the code and make a simple fix to
>>> retain the old functionality of the Popups?  And secondly, are there a lot
>>> more major changes to the widget system planned?   If so, we'll probably
>>> bundle our working version if IPython rather than try to maintain
>>> compatibility until the official 3.x release.
>>>
>>> Thanks
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150122/6db13f3e/attachment.html>

From doug.blank at gmail.com  Fri Jan 23 09:01:29 2015
From: doug.blank at gmail.com (Doug Blank)
Date: Fri, 23 Jan 2015 09:01:29 -0500
Subject: [IPython-dev] Jupyter + Processing for education
Message-ID: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com>

IPython community,

Thought you might be interested in seeing how students can use the new
Jupyterhub with a new, external kernel. This semester I am using the
language "Processing"... originally designed for artists by artists. My
colleagues have written a textbook on how to use it for teaching computer
science [1].

Here is a 5 minute video on how it looks to the student:

https://www.youtube.com/watch?v=V4TzARh-ClY

and here is a live notebook:

http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb

(mouse over some of the sketches)

If you know Processing, this is different from the regular Processing IDE:

* students log into the Jupyterhub system over the web
* uses the Processing-Javascript system
* each Jupyter notebook "cell" is a Processing "sketch"
* Shift+Tab shows Processing on-line help
* Tab will do completion (completes with Processing variables and functions)
* Rendered notebooks (eg, without a live computational engine/kernel behind
them) will still run
* Slight bit of interpreter feel: can execute draw/setup functions
interactively
* uses MetaKernel magics [2]

I like the setup for education because:

* I can control/configure the environment exactly (what buttons are
available, etc). I have a few nbextensions in the newly supported system
folder, and load them in the notebook.html template.
* I can update the system at any time
* I can track students' time-on-task
* Students will be able to submit their homework (as notebooks) via a button
* I can add more stuff as I think of it (like the "Publish" and "Submit
Homework" buttons)
* All of the benefits of using a notebook: by students, for students ("runs
like code; reads like a journal paper", "literate computing", "reproducible
research", etc)

Things I don't like:

* Processing -> Javascript error messages are terrible

We'll have all of the code easily installable (via pip) as soon as Jupyter
is out. Of course, you can also get the code directly [3].

It could be that a Processing IPython magic could turn Processing code into
a really fancy, low-level, dynamic widget system for Python. Because
rendered Processing sketches run in rendered notebooks, it also solves some
of those issues. There needs to be a way to get data to Processing. Also,
for live kernels, it would be handy to have Processing be able to set
Python variables.

-Doug

[1] - http://www.apress.com/9781430244646
[2] - https://github.com/Calysto/metakernel
[3] -
https://github.com/Calysto/calysto/tree/master/calysto/language/processing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/add33c91/attachment.html>

From python at elbonia.de  Fri Jan 23 11:42:43 2015
From: python at elbonia.de (Juergen Hasch)
Date: Fri, 23 Jan 2015 17:42:43 +0100
Subject: [IPython-dev] Jupyter + Processing for education
In-Reply-To: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com>
References: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com>
Message-ID: <54C27A03.6080408@elbonia.de>

Hey Doug,
this is nice!

I always wanted to something like that, but never got around. You can be sure I will play with it.

  Juergen

Am 23.01.2015 um 15:01 schrieb Doug Blank:
> IPython community,
> 
> Thought you might be interested in seeing how students can use the new Jupyterhub with a new, external kernel. This
> semester I am using the language "Processing"... originally designed for artists by artists. My colleagues have written
> a textbook on how to use it for teaching computer science [1].
> 
> Here is a 5 minute video on how it looks to the student:
> 
> https://www.youtube.com/watch?v=V4TzARh-ClY
> 
> and here is a live notebook:
> 
> http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb
> 
> (mouse over some of the sketches)
> 
> If you know Processing, this is different from the regular Processing IDE:
> 
> * students log into the Jupyterhub system over the web
> * uses the Processing-Javascript system
> * each Jupyter notebook "cell" is a Processing "sketch"
> * Shift+Tab shows Processing on-line help
> * Tab will do completion (completes with Processing variables and functions)
> * Rendered notebooks (eg, without a live computational engine/kernel behind them) will still run
> * Slight bit of interpreter feel: can execute draw/setup functions interactively
> * uses MetaKernel magics [2]
> 
> I like the setup for education because:
> 
> * I can control/configure the environment exactly (what buttons are available, etc). I have a few nbextensions in the
> newly supported system folder, and load them in the notebook.html template.
> * I can update the system at any time
> * I can track students' time-on-task
> * Students will be able to submit their homework (as notebooks) via a button
> * I can add more stuff as I think of it (like the "Publish" and "Submit Homework" buttons)
> * All of the benefits of using a notebook: by students, for students ("runs like code; reads like a journal paper",
> "literate computing", "reproducible research", etc)
> 
> Things I don't like:
> 
> * Processing -> Javascript error messages are terrible
> 
> We'll have all of the code easily installable (via pip) as soon as Jupyter is out. Of course, you can also get the code
> directly [3].
> 
> It could be that a Processing IPython magic could turn Processing code into a really fancy, low-level, dynamic widget
> system for Python. Because rendered Processing sketches run in rendered notebooks, it also solves some of those issues.
> There needs to be a way to get data to Processing. Also, for live kernels, it would be handy to have Processing be able
> to set Python variables.
> 
> -Doug
> 
> [1] - http://www.apress.com/9781430244646
> [2] - https://github.com/Calysto/metakernel
> [3] - https://github.com/Calysto/calysto/tree/master/calysto/language/processing
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 



From ellisonbg at gmail.com  Fri Jan 23 12:56:41 2015
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 23 Jan 2015 09:56:41 -0800
Subject: [IPython-dev] Jupyter + Processing for education
In-Reply-To: <54C27A03.6080408@elbonia.de>
References: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com>
	<54C27A03.6080408@elbonia.de>
Message-ID: <CAH4pYpRd9XRwg+uixWVcSuPOje+VtVAtw7uYKtn-6trzm6E3eA@mail.gmail.com>

Love it!

On Fri, Jan 23, 2015 at 8:42 AM, Juergen Hasch <python at elbonia.de> wrote:

> Hey Doug,
> this is nice!
>
> I always wanted to something like that, but never got around. You can be
> sure I will play with it.
>
>   Juergen
>
> Am 23.01.2015 um 15:01 schrieb Doug Blank:
> > IPython community,
> >
> > Thought you might be interested in seeing how students can use the new
> Jupyterhub with a new, external kernel. This
> > semester I am using the language "Processing"... originally designed for
> artists by artists. My colleagues have written
> > a textbook on how to use it for teaching computer science [1].
> >
> > Here is a 5 minute video on how it looks to the student:
> >
> > https://www.youtube.com/watch?v=V4TzARh-ClY
> >
> > and here is a live notebook:
> >
> > http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb
> >
> > (mouse over some of the sketches)
> >
> > If you know Processing, this is different from the regular Processing
> IDE:
> >
> > * students log into the Jupyterhub system over the web
> > * uses the Processing-Javascript system
> > * each Jupyter notebook "cell" is a Processing "sketch"
> > * Shift+Tab shows Processing on-line help
> > * Tab will do completion (completes with Processing variables and
> functions)
> > * Rendered notebooks (eg, without a live computational engine/kernel
> behind them) will still run
> > * Slight bit of interpreter feel: can execute draw/setup functions
> interactively
> > * uses MetaKernel magics [2]
> >
> > I like the setup for education because:
> >
> > * I can control/configure the environment exactly (what buttons are
> available, etc). I have a few nbextensions in the
> > newly supported system folder, and load them in the notebook.html
> template.
> > * I can update the system at any time
> > * I can track students' time-on-task
> > * Students will be able to submit their homework (as notebooks) via a
> button
> > * I can add more stuff as I think of it (like the "Publish" and "Submit
> Homework" buttons)
> > * All of the benefits of using a notebook: by students, for students
> ("runs like code; reads like a journal paper",
> > "literate computing", "reproducible research", etc)
> >
> > Things I don't like:
> >
> > * Processing -> Javascript error messages are terrible
> >
> > We'll have all of the code easily installable (via pip) as soon as
> Jupyter is out. Of course, you can also get the code
> > directly [3].
> >
> > It could be that a Processing IPython magic could turn Processing code
> into a really fancy, low-level, dynamic widget
> > system for Python. Because rendered Processing sketches run in rendered
> notebooks, it also solves some of those issues.
> > There needs to be a way to get data to Processing. Also, for live
> kernels, it would be handy to have Processing be able
> > to set Python variables.
> >
> > -Doug
> >
> > [1] - http://www.apress.com/9781430244646
> > [2] - https://github.com/Calysto/metakernel
> > [3] -
> https://github.com/Calysto/calysto/tree/master/calysto/language/processing
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/0a99ceea/attachment.html>

From doug.blank at gmail.com  Fri Jan 23 13:40:14 2015
From: doug.blank at gmail.com (Doug Blank)
Date: Fri, 23 Jan 2015 13:40:14 -0500
Subject: [IPython-dev] Jupyter + Processing for education
In-Reply-To: <CAH4pYpRd9XRwg+uixWVcSuPOje+VtVAtw7uYKtn-6trzm6E3eA@mail.gmail.com>
References: <CAAusYCjQYpYX5C62fjGrRHuUDaP5Wigt6CWj8=aLnumMyCnMEA@mail.gmail.com>
	<54C27A03.6080408@elbonia.de>
	<CAH4pYpRd9XRwg+uixWVcSuPOje+VtVAtw7uYKtn-6trzm6E3eA@mail.gmail.com>
Message-ID: <CAAusYCjLvWjV8sONX8=KwZL1Z60pL5NRKYjmdW_F9677XapDGQ@mail.gmail.com>

On Fri, Jan 23, 2015 at 12:56 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Love it!
>

Thanks, guys! BTW, the background music is "Jupiter's Descendants" ...
which is so punny and appropriate for the Calysto project :)  Anticipating
"Jupiter Ascending" due in theatres on Feb 9th... maybe you can have the
official beta launch then... 2 weeks three days!

-Doug

[1] - https://soundcloud.com/victory/cinematic-layout-jupiters


>
> On Fri, Jan 23, 2015 at 8:42 AM, Juergen Hasch <python at elbonia.de> wrote:
>
>> Hey Doug,
>> this is nice!
>>
>> I always wanted to something like that, but never got around. You can be
>> sure I will play with it.
>>
>>   Juergen
>>
>> Am 23.01.2015 um 15:01 schrieb Doug Blank:
>> > IPython community,
>> >
>> > Thought you might be interested in seeing how students can use the new
>> Jupyterhub with a new, external kernel. This
>> > semester I am using the language "Processing"... originally designed
>> for artists by artists. My colleagues have written
>> > a textbook on how to use it for teaching computer science [1].
>> >
>> > Here is a 5 minute video on how it looks to the student:
>> >
>> > https://www.youtube.com/watch?v=V4TzARh-ClY
>> >
>> > and here is a live notebook:
>> >
>> > http://jupyter.cs.brynmawr.edu/hub/kohara/public/Untitled.ipynb
>> >
>> > (mouse over some of the sketches)
>> >
>> > If you know Processing, this is different from the regular Processing
>> IDE:
>> >
>> > * students log into the Jupyterhub system over the web
>> > * uses the Processing-Javascript system
>> > * each Jupyter notebook "cell" is a Processing "sketch"
>> > * Shift+Tab shows Processing on-line help
>> > * Tab will do completion (completes with Processing variables and
>> functions)
>> > * Rendered notebooks (eg, without a live computational engine/kernel
>> behind them) will still run
>> > * Slight bit of interpreter feel: can execute draw/setup functions
>> interactively
>> > * uses MetaKernel magics [2]
>> >
>> > I like the setup for education because:
>> >
>> > * I can control/configure the environment exactly (what buttons are
>> available, etc). I have a few nbextensions in the
>> > newly supported system folder, and load them in the notebook.html
>> template.
>> > * I can update the system at any time
>> > * I can track students' time-on-task
>> > * Students will be able to submit their homework (as notebooks) via a
>> button
>> > * I can add more stuff as I think of it (like the "Publish" and "Submit
>> Homework" buttons)
>> > * All of the benefits of using a notebook: by students, for students
>> ("runs like code; reads like a journal paper",
>> > "literate computing", "reproducible research", etc)
>> >
>> > Things I don't like:
>> >
>> > * Processing -> Javascript error messages are terrible
>> >
>> > We'll have all of the code easily installable (via pip) as soon as
>> Jupyter is out. Of course, you can also get the code
>> > directly [3].
>> >
>> > It could be that a Processing IPython magic could turn Processing code
>> into a really fancy, low-level, dynamic widget
>> > system for Python. Because rendered Processing sketches run in rendered
>> notebooks, it also solves some of those issues.
>> > There needs to be a way to get data to Processing. Also, for live
>> kernels, it would be handy to have Processing be able
>> > to set Python variables.
>> >
>> > -Doug
>> >
>> > [1] - http://www.apress.com/9781430244646
>> > [2] - https://github.com/Calysto/metakernel
>> > [3] -
>> https://github.com/Calysto/calysto/tree/master/calysto/language/processing
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgranger at calpoly.edu and ellisonbg at gmail.com
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/c41a2c17/attachment.html>

From chris at plot.ly  Fri Jan 23 14:03:52 2015
From: chris at plot.ly (Chris P)
Date: Fri, 23 Jan 2015 14:03:52 -0500
Subject: [IPython-dev] Plotly Interactive Graph Widget
Message-ID: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com>

Hey IPython Community --

I wanted to share with you Plotly's new IPython graph widget. A couple GIFs:

Contour Plot Explorer
<http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092/Widgets%20-%20Contour%20Plotter.ipynb>
-
The bottom graph updates with the elevation and distance of the points that
are clicked on in the top-graph.
(Link to GIF: http://i.imgur.com/2Wbbo6l.gif)


Dataframe Explorer of NYC 311 calls
<http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092/Pandas%20Widget%20-%20Time%20Series,%20Search,%20and%20Filtering.ipynb>:
Zooming and panning in the top time-series graph updates the bottom graph.
Updating the search, dropdown, and slider values perform Pandas calls and
Plotly restyle commands.

(Link to GIF: http://i.imgur.com/h5GLIHc.gif)



This involved a couple of neat interactions with the Widget framework:
- Events from our JS graphing library (hover, click, zoom) get passed to
the Python kernel. Python callbacks can be assigned with e.g.
GraphWidget.on_click(callback)

- All the functions from our JS graphing lib are exposed in the Python
GraphWidget: restyle, add_traces, delete_traces, reorder_traces, relayout

- We're embedded these graphs as IFrames and all communication from the
IPython JS client happens through postMessage. Since the GraphWidget is
initialized with the unique URL of the graph, it's possible to swap graphs
in and out of IPython widgets. For example, that Contour Plot Explorer
<http://nbviewer.ipython.org/9827cb5086cfab8da092/Widgets%20-%20Contour%20Plotter.ipynb>
will work with any Contour Plot, the user just has to replace contour_plot
= GraphWidget('https://plot.ly/~bronsolo/63') with a URL of their own
contour plot.

The widget code is here:
https://github.com/plotly/python-api/tree/master/plotly/widgets. Any
feedback or suggestions is greatly appreciated!

Some comments from the development POV:
- JS tracebacks were really difficult to debug.
- For easy use by the user, I decided to inject the JS dynamically on import
<https://github.com/plotly/python-api/blob/master/plotly/widgets/graph_widget.py#L17-L22>.
Adding //@ source
<https://github.com/plotly/python-api/blob/master/plotly/widgets/graphWidget.js#L129>
to the end of the injected JS made the JS code appear in Chrome's dev
tools. Without //@ source, the dynamic JS was lost in a Chrome VM.
- I'm using Unicode(sync=True) to pass messages back and forth, but really
I felt like I should just be using the underlying comm interface instead of
the MVC framework. But, I had trouble finding good documentation on that
interface.
- Looking forward to a "knob" widget fora ipython etch-a-sketch
<http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092/Widgets%20-%20Sliders%20and%20Etch-A-Sketch.ipynb>
;)

Here are a few notebooks
<http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092>
demonstrating the graph widget, including an overview
<http://nbviewer.ipython.org/gist/chriddyp/9827cb5086cfab8da092>.

Finally, thanks a bunch to the IPython team for all the great work with
widgets!

Chris

-- 
Chris Parmer
514 571 5897
https://plot.ly/team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/432d3ade/attachment.html>

From edsrahn at gmail.com  Fri Jan 23 14:12:51 2015
From: edsrahn at gmail.com (Ed Rahn)
Date: Fri, 23 Jan 2015 14:12:51 -0500
Subject: [IPython-dev] setting up public ipython notebook server
Message-ID: <CADFkHPpFpMggaPowcGbDTONGkJfJWYSXH4+CSLY0Mi_3FohS7w@mail.gmail.com>

When I follow the instructions at
http://ipython.org/ipython-doc/dev/notebook/public_server.html
I get:
Exception in callback (<socket._socketobject object at 0x7f422f0baf30>,
<function null_wrapper at 0x7f422f07b578>)
    Traceback (most recent call last):
      File
"/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/ioloop.py",
line 837, in start
        handler_func(fd_obj, events)
      File
"/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/stack_context.py",
line 275, in null_wrapper
        return fn(*args, **kwargs)
      File
"/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/netutil.py",
line 220, in accept_handler
        callback(connection, address)
      File
"/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/tcpserver.py",
line 225, in _handle_connection
        do_handshake_on_connect=False)
      File
"/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/netutil.py",
line 453, in ssl_wrap_socket
        context = ssl_options_to_context(ssl_options)
      File
"/home/ed/equalog/venv/local/lib/python2.7/site-packages/tornado/netutil.py",
line 430, in ssl_options_to_context
        context.load_cert_chain(ssl_options['certfile'],
ssl_options.get('keyfile', None))
    TypeError: coercing to Unicode: need string or buffer, NoneType found

I try setting keyfile but then I get ssl errors during requests.

Ed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/39c8f831/attachment.html>

From jmhanly at me.com  Fri Jan 23 14:14:36 2015
From: jmhanly at me.com (John Hanly)
Date: Fri, 23 Jan 2015 14:14:36 -0500
Subject: [IPython-dev] Plotly Interactive Graph Widget
In-Reply-To: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com>
References: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com>
Message-ID: <3CC5A1CA-459D-4998-8EF6-50834129B82F@me.com>

wonderful!!  thanks.

Go Nats!
jmhanly at me.com




On Jan 23, 2015, at 2:03 PM, Chris P <chris at plot.ly> wrote:

> Hey IPython Community --
> 
> I wanted to share with you Plotly's new IPython graph widget. A couple GIFs:
> 
> Contour Plot Explorer - The bottom graph updates with the elevation and distance of the points that are clicked on in the top-graph.
> 
> (Link to GIF: http://i.imgur.com/2Wbbo6l.gif)
> 
> 
> Dataframe Explorer of NYC 311 calls: Zooming and panning in the top time-series graph updates the bottom graph. Updating the search, dropdown, and slider values perform Pandas calls and Plotly restyle commands.
> 
> 
> (Link to GIF: http://i.imgur.com/h5GLIHc.gif)
> 
> 
> 
> This involved a couple of neat interactions with the Widget framework:
> - Events from our JS graphing library (hover, click, zoom) get passed to the Python kernel. Python callbacks can be assigned with e.g. GraphWidget.on_click(callback)
> 
> - All the functions from our JS graphing lib are exposed in the Python GraphWidget: restyle, add_traces, delete_traces, reorder_traces, relayout
> 
> - We're embedded these graphs as IFrames and all communication from the IPython JS client happens through postMessage. Since the GraphWidget is initialized with the unique URL of the graph, it's possible to swap graphs in and out of IPython widgets. For example, that Contour Plot Explorer will work with any Contour Plot, the user just has to replace contour_plot = GraphWidget('https://plot.ly/~bronsolo/63') with a URL of their own contour plot.
> 
> The widget code is here: https://github.com/plotly/python-api/tree/master/plotly/widgets. Any feedback or suggestions is greatly appreciated!
> 
> Some comments from the development POV: 
> - JS tracebacks were really difficult to debug. 
> - For easy use by the user, I decided to inject the JS dynamically on import. Adding //@ source to the end of the injected JS made the JS code appear in Chrome's dev tools. Without //@ source, the dynamic JS was lost in a Chrome VM.
> - I'm using Unicode(sync=True) to pass messages back and forth, but really I felt like I should just be using the underlying comm interface instead of the MVC framework. But, I had trouble finding good documentation on that interface.
> - Looking forward to a "knob" widget fora ipython etch-a-sketch ;) 
> 
> Here are a few notebooks demonstrating the graph widget, including an overview. 
> 
> Finally, thanks a bunch to the IPython team for all the great work with widgets!
> 
> Chris
> 
> -- 
> Chris Parmer
> 514 571 5897
> https://plot.ly/team
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150123/753088bb/attachment.html>

From ssanderson at quantopian.com  Fri Jan 23 14:38:53 2015
From: ssanderson at quantopian.com (ssanderson)
Date: Fri, 23 Jan 2015 11:38:53 -0800 (PST)
Subject: [IPython-dev] setting up public ipython notebook server
In-Reply-To: <CADFkHPpFpMggaPowcGbDTONGkJfJWYSXH4+CSLY0Mi_3FohS7w@mail.gmail.com>
References: <CADFkHPpFpMggaPowcGbDTONGkJfJWYSXH4+CSLY0Mi_3FohS7w@mail.gmail.com>
Message-ID: <1422041933630-5084131.post@n6.nabble.com>

Hi Ed,

I believe the default instructions you linked create a .pem file that
contains both an SSL cert and a private key together in a single file.  You
can pass the path to that file via --certfile, and I believe the underlying
Tornado server does the right thing.

For the SSL errors you're seeing when you make request, are you connecting
the server over HTTPS?  It's expected that you'd get errors when connecting
over HTTP.

- Scott



--
View this message in context: http://python.6.x6.nabble.com/setting-up-public-ipython-notebook-server-tp5084129p5084131.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From bussonniermatthias at gmail.com  Sat Jan 24 16:02:12 2015
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sat, 24 Jan 2015 22:02:12 +0100
Subject: [IPython-dev] Plotly Interactive Graph Widget
In-Reply-To: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com>
References: <CADQk6vh9k=4PVYH1MBT47h=nULzntAnhSv6TSe_iHu5sQ0gSLQ@mail.gmail.com>
Message-ID: <AF48F7A9-835F-4013-8D06-5E6CF910D42E@gmail.com>

Thanks a lot for the announce, 

We are really happy to see what you have build with widgets ! 
Please also let us know any other feedback you have. 

We are converging on IPython 3.0, and once the beta is released we will try to 
improve documentation. 

If you have any blog post that describe some of the workaround you did, or even want to 
add things to our docs, please feel free to do so. 

Thanks a lot for this new release of Plot.ly ! 

Cheers, 
-- 
Matthias



Le 23 janv. 2015 ? 20:03, Chris P <chris at plot.ly> a ?crit :

> Hey IPython Community --
> 
> I wanted to share with you Plotly's new IPython graph widget. A couple GIFs:
> 
> Contour Plot Explorer - The bottom graph updates with the elevation and distance of the points that are clicked on in the top-graph.
> 
> (Link to GIF: http://i.imgur.com/2Wbbo6l.gif)
> 
> 
> Dataframe Explorer of NYC 311 calls: Zooming and panning in the top time-series graph updates the bottom graph. Updating the search, dropdown, and slider values perform Pandas calls and Plotly restyle commands.
> 
> 
> (Link to GIF: http://i.imgur.com/h5GLIHc.gif)
> 
> 
> 
> This involved a couple of neat interactions with the Widget framework:
> - Events from our JS graphing library (hover, click, zoom) get passed to the Python kernel. Python callbacks can be assigned with e.g. GraphWidget.on_click(callback)
> 
> - All the functions from our JS graphing lib are exposed in the Python GraphWidget: restyle, add_traces, delete_traces, reorder_traces, relayout
> 
> - We're embedded these graphs as IFrames and all communication from the IPython JS client happens through postMessage. Since the GraphWidget is initialized with the unique URL of the graph, it's possible to swap graphs in and out of IPython widgets. For example, that Contour Plot Explorer will work with any Contour Plot, the user just has to replace contour_plot = GraphWidget('https://plot.ly/~bronsolo/63') with a URL of their own contour plot.
> 
> The widget code is here: https://github.com/plotly/python-api/tree/master/plotly/widgets. Any feedback or suggestions is greatly appreciated!
> 
> Some comments from the development POV: 
> - JS tracebacks were really difficult to debug. 
> - For easy use by the user, I decided to inject the JS dynamically on import. Adding //@ source to the end of the injected JS made the JS code appear in Chrome's dev tools. Without //@ source, the dynamic JS was lost in a Chrome VM.
> - I'm using Unicode(sync=True) to pass messages back and forth, but really I felt like I should just be using the underlying comm interface instead of the MVC framework. But, I had trouble finding good documentation on that interface.
> - Looking forward to a "knob" widget fora ipython etch-a-sketch ;) 
> 
> Here are a few notebooks demonstrating the graph widget, including an overview. 
> 
> Finally, thanks a bunch to the IPython team for all the great work with widgets!
> 
> Chris
> 
> -- 
> Chris Parmer
> 514 571 5897
> https://plot.ly/team
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150124/c2d4cdb4/attachment.html>

From zaki.mughal at gmail.com  Sun Jan 25 13:26:42 2015
From: zaki.mughal at gmail.com (Zakariyya Mughal)
Date: Sun, 25 Jan 2015 12:26:42 -0600
Subject: [IPython-dev] Execution semantics and how to set them
Message-ID: <20150125182642.GA25051@quadra>

Hello,

I'm a language kernel author  and I'm looking at the execution semantics
document[^1] where I see a reference to the "single" execution mode.

It says that the code
 
    for i in range(10):
        i**2
 
running in "single" mode would display 10 lines.

I looked through the IPython source and I see that the execution
semantics are set as an argument for `run_ast_nodes`[^2], but I see no
way that the given for-loop would give multiple outputs unless the
`interactivity` parameter is set to `last` or `all`.

Is there a way to activate this execution mode through a front-end and
pass that information to the back-end using the messaging protocol?

Or should I just implement the logic at the end of the document ("The
code field is split into individual blocks...") in my language back-end?

Cheers,
- Zaki Mughal

[^1]: Execution semantics spec <http://ipython.org/ipython-doc/dev/development/execution.html#execution-semantics>

[^2]: run_ast_nodes() <https://github.com/ipython/ipython/blob/d0fbe137/IPython/core/interactiveshell.py#L2919>


From takowl at gmail.com  Sun Jan 25 13:57:01 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 25 Jan 2015 10:57:01 -0800
Subject: [IPython-dev] Execution semantics and how to set them
In-Reply-To: <20150125182642.GA25051@quadra>
References: <20150125182642.GA25051@quadra>
Message-ID: <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com>

Hi Zaki,

On 25 January 2015 at 10:26, Zakariyya Mughal <zaki.mughal at gmail.com> wrote:

>
> I'm a language kernel author  and I'm looking at the execution semantics
> document[^1] where I see a reference to the "single" execution mode.
>
> It says that the code
>
>     for i in range(10):
>         i**2
>
> running in "single" mode would display 10 lines.
>
> I looked through the IPython source and I see that the execution
> semantics are set as an argument for `run_ast_nodes`[^2], but I see no
> way that the given for-loop would give multiple outputs unless the
> `interactivity` parameter is set to `last` or `all`.


First, let me emphasise that all of this is only relevant for executing
Python code. If you're building a kernel for another language, you should
ignore all of this. That documentation page is only about IPython's own
kernel.

The three modes described on that page (exec, eval, single) are compilation
modes of Python itself, passed as arguments to the built-in compile()
function.

The parameter passed to our run_ast_nodes() method (which can be all, none,
last or last_expr) controls which of a list of AST nodes are compiled in
'single' mode. Our default of last_expr is chosen specifically so that the
for loop example above does not produce output, because we think that's
more likely to be annoying than useful. There is no way to control this
from the frontend, and I don't think we have any intention to add such a
feature.

For your own kernel, you should implement whatever display of output makes
sense given the language and the likely users. For Python, we decided that
it makes sense to automatically display the result of the last statement if
that statement is an expression, and implemented that using the details
described above. For R, I implemented similar behaviour based on the
evaluate package by creating an output_handler with a value= parameter.

Which language are you working on a kernel for?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150125/2c39eb42/attachment.html>

From bussonniermatthias at gmail.com  Sun Jan 25 14:06:39 2015
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 25 Jan 2015 20:06:39 +0100
Subject: [IPython-dev] Execution semantics and how to set them
In-Reply-To: <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com>
References: <20150125182642.GA25051@quadra>
	<CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com>
Message-ID: <56F40087-43D3-4290-A681-568E072E1D66@gmail.com>


Le 25 janv. 2015 ? 19:57, Thomas Kluyver a ?crit :
> First, let me emphasise that all of this is only relevant for executing Python code. If you're building a kernel for another language, you should ignore all of this. That documentation page is only about IPython's own kernel.
> 
> The three modes described on that page (exec, eval, single) are compilation modes of Python itself, passed as arguments to the built-in compile() function.
> 
> The parameter passed to our run_ast_nodes() method (which can be all, none, last or last_expr) controls which of a list of AST nodes are compiled in 'single' mode. Our default of last_expr is chosen specifically so that the for loop example above does not produce output, because we think that's more likely to be annoying than useful. There is no way to control this from the frontend, and I don't think we have any intention to add such a feature.
> 
> For your own kernel, you should implement whatever display of output makes sense given the language and the likely users. For Python, we decided that it makes sense to automatically display the result of the last statement if that statement is an expression, and implemented that using the details described above. For R, I implemented similar behaviour based on the evaluate package by creating an output_handler with a value= parameter.
> 
> Which language are you working on a kernel for?

Shot in the dark, helped by google IPerl: 

https://github.com/zmughal/p5-Devel-IPerl

-- 
M

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150125/c844670a/attachment.html>

From zaki.mughal at gmail.com  Sun Jan 25 14:16:59 2015
From: zaki.mughal at gmail.com (Zakariyya Mughal)
Date: Sun, 25 Jan 2015 13:16:59 -0600
Subject: [IPython-dev] Execution semantics and how to set them
In-Reply-To: <CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com>
References: <20150125182642.GA25051@quadra>
	<CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com>
Message-ID: <20150125191658.GB25051@quadra>

On 2015-01-25 at 10:57:01 -0800, Thomas Kluyver wrote:
> Hi Zaki,
> 
> On 25 January 2015 at 10:26, Zakariyya Mughal <zaki.mughal at gmail.com> wrote:
> 
> >
> > I'm a language kernel author  and I'm looking at the execution semantics
> > document[^1] where I see a reference to the "single" execution mode.
> >
> > It says that the code
> >
> >     for i in range(10):
> >         i**2
> >
> > running in "single" mode would display 10 lines.
> >
> > I looked through the IPython source and I see that the execution
> > semantics are set as an argument for `run_ast_nodes`[^2], but I see no
> > way that the given for-loop would give multiple outputs unless the
> > `interactivity` parameter is set to `last` or `all`.
> 
> 
> First, let me emphasise that all of this is only relevant for executing
> Python code. If you're building a kernel for another language, you should
> ignore all of this. That documentation page is only about IPython's own
> kernel.
> 
> The three modes described on that page (exec, eval, single) are compilation
> modes of Python itself, passed as arguments to the built-in compile()
> function.
> 
> The parameter passed to our run_ast_nodes() method (which can be all, none,
> last or last_expr) controls which of a list of AST nodes are compiled in
> 'single' mode. Our default of last_expr is chosen specifically so that the
> for loop example above does not produce output, because we think that's
> more likely to be annoying than useful. There is no way to control this
> from the frontend, and I don't think we have any intention to add such a
> feature.
> 
> For your own kernel, you should implement whatever display of output makes
> sense given the language and the likely users. For Python, we decided that
> it makes sense to automatically display the result of the last statement if
> that statement is an expression, and implemented that using the details
> described above. For R, I implemented similar behaviour based on the
> evaluate package by creating an output_handler with a value= parameter.

I see. It's easy enough to just call print if a user wants that.

> Which language are you working on a kernel for?

I wrote one for Perl <https://github.com/zmughal/p5-Devel-IPerl>.

I released it back in August and I've been meaning to write a blog post
and announce it more widely, but I first wanted to make sure it
installed everywhere.

Cheers,
- Zaki Mughal

> 
> Thomas

> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Sun Jan 25 14:31:28 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 25 Jan 2015 11:31:28 -0800
Subject: [IPython-dev] Execution semantics and how to set them
In-Reply-To: <20150125191658.GB25051@quadra>
References: <20150125182642.GA25051@quadra>
	<CAOvn4qi5Ud7DrtPBXLKKBYZYOGXd-6uAgqVPjLuihhiTFbCSKQ@mail.gmail.com>
	<20150125191658.GB25051@quadra>
Message-ID: <CAOvn4qhyS6Yh8zTn8VE6_DH-N2F24yi1OykY7gEBi_xH0MaEWw@mail.gmail.com>

On 25 January 2015 at 11:16, Zakariyya Mughal <zaki.mughal at gmail.com> wrote:

> I wrote one for Perl <https://github.com/zmughal/p5-Devel-IPerl>.
>


Great! I see it's already been added to the list of available kernels, as
well.
https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages

Feel free to ask if you have any other questions we can help with.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150125/50e611a0/attachment.html>

From markbak at gmail.com  Mon Jan 26 11:17:27 2015
From: markbak at gmail.com (Mark Bakker)
Date: Mon, 26 Jan 2015 17:17:27 +0100
Subject: [IPython-dev] ginput plans for IPython Notebook with inline figures
Message-ID: <CAEX=yabBw-Uj78hki=-6EFmUFh8zhrw7zMox5vOhTuUMuPvchQ@mail.gmail.com>

Dear IPython List,

Are there any plans for getting ginput to work when figures are inline in a
Notebook?

It would be so cool to have that working together with the other widget
machinery.

Or is this something the matplotlib developers should pick up?

Thanks,

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150126/edc144ed/attachment.html>

From takowl at gmail.com  Mon Jan 26 12:47:03 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 26 Jan 2015 09:47:03 -0800
Subject: [IPython-dev] ginput plans for IPython Notebook with inline
	figures
In-Reply-To: <CAEX=yabBw-Uj78hki=-6EFmUFh8zhrw7zMox5vOhTuUMuPvchQ@mail.gmail.com>
References: <CAEX=yabBw-Uj78hki=-6EFmUFh8zhrw7zMox5vOhTuUMuPvchQ@mail.gmail.com>
Message-ID: <CAOvn4qizD2-tydoWhk-BTM5LOcOu7Sp_-c7TxsD7wCZ7K-gsiA@mail.gmail.com>

On 26 January 2015 at 08:17, Mark Bakker <markbak at gmail.com> wrote:

> Are there any plans for getting ginput to work when figures are inline in
> a Notebook?


Do you mean something equivalent to the Matlab ginput() function?
http://www.mathworks.com/help/matlab/ref/ginput.html

I'd say that looks like an issue for Matplotlib. I wouldn't be surprised if
something similar is already possible using the nbagg backend.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150126/8a2c4430/attachment.html>

From markbak at gmail.com  Mon Jan 26 14:41:56 2015
From: markbak at gmail.com (Mark Bakker)
Date: Mon, 26 Jan 2015 20:41:56 +0100
Subject: [IPython-dev] ginput plans for IPython Notebook with inline
	figures
Message-ID: <CAEX=yaZQX5E_xZOOA=ME9t=NvYBzgQtpojsBRXqaHj9iaK+wTQ@mail.gmail.com>

Unfortunately, ginput doesn't work yet in the nbagg backend.
I'll mail the matplotlib list.
Thanks,
Mark

On 26 January 2015 at 08:17, Mark Bakker <markbak at gmail.com> wrote:

> Are there any plans for getting ginput to work when figures are inline in
> a Notebook?


Do you mean something equivalent to the Matlab ginput() function?
http://www.mathworks.com/help/matlab/ref/ginput.html

I'd say that looks like an issue for Matplotlib. I wouldn't be surprised if
something similar is already possible using the nbagg backend.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150126/0c901d13/attachment.html>

From maidos93 at laposte.net  Wed Jan 28 07:40:25 2015
From: maidos93 at laposte.net (thwiouz)
Date: Wed, 28 Jan 2015 04:40:25 -0800 (PST)
Subject: [IPython-dev] IPython parallel and objects
Message-ID: <1422448825555-5084497.post@n6.nabble.com>

Hi guys,

I'd like to parallelize the following loop:

m1 = np.arange(0, 10, 100)
m2 = np.arange(0, 10, 100)

for m_ in m1:
    for n_2 in m2:
        tmp = optimizer.compute_expectation([m_, n_])
        res.append(tmp)

with type(optimizer) some class that I've defined. How could I handle it? 

I tried to do the same as in
http://stackoverflow.com/questions/9363118/parallelise-nested-for-loop-in-ipython
so I created a lambda function f

f = lambda x, y: optimizer.compute_expectation([x, y])

but it's not working with a load balanced view, saying that optimizer is not
defined.

Could someone help?

Thanks,




--
View this message in context: http://python.6.x6.nabble.com/IPython-parallel-and-objects-tp5084497.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From john at omernik.com  Wed Jan 28 10:08:14 2015
From: john at omernik.com (John Omernik)
Date: Wed, 28 Jan 2015 09:08:14 -0600
Subject: [IPython-dev] Display Function from with in a Function
Message-ID: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com>

Greetings all, I am using iPython notebook, I can do this:

from IPython.display import display, HTML

h = "<H1>Oh My </H1>"
HTML(h)

and get the expected result

But if I do this:

from IPython.display import display, HTML

def runme(html):
    HTML(html)


h = "<H1>Oh My </H1>
runme(h)

I get no results.

I've noticed this is true for other displays as well, how can I add
displays from withi functions?

Thanks!

John


From cyrille.rossant at gmail.com  Wed Jan 28 10:17:11 2015
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Wed, 28 Jan 2015 16:17:11 +0100
Subject: [IPython-dev] Display Function from with in a Function
In-Reply-To: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com>
References: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com>
Message-ID: <CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com>

HTML(...) will only display something if it's the last statement of a
code cell. This is because IPython always displays the result of the
last result of a cell (unless you terminate the line with a semicolon
';').

A more reliable way of displaying stuff is to use display() or
display_html(). This will work from within functions. See
http://ipython.org/ipython-doc/dev/api/generated/IPython.display.html
for more details.

2015-01-28 16:08 GMT+01:00 John Omernik <john at omernik.com>:
> Greetings all, I am using iPython notebook, I can do this:
>
> from IPython.display import display, HTML
>
> h = "<H1>Oh My </H1>"
> HTML(h)
>
> and get the expected result
>
> But if I do this:
>
> from IPython.display import display, HTML
>
> def runme(html):
>     HTML(html)
>
>
> h = "<H1>Oh My </H1>
> runme(h)
>
> I get no results.
>
> I've noticed this is true for other displays as well, how can I add
> displays from withi functions?
>
> Thanks!
>
> John
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From moritz.beber at gmail.com  Wed Jan 28 10:18:29 2015
From: moritz.beber at gmail.com (Moritz Beber)
Date: Wed, 28 Jan 2015 16:18:29 +0100
Subject: [IPython-dev] IPython parallel and objects
In-Reply-To: <1422448825555-5084497.post@n6.nabble.com>
References: <1422448825555-5084497.post@n6.nabble.com>
Message-ID: <CAFOFTpSke6Oiwd3p4KOMPdbPjrGPD4qNHxEAZcoCG-79Q4Fr=g@mail.gmail.com>

Hi,

On Wed, Jan 28, 2015 at 1:40 PM, thwiouz <maidos93 at laposte.net> wrote:

> Hi guys,
>
> I'd like to parallelize the following loop:
>
> m1 = np.arange(0, 10, 100)
> m2 = np.arange(0, 10, 100)
>
> for m_ in m1:
>     for n_2 in m2:
>         tmp = optimizer.compute_expectation([m_, n_])
>         res.append(tmp)
>
> with type(optimizer) some class that I've defined. How could I handle it?
>
> I tried to do the same as in
>
> http://stackoverflow.com/questions/9363118/parallelise-nested-for-loop-in-ipython
> so I created a lambda function f
>
> f = lambda x, y: optimizer.compute_expectation([x, y])
>
> but it's not working with a load balanced view, saying that optimizer is
> not
> defined.
>

IPython only automatically transmits the function definition used in a
`map` call or similar. Your optimizer is inside your lambda function and
thus unknown on the remote kernels. In order to overcome this, take a look
here:

http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#remote-function-decorators

and here:

http://ipython.org/ipython-doc/2/parallel/parallel_task.html#dependencies

this could be helpful, too:

http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#moving-python-objects-around


HTH,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150128/91860336/attachment.html>

From john at omernik.com  Wed Jan 28 10:34:26 2015
From: john at omernik.com (John Omernik)
Date: Wed, 28 Jan 2015 09:34:26 -0600
Subject: [IPython-dev] Display Function from with in a Function
In-Reply-To: <CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com>
References: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com>
	<CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com>
Message-ID: <CAKOFcwr4KPj99J51Y0Q8+GqwynuCiLDcJS9i=FzKUYQbd7TTdg@mail.gmail.com>

So I tried this:

from IPython.display import display, display_html


html = "<H1>OH MY </H1>"
def runme(html):
   display_html(html)

runme(html)

# This produced no results/output Then I tried
def runme(html):
   display(html)

runme(html)

# This printed: '<H1> OH MY </H1>'

Which isn't what HTML(html) displayed (it displayed it rendered).  So
I had a hunch...

h = display_html(html)

type(h)
NoneType

so then in my function I did this:

def runme(html):
    h = HTML(html)
    display(h)
    print "did it work?"


and then runme(html) produced the expected Heading 1 OH MY and the
printed statement after.... thanks for the pointer in the right
direction!

John



On Wed, Jan 28, 2015 at 9:17 AM, Cyrille Rossant
<cyrille.rossant at gmail.com> wrote:
> HTML(...) will only display something if it's the last statement of a
> code cell. This is because IPython always displays the result of the
> last result of a cell (unless you terminate the line with a semicolon
> ';').
>
> A more reliable way of displaying stuff is to use display() or
> display_html(). This will work from within functions. See
> http://ipython.org/ipython-doc/dev/api/generated/IPython.display.html
> for more details.
>
> 2015-01-28 16:08 GMT+01:00 John Omernik <john at omernik.com>:
>> Greetings all, I am using iPython notebook, I can do this:
>>
>> from IPython.display import display, HTML
>>
>> h = "<H1>Oh My </H1>"
>> HTML(h)
>>
>> and get the expected result
>>
>> But if I do this:
>>
>> from IPython.display import display, HTML
>>
>> def runme(html):
>>     HTML(html)
>>
>>
>> h = "<H1>Oh My </H1>
>> runme(h)
>>
>> I get no results.
>>
>> I've noticed this is true for other displays as well, how can I add
>> displays from withi functions?
>>
>> Thanks!
>>
>> John
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From cyrille.rossant at gmail.com  Wed Jan 28 10:38:44 2015
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Wed, 28 Jan 2015 16:38:44 +0100
Subject: [IPython-dev] Display Function from with in a Function
In-Reply-To: <CAKOFcwr4KPj99J51Y0Q8+GqwynuCiLDcJS9i=FzKUYQbd7TTdg@mail.gmail.com>
References: <CAKOFcwrkU4TG-KUHV=4JY7GJDc0d01An=0hwZjsgLY6vo32tMw@mail.gmail.com>
	<CA+-1RQQ3PjM=0RJ8yOGj8RDH2-B5cPUALGFN91NjzpkE+uuGPg@mail.gmail.com>
	<CAKOFcwr4KPj99J51Y0Q8+GqwynuCiLDcJS9i=FzKUYQbd7TTdg@mail.gmail.com>
Message-ID: <CA+-1RQThvE29NQLSNtREUiQWLdoFanTwX1PystNf6D1nGp+TTQ@mail.gmail.com>

You're welcome! Also have a look at the "raw=True" option to 'display()'.

2015-01-28 16:34 GMT+01:00 John Omernik <john at omernik.com>:
> So I tried this:
>
> from IPython.display import display, display_html
>
>
> html = "<H1>OH MY </H1>"
> def runme(html):
>    display_html(html)
>
> runme(html)
>
> # This produced no results/output Then I tried
> def runme(html):
>    display(html)
>
> runme(html)
>
> # This printed: '<H1> OH MY </H1>'
>
> Which isn't what HTML(html) displayed (it displayed it rendered).  So
> I had a hunch...
>
> h = display_html(html)
>
> type(h)
> NoneType
>
> so then in my function I did this:
>
> def runme(html):
>     h = HTML(html)
>     display(h)
>     print "did it work?"
>
>
> and then runme(html) produced the expected Heading 1 OH MY and the
> printed statement after.... thanks for the pointer in the right
> direction!
>
> John
>
>
>
> On Wed, Jan 28, 2015 at 9:17 AM, Cyrille Rossant
> <cyrille.rossant at gmail.com> wrote:
>> HTML(...) will only display something if it's the last statement of a
>> code cell. This is because IPython always displays the result of the
>> last result of a cell (unless you terminate the line with a semicolon
>> ';').
>>
>> A more reliable way of displaying stuff is to use display() or
>> display_html(). This will work from within functions. See
>> http://ipython.org/ipython-doc/dev/api/generated/IPython.display.html
>> for more details.
>>
>> 2015-01-28 16:08 GMT+01:00 John Omernik <john at omernik.com>:
>>> Greetings all, I am using iPython notebook, I can do this:
>>>
>>> from IPython.display import display, HTML
>>>
>>> h = "<H1>Oh My </H1>"
>>> HTML(h)
>>>
>>> and get the expected result
>>>
>>> But if I do this:
>>>
>>> from IPython.display import display, HTML
>>>
>>> def runme(html):
>>>     HTML(html)
>>>
>>>
>>> h = "<H1>Oh My </H1>
>>> runme(h)
>>>
>>> I get no results.
>>>
>>> I've noticed this is true for other displays as well, how can I add
>>> displays from withi functions?
>>>
>>> Thanks!
>>>
>>> John
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ctaf42 at gmail.com  Thu Jan 29 11:57:45 2015
From: ctaf42 at gmail.com (cedric gestes)
Date: Thu, 29 Jan 2015 08:57:45 -0800 (PST)
Subject: [IPython-dev] Dockable widgets for IPython
Message-ID: <1422550665175-5084622.post@n6.nabble.com>

I hacked a prototype of dockable widgets using DockSpawn.

see
http://www.hostingpics.net/viewer.php?id=990121201501291741191600x900scrot.png
for a screenshot.

Before going further I would like to take your advice on the subject. 

Is that something wanted in IPython?

We can imagine having cells and widgets in docks.

The prototype is hackish and just a proof of concept.

You can find it github on the branch dockspawn of:
github.com/ctaf42/ipython and github.com/ctaf42/ipython-components

just checkout, run IPython and open a notebook like this one:
http://nbviewer.ipython.org/gist/ctaf42/e029c475b08bdfe43f6a

Cedric




--
View this message in context: http://python.6.x6.nabble.com/Dockable-widgets-for-IPython-tp5084622.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From jeanbigboute at gmail.com  Thu Jan 29 12:28:27 2015
From: jeanbigboute at gmail.com (Jean Bigboute)
Date: Thu, 29 Jan 2015 09:28:27 -0800
Subject: [IPython-dev] Two basic Notebook questions (Python focus)
Message-ID: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com>

1) I often go back and forth in cells, tweaking functions.  I'd like to be
able to show the current code for a function in a cell (not the pager) when
I'm not sure which function definition I last executed


%psource function
works nicely but the result goes in the pager, not the cell.

These two options work but require remembering things which gets tricky
when I use different computers:

import inspect
print inspect.getsource(function)

and the pager redefinition shown in
http://nbviewer.ipython.org/gist/minrk/7715212 has a fix which requires
imports

I am sure there must be a way to do this and I just haven't searched on the
correct terms...

2) Is it possible to paste images from the clipboard buffer into a cell
without first creating a separate image file and then linking to that file?

Thanks,

JBB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/92da9cb7/attachment.html>

From nick.bollweg at gmail.com  Thu Jan 29 13:32:03 2015
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Thu, 29 Jan 2015 13:32:03 -0500
Subject: [IPython-dev] Dockable widgets for IPython
In-Reply-To: <1422550665175-5084622.post@n6.nabble.com>
References: <1422550665175-5084622.post@n6.nabble.com>
Message-ID: <CACejjWw+vbe=OU9X-c2VaWHne8HJ36Kt-hJjHJ8Fdy+0OEw71w@mail.gmail.com>

cedric:

this looks very cool: as the "ipython is not a platform" party line
softens, ideas like this are important for pushing what "interactive
computing" can be like. My immediate concern would be in not knowing which
cell a particular widget came from. Additionally, from a purely stylistic
sense, try using bootstrap panels and fontawesome to make it flow more
seamlessly with the notebook UI.

I think several folks have been looking for this in the past, especially
with creating such controls at notebook load time:
http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-td5079080.html#a5079562

I'd say near-term, see if you can repackage this into an nbextension that
someone can optionally add at profile config/run-time. If it can be made
into pure javascript that stores its data in the notebook/cell metadata,
that lowers the barrier to entry... and makes it more feasible to use with
other kernels.

Otherwise, if you need the python part, there is the approach taken by
cite2c <https://github.com/takluyver/cite2c>. Specifically, allowing the
configuration of the web app (not the kernel) to include loading custom
extensions <https://github.com/takluyver/cite2c/blob/master/install.py#L11>.

i have been interested in alternate editable layouts for some time: of them
i think live_reveal <https://github.com/damianavila/live_reveal>, while
still a little fiddly to install, has the most potential for becoming
something that would make sense to ship with the notebook experience,
especially as it is cell-based, (really) front-end only and its metadata is
already integrated into nvconvert and (soon, hopefully!) nbviewer. If your
widgets could survive nbviewer (either with or without help from a custom
converter) that would be really cool... once static widgets are supported.

i have a bit of python widget code that will populate widgets on top of SVG
regions (as defined by layers in inkscape): this makes it easy to build up
full-screen capable dashboards without requiring a lot of CSS/JS knowledge.

SVGLayout(svg="dashboard.svg",
          children={"chart": SomeChartWidget()})

i want to refactor it into pure javascript as well.

keep the list posted!

On Thu, Jan 29, 2015 at 11:57 AM, cedric gestes <ctaf42 at gmail.com> wrote:

> I hacked a prototype of dockable widgets using DockSpawn.
>
> see
>
> http://www.hostingpics.net/viewer.php?id=990121201501291741191600x900scrot.png
> for a screenshot.
>
> Before going further I would like to take your advice on the subject.
>
> Is that something wanted in IPython?
>
> We can imagine having cells and widgets in docks.
>
> The prototype is hackish and just a proof of concept.
>
> You can find it github on the branch dockspawn of:
> github.com/ctaf42/ipython and github.com/ctaf42/ipython-components
>
> just checkout, run IPython and open a notebook like this one:
> http://nbviewer.ipython.org/gist/ctaf42/e029c475b08bdfe43f6a
>
> Cedric
>
>
>
>
> --
> View this message in context:
> http://python.6.x6.nabble.com/Dockable-widgets-for-IPython-tp5084622.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/517fdb2d/attachment.html>

From mjandrews.net at gmail.com  Thu Jan 29 18:59:49 2015
From: mjandrews.net at gmail.com (Mark Andrews)
Date: Thu, 29 Jan 2015 23:59:49 +0000
Subject: [IPython-dev] problems with ipython qtconsole --pylab on arch linux
Message-ID: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com>

After a recent upgrade on arch linux (where I have been using ipython for
years), I just encountered the following problem when I start ipython
(2.3.1) with "ipython2 qtconsole --pylab"

Using matplotlib backend: Qt5Agg

ERROR:tornado.application:Exception in callback <functools.partial object
at 0x7fcac006c838>

Traceback (most recent call last):

File "/usr/lib/python2.7/site-packages/tornado/ioloop.py", line 565, in
_run_callback

ret = callback()

File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line 275,
in null_wrapper

return fn(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/ipkernel.py",
line 262, in enter_eventloop

self.eventloop(self)

File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py",
line 71, in loop_qt5

return loop_qt4(kernel)

File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py",
line 60, in loop_qt4

kernel.app = get_app_qt4([" "])

File "/usr/lib/python2.7/site-packages/IPython/lib/guisupport.py", line
111, in get_app_qt4

from IPython.external.qt_for_kernel import QtGui

File "/usr/lib/python2.7/site-packages/IPython/external/qt_for_kernel.py",
line 84, in <module>

from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API

File "/usr/lib/python2.7/site-packages/IPython/external/qt.py", line 17, in
<module>

(QT_API, QT_API_PYSIDE, QT_API_PYQT))

RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyside', 'pyqt'


If I start with "ipython2 qtconsole" and then do

%matplotlib inline

import matplotlib.pyplot


then plotting works fine.


I am guessing that this is something that arises only with arch, and has
something to do with its packages for python Qt bindings.


Here are some of other relevant packages I am using

python2-matplotlib 1.4.2-3

python2-pyqt4 4.11.3-1
python2-pyqt5 5.4.0-2


It is not a major hassle but it would be good know what is causing this.


-m


ps. ipython2 is arch linux's name for ipython 2.x, while the name ipython
is for ipython 3.x.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/e759b1d0/attachment.html>

From takowl at gmail.com  Thu Jan 29 19:09:37 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 29 Jan 2015 16:09:37 -0800
Subject: [IPython-dev] problems with ipython qtconsole --pylab on arch
	linux
In-Reply-To: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com>
References: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com>
Message-ID: <CAOvn4qg4KJO76NmhSxG4ExfK6=4XnNNsM-Sri4p=_EO7fXcyuA@mail.gmail.com>

I'd hazard a guess that Arch has set matplotlib's default backend to
qt5agg, but that's not supported by the released version of IPython. 3.0
and 2.4 will support Qt 5.

In the meantime, you should be able to work around it by specifying another
backend, e.g. --pylab qt4 or --pylab inline.

Thomas

On 29 January 2015 at 15:59, Mark Andrews <mjandrews.net at gmail.com> wrote:

> After a recent upgrade on arch linux (where I have been using ipython for
> years), I just encountered the following problem when I start ipython
> (2.3.1) with "ipython2 qtconsole --pylab"
>
> Using matplotlib backend: Qt5Agg
>
> ERROR:tornado.application:Exception in callback <functools.partial object
> at 0x7fcac006c838>
>
> Traceback (most recent call last):
>
> File "/usr/lib/python2.7/site-packages/tornado/ioloop.py", line 565, in
> _run_callback
>
> ret = callback()
>
> File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line
> 275, in null_wrapper
>
> return fn(*args, **kwargs)
>
> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/ipkernel.py",
> line 262, in enter_eventloop
>
> self.eventloop(self)
>
> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py",
> line 71, in loop_qt5
>
> return loop_qt4(kernel)
>
> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py",
> line 60, in loop_qt4
>
> kernel.app = get_app_qt4([" "])
>
> File "/usr/lib/python2.7/site-packages/IPython/lib/guisupport.py", line
> 111, in get_app_qt4
>
> from IPython.external.qt_for_kernel import QtGui
>
> File "/usr/lib/python2.7/site-packages/IPython/external/qt_for_kernel.py",
> line 84, in <module>
>
> from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API
>
> File "/usr/lib/python2.7/site-packages/IPython/external/qt.py", line 17,
> in <module>
>
> (QT_API, QT_API_PYSIDE, QT_API_PYQT))
>
> RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyside', 'pyqt'
>
>
> If I start with "ipython2 qtconsole" and then do
>
> %matplotlib inline
>
> import matplotlib.pyplot
>
>
> then plotting works fine.
>
>
> I am guessing that this is something that arises only with arch, and has
> something to do with its packages for python Qt bindings.
>
>
> Here are some of other relevant packages I am using
>
> python2-matplotlib 1.4.2-3
>
> python2-pyqt4 4.11.3-1
> python2-pyqt5 5.4.0-2
>
>
> It is not a major hassle but it would be good know what is causing this.
>
>
> -m
>
>
> ps. ipython2 is arch linux's name for ipython 2.x, while the name ipython
> is for ipython 3.x.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/324cfc9e/attachment.html>

From mjandrews.net at gmail.com  Thu Jan 29 19:22:43 2015
From: mjandrews.net at gmail.com (Mark Andrews)
Date: Fri, 30 Jan 2015 00:22:43 +0000
Subject: [IPython-dev] problems with ipython qtconsole --pylab on arch
	linux
In-Reply-To: <CAOvn4qg4KJO76NmhSxG4ExfK6=4XnNNsM-Sri4p=_EO7fXcyuA@mail.gmail.com>
References: <CABj652bJQuykLd5kdoTb7NXqHzs8n=7bAL01moTi_LiZqF8XMA@mail.gmail.com>
	<CAOvn4qg4KJO76NmhSxG4ExfK6=4XnNNsM-Sri4p=_EO7fXcyuA@mail.gmail.com>
Message-ID: <CABj652ZrE8UVBEwf9UsrSKkboT06iuFR9Fg9iy318drbGOM+Pg@mail.gmail.com>

That seems like a reasonable hypothesis. I am not too sure how to check if
matplotlib's default backend to has been recently set to qt5agg, but I will
try.
In the meantime, I can confirm that
ipython2 qtconsole --pylab qt4
ipython2 qtconsole --pylab inline
work fine.
Thanks!
-m


On Fri, Jan 30, 2015 at 12:09 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> I'd hazard a guess that Arch has set matplotlib's default backend to
> qt5agg, but that's not supported by the released version of IPython. 3.0
> and 2.4 will support Qt 5.
>
> In the meantime, you should be able to work around it by specifying
> another backend, e.g. --pylab qt4 or --pylab inline.
>
> Thomas
>
> On 29 January 2015 at 15:59, Mark Andrews <mjandrews.net at gmail.com> wrote:
>
>> After a recent upgrade on arch linux (where I have been using ipython for
>> years), I just encountered the following problem when I start ipython
>> (2.3.1) with "ipython2 qtconsole --pylab"
>>
>> Using matplotlib backend: Qt5Agg
>>
>> ERROR:tornado.application:Exception in callback <functools.partial object
>> at 0x7fcac006c838>
>>
>> Traceback (most recent call last):
>>
>> File "/usr/lib/python2.7/site-packages/tornado/ioloop.py", line 565, in
>> _run_callback
>>
>> ret = callback()
>>
>> File "/usr/lib/python2.7/site-packages/tornado/stack_context.py", line
>> 275, in null_wrapper
>>
>> return fn(*args, **kwargs)
>>
>> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/ipkernel.py",
>> line 262, in enter_eventloop
>>
>> self.eventloop(self)
>>
>> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py",
>> line 71, in loop_qt5
>>
>> return loop_qt4(kernel)
>>
>> File "/usr/lib/python2.7/site-packages/IPython/kernel/zmq/eventloops.py",
>> line 60, in loop_qt4
>>
>> kernel.app = get_app_qt4([" "])
>>
>> File "/usr/lib/python2.7/site-packages/IPython/lib/guisupport.py", line
>> 111, in get_app_qt4
>>
>> from IPython.external.qt_for_kernel import QtGui
>>
>> File
>> "/usr/lib/python2.7/site-packages/IPython/external/qt_for_kernel.py", line
>> 84, in <module>
>>
>> from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API
>>
>> File "/usr/lib/python2.7/site-packages/IPython/external/qt.py", line 17,
>> in <module>
>>
>> (QT_API, QT_API_PYSIDE, QT_API_PYQT))
>>
>> RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyside', 'pyqt'
>>
>>
>> If I start with "ipython2 qtconsole" and then do
>>
>> %matplotlib inline
>>
>> import matplotlib.pyplot
>>
>>
>> then plotting works fine.
>>
>>
>> I am guessing that this is something that arises only with arch, and has
>> something to do with its packages for python Qt bindings.
>>
>>
>> Here are some of other relevant packages I am using
>>
>> python2-matplotlib 1.4.2-3
>>
>> python2-pyqt4 4.11.3-1
>> python2-pyqt5 5.4.0-2
>>
>>
>> It is not a major hassle but it would be good know what is causing this.
>>
>>
>> -m
>>
>>
>> ps. ipython2 is arch linux's name for ipython 2.x, while the name ipython
>> is for ipython 3.x.
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/1b7e0021/attachment.html>

From alexgarciac at gmail.com  Thu Jan 29 22:07:13 2015
From: alexgarciac at gmail.com (Alexander Garcia Castro)
Date: Thu, 29 Jan 2015 22:07:13 -0500
Subject: [IPython-dev] ipython in scholarly communication
Message-ID: <CALAe=OLFhnyzXH0svqkX9kov_vbT-7QDnhJt7+RziNf6AqRWyg@mail.gmail.com>

Dear all, Sepublica is particularly interested in getting submissions from
ipython usrs/developers. we are very interested in seeing new ways of
scholarly communication in action and we believe that ipython illustrates a
path to be followed.

SePublica 2015: Do show, do not tell!
=====================================

Workshop website: http://sepublica.info/
Event hashtags: #SePublica #ESWC2015

in conjunction with

ESWC 2015
The 12th European Semantic Web Conference
http://2015.eswc-conferences.org/

At SePublica we are interested in addressing the question, how is Semantic
Web technology being used as part of publication workflows.

Challenge us with a creative submission. Let us see and experience your
take on publications for the Web of data; we want to see NEW ideas in
action, INNOVATION that makes our lives easier. Is it about scholarly
communication? Then what problem are you solving? How are you making the
content interoperable? How is this different/better from traditional
approaches?


Important Dates
---------------
* Submission deadline: 2015-03-06
* Notifications: 2015-04-03
* Camera ready version: 2015-04-17


Submission
----------
Easychair: https://www.easychair.org/conferences/?conf=sepublica2015

Submissions will be peer reviewed. Accepted works have to be presented at
the workshop (requires registering for the ESWC conference and the
workshop). Every accepted submission will be published in the proceedings
of the workshop.

Submission Types
----------------
### Call for Submissions
Deadline: 2015-03-06. Authors may submit their content in whatever way they
consider is best for their purposes (e.g., HTML, RDFS/OWL, IPython, PDF, or
any publicly executable script, etc). Please feel free to contact us to
discuss your particular case if necessary. We will be happy to assist
authors and support them as much as we can. Submissions are format free,
GIT URLs are accepted just like OWL, XML etc. However, reviewers should not
have to install software in order to review the submissions. Datasets are
also accepted. In all cases everything must be publicly available. For
those of you submitting a PDF please use the LNCS format, 12 pages is the
limit ?including references. Please, note that (X)HTML(+RDFa) submissions
are also welcome as long as the layout complies with the LNCS style. For
example, authors may use the template provided at
https://github.com/csarven/linked-research

### Call for Polemics
Deadline: 2015-04-24. We would like to invite authors to send us a one page
manuscript, 20 lines max, describing their position with respect to new
technologies supporting the publication workflow: What are the most
pressing issues to be addressed? What is their position with respect to the
overall problem? What innovation is needed? etc. Polemics authors will have
only 5 minutes to present; the format of this session is sequential, after
each presentation the next follows with no questions in between. There will
be a discussion and summary of all the issues at the end of all the
polemics session.

Issues to be addressed
----------------------
* Should papers be semantically annotated?
* How would you search or query for research objects?
* How can research objects be reused?
* What is an executable semantic paper?
* How can we create interactive research documents to better communicate
and educate?
* Is there a format for semantic publishing or does it require a
combination of formats?
* How can research documents be persistent and archived? Which formats are
preferable and why?
* If we subscribe to science, free and open access to knowledge, what's the
purpose of the arrangement between conferences and publishers? Or can big
conferences manage the publication themselves?
* Is there visible change towards semantic authoring and publishing? If
not, what are some of the underlying challenges to get there?
* Assume a 3-year plan to transition the SW research community to
completely publish using SW technologies. Identify the milestones and
explain how it would happen.
* Backend use of semantics by publishers in book, news, and scholarly
publishing
* Publishing data-driven arguments for general news consumption or in
scholarly papers
* Historical changes in publishing since the advent of online publishing:
what is going right?

Organizing Committee
--------------------
* Phillip Lord
* Alexander Garcia Castro
* Sarven Capadisli
* Jodi Schneider

-- 
Alexander Garcia
http://www.alexandergarcia.name/
http://www.usefilm.com/photographer/75943.html
http://www.linkedin.com/in/alexgarciac
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/c9a9aa39/attachment.html>

From rgbkrk at gmail.com  Thu Jan 29 22:10:46 2015
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Thu, 29 Jan 2015 21:10:46 -0600
Subject: [IPython-dev] ipython/ipython Docker image switching to
	debian:jessie
Message-ID: <CA+tbMaWiHpytqyneOuZyVNPVM3ZxfBo__de7498-HE4xixq4Fw@mail.gmail.com>

In https://github.com/ipython/ipython/pull/7391 we're switching our base
Docker image over to debian:jessie.

Let us know if you need more time before we merge this, which will result
in triggering builds across all the ipython/* images on Docker Hub.

-- 
Kyle Kelley (@rgbkrk <https://twitter.com/rgbkrk>; http://lambdaops.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150129/749b9d54/attachment.html>

From bussonniermatthias at gmail.com  Fri Jan 30 05:28:58 2015
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 30 Jan 2015 11:28:58 +0100
Subject: [IPython-dev] Two basic Notebook questions (Python focus)
In-Reply-To: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com>
References: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com>
Message-ID: <81C8AF0D-F5AB-4404-A83D-7F82A58B0D73@gmail.com>


Le 29 janv. 2015 ? 18:28, Jean Bigboute <jeanbigboute at gmail.com> a ?crit :

> 1) I often go back and forth in cells, tweaking functions.  I'd like to be able to show the current code for a function in a cell (not the pager) when I'm not sure which function definition I last executed
> 
> 
> %psource function
> works nicely but the result goes in the pager, not the cell.
> 
> These two options work but require remembering things which gets tricky when I use different computers:
> 
> import inspect
> print inspect.getsource(function)
> 
> and the pager redefinition shown in
> http://nbviewer.ipython.org/gist/minrk/7715212 has a fix which requires imports
> 
> I am sure there must be a way to do this and I just haven't searched on the correct terms...

In IPython master there is an option to redirect all pager things to output. 
The plan it to make output detachable. 

try --InteractiveShell.display_page=True

see PR 7009. 


> 2) Is it possible to paste images from the clipboard buffer into a cell without first creating a separate image file and then linking to that file?

No, 

I would suggest looking at github.com/IPython-contrib for an extension to do that. 
There are some complication to do that though, in particular when "uploading" do you want to upload
to server or kernel ? 

Now that we have content API it should be easier to implement if it is not yet done. 
-- 
M

> 
> Thanks,
> 
> JBB
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/b4f16ff5/attachment.html>

From bussonniermatthias at gmail.com  Fri Jan 30 05:30:32 2015
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 30 Jan 2015 11:30:32 +0100
Subject: [IPython-dev] Dockable widgets for IPython
In-Reply-To: <CACejjWw+vbe=OU9X-c2VaWHne8HJ36Kt-hJjHJ8Fdy+0OEw71w@mail.gmail.com>
References: <1422550665175-5084622.post@n6.nabble.com>
	<CACejjWw+vbe=OU9X-c2VaWHne8HJ36Kt-hJjHJ8Fdy+0OEw71w@mail.gmail.com>
Message-ID: <DF2B4B6D-485D-496A-80DB-017C1D923A39@gmail.com>

Hi, 

To add to what Nicholas said, 
After IPython 3.0 we will split the widget in a separate repository. 
It will be easier to contribute/discuss than just before 3.0 release
or while widget are still tightly entangled with IPython code. 
-- 
M


Le 29 janv. 2015 ? 19:32, Nicholas Bollweg <nick.bollweg at gmail.com> a ?crit :

> cedric:
> 
> this looks very cool: as the "ipython is not a platform" party line softens, ideas like this are important for pushing what "interactive computing" can be like. My immediate concern would be in not knowing which cell a particular widget came from. Additionally, from a purely stylistic sense, try using bootstrap panels and fontawesome to make it flow more seamlessly with the notebook UI.
> 
> I think several folks have been looking for this in the past, especially with creating such controls at notebook load time:
> http://python.6.x6.nabble.com/How-To-create-persistent-interactive-UI-elements-outside-cells-td5079080.html#a5079562
> 
> I'd say near-term, see if you can repackage this into an nbextension that someone can optionally add at profile config/run-time. If it can be made into pure javascript that stores its data in the notebook/cell metadata, that lowers the barrier to entry... and makes it more feasible to use with other kernels.
> 
> Otherwise, if you need the python part, there is the approach taken by cite2c. Specifically, allowing the configuration of the web app (not the kernel) to include loading custom extensions.
> 
> i have been interested in alternate editable layouts for some time: of them i think live_reveal, while still a little fiddly to install, has the most potential for becoming something that would make sense to ship with the notebook experience, especially as it is cell-based, (really) front-end only and its metadata is already integrated into nvconvert and (soon, hopefully!) nbviewer. If your widgets could survive nbviewer (either with or without help from a custom converter) that would be really cool... once static widgets are supported.
> 
> i have a bit of python widget code that will populate widgets on top of SVG regions (as defined by layers in inkscape): this makes it easy to build up full-screen capable dashboards without requiring a lot of CSS/JS knowledge.
> 
> SVGLayout(svg="dashboard.svg",
>           children={"chart": SomeChartWidget()})
> 
> i want to refactor it into pure javascript as well.
> 
> keep the list posted!
> 
> On Thu, Jan 29, 2015 at 11:57 AM, cedric gestes <ctaf42 at gmail.com> wrote:
> I hacked a prototype of dockable widgets using DockSpawn.
> 
> see
> http://www.hostingpics.net/viewer.php?id=990121201501291741191600x900scrot.png
> for a screenshot.
> 
> Before going further I would like to take your advice on the subject.
> 
> Is that something wanted in IPython?
> 
> We can imagine having cells and widgets in docks.
> 
> The prototype is hackish and just a proof of concept.
> 
> You can find it github on the branch dockspawn of:
> github.com/ctaf42/ipython and github.com/ctaf42/ipython-components
> 
> just checkout, run IPython and open a notebook like this one:
> http://nbviewer.ipython.org/gist/ctaf42/e029c475b08bdfe43f6a
> 
> Cedric
> 
> 
> 
> 
> --
> View this message in context: http://python.6.x6.nabble.com/Dockable-widgets-for-IPython-tp5084622.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/152431be/attachment.html>

From doug.blank at gmail.com  Fri Jan 30 05:38:52 2015
From: doug.blank at gmail.com (Doug Blank)
Date: Fri, 30 Jan 2015 05:38:52 -0500
Subject: [IPython-dev] Two basic Notebook questions (Python focus)
In-Reply-To: <81C8AF0D-F5AB-4404-A83D-7F82A58B0D73@gmail.com>
References: <CAEen6ux7Fi4JNCg-uab2ePgs4g1J72XhhXAdcM4sGABTJscy5Q@mail.gmail.com>
	<81C8AF0D-F5AB-4404-A83D-7F82A58B0D73@gmail.com>
Message-ID: <CAAusYCjxv+C0kbWK-mwgOE0gv1a6byABPjb44KqbbBzAaMyWyg@mail.gmail.com>

Drag and drop images via new API is done by juhasch:

https://github.com/ipython-contrib/IPython-notebook-extensions/wiki/drag-and-drop

-Doug

On Fri, Jan 30, 2015 at 5:28 AM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

>
> Le 29 janv. 2015 ? 18:28, Jean Bigboute <jeanbigboute at gmail.com> a ?crit :
>
> 1) I often go back and forth in cells, tweaking functions.  I'd like to be
> able to show the current code for a function in a cell (not the pager) when
> I'm not sure which function definition I last executed
>
>
> %psource function
> works nicely but the result goes in the pager, not the cell.
>
> These two options work but require remembering things which gets tricky
> when I use different computers:
>
> import inspect
> print inspect.getsource(function)
>
> and the pager redefinition shown in
> http://nbviewer.ipython.org/gist/minrk/7715212 has a fix which requires
> imports
>
> I am sure there must be a way to do this and I just haven't searched on
> the correct terms...
>
>
> In IPython master there is an option to redirect all pager things to
> output.
> The plan it to make output detachable.
>
> try --InteractiveShell.display_page=True
>
> see PR 7009.
>
>
> 2) Is it possible to paste images from the clipboard buffer into a cell
> without first creating a separate image file and then linking to that file?
>
>
> No,
>
> I would suggest looking at github.com/IPython-contrib for an extension to
> do that.
> There are some complication to do that though, in particular when
> "uploading" do you want to upload
> to server or kernel ?
>
> Now that we have content API it should be easier to implement if it is not
> yet done.
> --
> M
>
>
> Thanks,
>
> JBB
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/e8996ec7/attachment.html>

From nick.bollweg at gmail.com  Fri Jan 30 14:32:28 2015
From: nick.bollweg at gmail.com (Nicholas Bollweg)
Date: Fri, 30 Jan 2015 14:32:28 -0500
Subject: [IPython-dev] ipython in scholarly communication
In-Reply-To: <CALAe=OLFhnyzXH0svqkX9kov_vbT-7QDnhJt7+RziNf6AqRWyg@mail.gmail.com>
References: <CALAe=OLFhnyzXH0svqkX9kov_vbT-7QDnhJt7+RziNf6AqRWyg@mail.gmail.com>
Message-ID: <CACejjWye5sHBFhusOtKc7dSL=kwJET0TGevXWrG1SYaEY3-_7Q@mail.gmail.com>

I'm pretty interested in this.

Here's the last discussion I recall about SW on this list:
http://python.6.x6.nabble.com/Notebook-format-quot-incompatible-quot-changes-td5076324.html#a5076589

We talked at one point about creating an IPEP for incrementally adding
linked data to various things across the ecosystem, perhaps this is a good
time to dust that off. Here is the IPEP that I never actually posted:
http://nbviewer.ipython.org/gist/bollwyvl/6edda34649ba4b5cbaf9

Forks, comments welcome!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/be2e4a62/attachment.html>

From jeanbigboute at gmail.com  Fri Jan 30 18:26:40 2015
From: jeanbigboute at gmail.com (Jean Bigboute)
Date: Fri, 30 Jan 2015 15:26:40 -0800
Subject: [IPython-dev] Two basic Notebook questions (Python focus)
Message-ID: <CAEen6uz_OfHFL4JfKHBTdeCYqHUVdnAM4G8rjF-wMhoB4T=mdA@mail.gmail.com>

Thanks for the suggestions on how to disable the pager and on the
options for pasting images. I am not savvy enough with IPython to run
from the master so I'll keep an eye out for the upcoming switch.

A couple of followup questions:

a) Listing the code of a function seems like a useful feature to have,
especially for relatively new/occasional Python programmers such as
myself.  Should I request it through the appropriate channels?

b) Is it possible to pass arguments to a magic function?
e.g.
import inspect
print inspect.getsource(function)

I am sure I can remember to import inspect.

I tried to create an alias for the print inspect.getsource(function)
but couldn't figure out how to pass the desired function name as an
argument

Thanks,

JBB


From benjaminrk at gmail.com  Fri Jan 30 18:30:52 2015
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 30 Jan 2015 15:30:52 -0800
Subject: [IPython-dev] IPython 2.4, 3.0b1
Message-ID: <CAHNn8BXm6xtXNv2o0jf5uxgYrp1WUAtsYWqWCh_XnVadx=qu_w@mail.gmail.com>

Hello,

It?s been a long time coming, but we have begun the release cycle for
IPython 3.0, with the first beta. You can take it for a spin with:

pip install --pre 'ipython[all]'

IPython 3.0 brings a lot of changes - optional terminals in the browser,
editing non-notebook text files, major notebook format and message spec
revisions, UI overhaul, etc. Take it for a spin, and let us know what you
find. Check out the release notes
<http://ipython.org/ipython-doc/dev/whatsnew/version3.html#release-3-0> for
more info. We still have a few known bugs and UI issues to address before
we get to 3.0 final, but we?ll get there soon.

We also released IPython 2.4 today, which adds to 2.x the ability to *read*
the new notebook format coming in IPython 3.0. 2.4 also fixes some
compatibility issues with PyQt5 and pygments 2.

With your help, 3.0 final will see the light of day before too long.

-MinRK, on behalf of the IPython team
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/25f06dd5/attachment.html>

From takowl at gmail.com  Fri Jan 30 18:48:59 2015
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 30 Jan 2015 15:48:59 -0800
Subject: [IPython-dev] Two basic Notebook questions (Python focus)
In-Reply-To: <CAEen6uz_OfHFL4JfKHBTdeCYqHUVdnAM4G8rjF-wMhoB4T=mdA@mail.gmail.com>
References: <CAEen6uz_OfHFL4JfKHBTdeCYqHUVdnAM4G8rjF-wMhoB4T=mdA@mail.gmail.com>
Message-ID: <CAOvn4qjuysh_1z1tCJK1YWOC53QCjZdnCEwudS8Co1G-s1uK=w@mail.gmail.com>

On 30 January 2015 at 15:26, Jean Bigboute <jeanbigboute at gmail.com> wrote:

> b) Is it possible to pass arguments to a magic function?
> e.g.
> import inspect
> print inspect.getsource(function)
>

This is probably easiest to do in a plain Python function. Create a file
~/.ipython/profile_default/startup/printsource.py (which IPython will
automatically run when it starts, and put in it:

import inspect
def source(func)
    print inspect.getsource(function)

Now you can just type 'source(f)' to see a function in your notebook. Be
aware, though, that if you share that notebook, other people won't have the
source() function defined. You could also define it at the top of the
notebook if you prefer.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150130/abd3b6cb/attachment.html>

From cpdiehl at gmail.com  Sat Jan 31 11:07:50 2015
From: cpdiehl at gmail.com (diehl)
Date: Sat, 31 Jan 2015 08:07:50 -0800 (PST)
Subject: [IPython-dev] Complex UI Examples with Widgets
Message-ID: <1422720470245-5084847.post@n6.nabble.com>

Hi All, 

I'm looking for IPython notebooks containing complex UIs built with the
widget framework. Just trying to get a sense of what is in the realm of the
possible, especially when it comes to composition and positioning of the
widgets in the notebook. Any examples you can refer me to would be greatly
appreciated!

Thanks, 
Chris



--
View this message in context: http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From hughesadam87 at gmail.com  Sat Jan 31 13:05:24 2015
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Sat, 31 Jan 2015 13:05:24 -0500
Subject: [IPython-dev] Complex UI Examples with Widgets
In-Reply-To: <1422720470245-5084847.post@n6.nabble.com>
References: <1422720470245-5084847.post@n6.nabble.com>
Message-ID: <CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com>

We have a fairly involved one.  The traitlets Model is in specgram.py and
the GUI/Widget elements are in specgui.py

https://github.com/hugadams/scikit-spectra/tree/master/skspec/interact/ipynbs

Here's a video of it in action:

http://hugadams.github.io/scikit-spectra/

On Sat, Jan 31, 2015 at 11:07 AM, diehl <cpdiehl at gmail.com> wrote:

> Hi All,
>
> I'm looking for IPython notebooks containing complex UIs built with the
> widget framework. Just trying to get a sense of what is in the realm of the
> possible, especially when it comes to composition and positioning of the
> widgets in the notebook. Any examples you can refer me to would be greatly
> appreciated!
>
> Thanks,
> Chris
>
>
>
> --
> View this message in context:
> http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150131/45d63d36/attachment.html>

From cpdiehl at gmail.com  Sat Jan 31 13:51:58 2015
From: cpdiehl at gmail.com (Chris Diehl)
Date: Sat, 31 Jan 2015 10:51:58 -0800
Subject: [IPython-dev] Complex UI Examples with Widgets
In-Reply-To: <CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com>
References: <1422720470245-5084847.post@n6.nabble.com>
	<CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com>
Message-ID: <CABYryNBVcsjieDsKE=S-u5cNGfbNeZwLT4qsf+Qe+yVBv17CtQ@mail.gmail.com>

This is a great example Adam. Thanks!

On Sat, Jan 31, 2015 at 10:05 AM, Adam Hughes <hughesadam87 at gmail.com>
wrote:

> We have a fairly involved one.  The traitlets Model is in specgram.py and
> the GUI/Widget elements are in specgui.py
>
>
> https://github.com/hugadams/scikit-spectra/tree/master/skspec/interact/ipynbs
>
> Here's a video of it in action:
>
> http://hugadams.github.io/scikit-spectra/
>
> On Sat, Jan 31, 2015 at 11:07 AM, diehl <cpdiehl at gmail.com> wrote:
>
>> Hi All,
>>
>> I'm looking for IPython notebooks containing complex UIs built with the
>> widget framework. Just trying to get a sense of what is in the realm of
>> the
>> possible, especially when it comes to composition and positioning of the
>> widgets in the notebook. Any examples you can refer me to would be greatly
>> appreciated!
>>
>> Thanks,
>> Chris
>>
>>
>>
>> --
>> View this message in context:
>> http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html
>> Sent from the IPython - Development mailing list archive at Nabble.com.
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150131/5190a1ee/attachment.html>

From hughesadam87 at gmail.com  Sat Jan 31 18:19:43 2015
From: hughesadam87 at gmail.com (Adam Hughes)
Date: Sat, 31 Jan 2015 18:19:43 -0500
Subject: [IPython-dev] Complex UI Examples with Widgets
In-Reply-To: <CABYryNBVcsjieDsKE=S-u5cNGfbNeZwLT4qsf+Qe+yVBv17CtQ@mail.gmail.com>
References: <1422720470245-5084847.post@n6.nabble.com>
	<CAMHV+dDkOAJahHX2Tfn4wGV-uh7u8bEig-QikgOnVC-WvdeQ8g@mail.gmail.com>
	<CABYryNBVcsjieDsKE=S-u5cNGfbNeZwLT4qsf+Qe+yVBv17CtQ@mail.gmail.com>
Message-ID: <CAMHV+dB-cz4DCzRDzNTMkMky4HpLLsZZu2K2iqBj-f1TYebw6w@mail.gmail.com>

No prob, feel free to pick from it.

On Sat, Jan 31, 2015 at 1:51 PM, Chris Diehl <cpdiehl at gmail.com> wrote:

> This is a great example Adam. Thanks!
>
> On Sat, Jan 31, 2015 at 10:05 AM, Adam Hughes <hughesadam87 at gmail.com>
> wrote:
>
>> We have a fairly involved one.  The traitlets Model is in specgram.py and
>> the GUI/Widget elements are in specgui.py
>>
>>
>> https://github.com/hugadams/scikit-spectra/tree/master/skspec/interact/ipynbs
>>
>> Here's a video of it in action:
>>
>> http://hugadams.github.io/scikit-spectra/
>>
>> On Sat, Jan 31, 2015 at 11:07 AM, diehl <cpdiehl at gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I'm looking for IPython notebooks containing complex UIs built with the
>>> widget framework. Just trying to get a sense of what is in the realm of
>>> the
>>> possible, especially when it comes to composition and positioning of the
>>> widgets in the notebook. Any examples you can refer me to would be
>>> greatly
>>> appreciated!
>>>
>>> Thanks,
>>> Chris
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://python.6.x6.nabble.com/Complex-UI-Examples-with-Widgets-tp5084847.html
>>> Sent from the IPython - Development mailing list archive at Nabble.com.
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150131/d4d97d1e/attachment.html>

From jeanbigboute at gmail.com  Sat Jan 31 22:41:41 2015
From: jeanbigboute at gmail.com (Jean Bigboute)
Date: Sat, 31 Jan 2015 19:41:41 -0800
Subject: [IPython-dev] Two basic Notebook questions (Python focus)
Message-ID: <CAEen6uyY6u=z95stNyJ5qxk=h=T62NWJoqdrA5S2_5RoJmCb1w@mail.gmail.com>

On Sat, Jan 31, 2015 at 10:00 AM,  <ipython-dev-request at scipy.org> wrote:

>
> Message: 4
> Date: Fri, 30 Jan 2015 15:48:59 -0800
> From: Thomas Kluyver <takowl at gmail.com>
> Subject: Re: [IPython-dev] Two basic Notebook questions (Python focus)

> On 30 January 2015 at 15:26, Jean Bigboute <jeanbigboute at gmail.com> wrote:
>
>> b) Is it possible to pass arguments to a magic function?
>> e.g.
>> import inspect
>> print inspect.getsource(function)
>>
>
> This is probably easiest to do in a plain Python function. Create a file
> ~/.ipython/profile_default/startup/printsource.py (which IPython will
> automatically run when it starts, and put in it:
>
> import inspect
> def source(func)
>     print inspect.getsource(function)
>
> Now you can just type 'source(f)' to see a function in your notebook. Be
> aware, though, that if you share that notebook, other people won't have the
> source() function defined. You could also define it at the top of the
> notebook if you prefer.
>
> Thomas
>

I tried this on Anaconda and Canopy without success. (I also included
a colon at the end of the def source statement.

The code never loaded into the Notebook interface.

On Anaconda:
I then tried plain ipython (no notebook) and got the  error shown below.

I systematically deleted lines in the startup file and eventually had
just a blank file.  Error still results.

The Canopy  distribution is on my work machine so I don't know what
pure ipython would have done on it.

=====================
[TerminalIPythonApp] WARNING | Error in loading extension: kernmagic
Check your config files in /{PATH}/.ipython/profile_default
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/{PATH}/anaconda/lib/python2.7/site-packages/IPython/core/extensions.pyc
in load_extension(self, module_str)
     96             if module_str not in sys.modules:
     97                 with prepended_to_syspath(self.ipython_extension_dir):
---> 98                     __import__(module_str)
     99             mod = sys.modules[module_str]
    100             if self._call_load_ipython_extension(mod):

ImportError: No module named kernmagic