From fperez.net at gmail.com  Mon Sep  2 00:29:12 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 1 Sep 2013 21:29:12 -0700
Subject: [IPython-dev] Curly braces stripped from magic input
In-Reply-To: <5222376C.7070609@third-bit.com>
References: <CAMqnrZpxXpYr+7V1amL5XkkGP2QTxMjF15U2YaRqK=r0SiVZYg@mail.gmail.com>
	<CAOvn4qha2pTzq67EAUx2u8S181NDn4O+XXvFt3XtuNQyXAWQpA@mail.gmail.com>
	<CAMqnrZq3zSqq=Q0kzs8PQwx6hP01vpN6aEur+qVdY+nv0brwkw@mail.gmail.com>
	<CAOvn4qia9CvP42Te+6GTsecWXpAHgaK992sTS5oOxe-rpe=95Q@mail.gmail.com>
	<5222376C.7070609@third-bit.com>
Message-ID: <CAHAreOqed_LGifBrOT0waRYkuoxKHp+jpTwCzwV4ZfGY_GWd3Q@mail.gmail.com>

On Sat, Aug 31, 2013 at 11:35 AM, Greg Wilson <gvwilson at third-bit.com> wrote:
> On 2013-08-31 2:21 PM, Thomas Kluyver wrote:
>
> On 31 August 2013 10:07, Matt Davis <jiffyclub at gmail.com> wrote:
>>
>> Is there any way around this, or do we just have to tell our students to
>> use \d{{3,4}} instead of valid regex?
>
>
> No, I don't think we currently have any way to disable this. You may prefer
> to declare regexes in regular Python code rather than within IPython magic
> commands.
>
> Hi Thomas,
> The point of this tool is to allow us to teach regular expressions *without*
> having to embed them in Python (or anything else) --- we've found they're
> easier for newcomers to digest if they can wrap their heads around
> \d+\s+\d+\b first, and then worry about raw strings, re.whatever, match
> objects, and so on.  What we have now lets us do this:
>
> %%regex a+b
> xyz
> aaabxx
> xabbbx
> xyzab
> xabxabx
>
> which is about as simple as it can get.  We can tell them to double the {}'s
> if we have to, but it seems weird not to have access to the original
> (unchomped) string...

Yes, it's weird and sub-optimal, and just a quirk of the historical
path followed by the implementation.  While we've cleaned up a LOT of
the magics machinery, there's always improvements to be done...

The culprit here is this line of code:

https://github.com/ipython/ipython/blob/master/IPython/core/interactiveshell.py#L2127

where we call var_expand *unconditionally* outside the magic object
itself.  So magics don't have a chance to know what the line before
expansion looked like...

I'd like to rewrite run_line_magic and run_cell_magic so they do even
less and hand over the input as quickly as possible to the magics
themselves. They can then decide whether to expand variables or not,
and would have access to the original buffer.

It's pretty straightforward to do, and would give us a clean and
simple solution to situations like this.

Fancy a PR, Greg? :)

Cheers,

f


From franz.bergesund at gmail.com  Mon Sep  2 09:32:43 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 15:32:43 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by side
Message-ID: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>

Dear List,

I have a script that produces some image and I'm creating a notebook for
testing/logging.

For comparison reasons, I would like to load to images in a cell and show
them side by side.

from IPython.display import Image
constr = Image(filename="fig1.png")
sigma = Image(filename="fig2.png")
display(constr, sigma)


put the figures one after the other. I've tried to play with the width
keyword, but that does not help.

Making an HTML table with the output of "Image" does not work (I don't know
if I should add "of course")

s = """<table>
<tr>
<th>{f1}</th>
<th>{f2}</th>
</tr>""".format(f1=constr, f2=sigma)
t=HTML(s)
display(t)

Is there any way to do what I want directly in the notebook? (I guess that
there is some python library to do it, but I think that the notebook should
be able to do it)

Cheers,

Fra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/53d9da35/attachment.html>

From zvoros at gmail.com  Mon Sep  2 10:03:49 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 02 Sep 2013 16:03:49 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
 side
In-Reply-To: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
Message-ID: <52249AC5.6060807@gmail.com>

Hi Francesco,

It might be a hack, but this works:

with open("roche1.png", "rb") as image_file:
     im1 = 'data:image/png;base64,' + base64.b64encode(image_file.read())

with open("roche2.png", "rb") as image_file:
     im2 = 'data:image/png;base64,' + base64.b64encode(image_file.read())

s = """<table>
<tr>
<th><img src="%s"/></th>
<th><img src="%s"/></th>
</tr></table>"""%(im1, im2)
t=HTML(s)
display(t)


Cheers,
Zolt?n

On 02/09/13 15:32, Francesco Montesano wrote:
> Dear List,
>
> I have a script that produces some image and I'm creating a notebook 
> for testing/logging.
>
> For comparison reasons, I would like to load to images in a cell and 
> show them side by side.
>
>     from IPython.display import Image
>     constr = Image(filename="fig1.png")
>     sigma = Image(filename="fig2.png")
>     display(constr, sigma)
>
>
> put the figures one after the other. I've tried to play with the width 
> keyword, but that does not help.
>
> Making an HTML table with the output of "Image" does not work (I don't 
> know if I should add "of course")
>
> s = """<table>
> <tr>
> <th>{f1}</th>
> <th>{f2}</th>
> </tr>""".format(f1=constr, f2=sigma)
> t=HTML(s)
> display(t)
>
> Is there any way to do what I want directly in the notebook? (I guess 
> that there is some python library to do it, but I think that the 
> notebook should be able to do it)
>
> Cheers,
>
> Fra
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/de587b82/attachment.html>

From stefan at sun.ac.za  Mon Sep  2 10:28:52 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 2 Sep 2013 16:28:52 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
Message-ID: <CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>

On Mon, Sep 2, 2013 at 3:32 PM, Francesco Montesano
<franz.bergesund at gmail.com> wrote:
> For comparison reasons, I would like to load to images in a cell and show
> them side by side.

Perhaps something like this:

http://nbviewer.ipython.org/urls/gist.github.com/stefanv/5013257/raw/5a5271a65371e87cced2182025df2d03751b780b/side_by_side.ipynb

St?fan


From franz.bergesund at gmail.com  Mon Sep  2 10:51:25 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 16:51:25 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <52249AC5.6060807@gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<52249AC5.6060807@gmail.com>
Message-ID: <CAOCdBKJm5sbh5nckDCZTwsCLSGqkq6+PyFjJUaTnGsDm0pfg+A@mail.gmail.com>

Dear Zoltan,

Thanks for the trick.
I'm writing a function that takes a list of figures (and headers) and
builds a table like you suggest.
In the function I do

    import base64
    pngs = []
    for fn in images:  # loop over the input file names
        with open(fn, "rb") as f:
            pngs.append('data:image/png;base64,' +
base64.b64encode(f.read()))

But I get the following error: on the last line:
TypeError: Can't convert 'bytes' object to str implicitly

Is there a way to disable table lines in HTML?

for reference: I'm using python3.3

Cheers,

Fra



2013/9/2 Zolt?n V?r?s <zvoros at gmail.com>

>  Hi Francesco,
>
> It might be a hack, but this works:
>
> with open("roche1.png", "rb") as image_file:
>     im1 = 'data:image/png;base64,' + base64.b64encode(image_file.read())
>
> with open("roche2.png", "rb") as image_file:
>     im2 = 'data:image/png;base64,' + base64.b64encode(image_file.read())
>
> s = """<table>
> <tr>
> <th><img src="%s"/></th>
> <th><img src="%s"/></th>
> </tr></table>"""%(im1, im2)
> t=HTML(s)
> display(t)
>
>
> Cheers,
> Zolt?n
>
>
> On 02/09/13 15:32, Francesco Montesano wrote:
>
> Dear List,
>
>  I have a script that produces some image and I'm creating a notebook for
> testing/logging.
>
>  For comparison reasons, I would like to load to images in a cell and
> show them side by side.
>
>  from IPython.display import Image
>  constr = Image(filename="fig1.png")
>  sigma = Image(filename="fig2.png")
>  display(constr, sigma)
>
>
>  put the figures one after the other. I've tried to play with the width
> keyword, but that does not help.
>
>  Making an HTML table with the output of "Image" does not work (I don't
> know if I should add "of course")
>
>  s = """<table>
> <tr>
> <th>{f1}</th>
> <th>{f2}</th>
> </tr>""".format(f1=constr, f2=sigma)
> t=HTML(s)
> display(t)
>
>  Is there any way to do what I want directly in the notebook? (I guess
> that there is some python library to do it, but I think that the notebook
> should be able to do it)
>
>  Cheers,
>
>  Fra
>
>
> _______________________________________________
> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/2a10e53d/attachment.html>

From zvoros at gmail.com  Mon Sep  2 10:57:32 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 02 Sep 2013 16:57:32 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
 side
In-Reply-To: <CAOCdBKJm5sbh5nckDCZTwsCLSGqkq6+PyFjJUaTnGsDm0pfg+A@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<52249AC5.6060807@gmail.com>
	<CAOCdBKJm5sbh5nckDCZTwsCLSGqkq6+PyFjJUaTnGsDm0pfg+A@mail.gmail.com>
Message-ID: <5224A75C.1010407@gmail.com>


Hi Francesco,
>
>     import base64
>     pngs = []
>     for fn in images:  # loop over the input file names
>         with open(fn, "rb") as f:
>             pngs.append('data:image/png;base64,' + 
> base64.b64encode(f.read()))
>
> But I get the following error: on the last line:
> TypeError: Can't convert 'bytes' object to str implicitly
>
> Is there a way to disable table lines in HTML?
 From this, it seems to me that you encounter the problem on the 
png-to-base64 conversion. Is that correct? Because afterwards the image 
is encoded in base64, so it's not binary anymore, and I don't see why 
html should fail at that point. But if it is an error in b64encode, then 
you probably don't have a valid input file.

Cheers,
Zolt?n
>
>
> 2013/9/2 Zolt?n V?r?s <zvoros at gmail.com <mailto:zvoros at gmail.com>>
>
>     Hi Francesco,
>
>     It might be a hack, but this works:
>
>     with open("roche1.png", "rb") as image_file:
>         im1 = 'data:image/png;base64,' +
>     base64.b64encode(image_file.read())
>
>     with open("roche2.png", "rb") as image_file:
>         im2 = 'data:image/png;base64,' +
>     base64.b64encode(image_file.read())
>
>     s = """<table>
>     <tr>
>     <th><img src="%s"/></th>
>     <th><img src="%s"/></th>
>     </tr></table>"""%(im1, im2)
>     t=HTML(s)
>     display(t)
>
>
>     Cheers,
>     Zolt?n
>
>
>     On 02/09/13 15:32, Francesco Montesano wrote:
>>     Dear List,
>>
>>     I have a script that produces some image and I'm creating a
>>     notebook for testing/logging.
>>
>>     For comparison reasons, I would like to load to images in a cell
>>     and show them side by side.
>>
>>         from IPython.display import Image
>>         constr = Image(filename="fig1.png")
>>         sigma = Image(filename="fig2.png")
>>         display(constr, sigma)
>>
>>
>>     put the figures one after the other. I've tried to play with the
>>     width keyword, but that does not help.
>>
>>     Making an HTML table with the output of "Image" does not work (I
>>     don't know if I should add "of course")
>>
>>     s = """<table>
>>     <tr>
>>     <th>{f1}</th>
>>     <th>{f2}</th>
>>     </tr>""".format(f1=constr, f2=sigma)
>>     t=HTML(s)
>>     display(t)
>>
>>     Is there any way to do what I want directly in the notebook? (I
>>     guess that there is some python library to do it, but I think
>>     that the notebook should be able to do it)
>>
>>     Cheers,
>>
>>     Fra
>>
>>
>>     _______________________________________________
>>     IPython-dev mailing list
>>     IPython-dev at scipy.org  <mailto:IPython-dev at scipy.org>
>>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/e42e67fd/attachment.html>

From franz.bergesund at gmail.com  Mon Sep  2 11:14:05 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 17:14:05 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <5224A75C.1010407@gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<52249AC5.6060807@gmail.com>
	<CAOCdBKJm5sbh5nckDCZTwsCLSGqkq6+PyFjJUaTnGsDm0pfg+A@mail.gmail.com>
	<5224A75C.1010407@gmail.com>
Message-ID: <CAOCdBKJwFaA7TwHM2-h-Thq9fdC3EgzqjZxa2MW-aVPSAhRfBw@mail.gmail.com>

Dear Zoltan,

I've split the line with the append

            fig_base64 = base64.b64encode(f.read())  # this works: so file
read and encoded without problems
            full_html = 'data:image/png;base64,' + fig_base64   #
error TypeError: Can't convert 'bytes' object to str implicitly
            pngs.append(full_html)

Now if I try to print(fig_base64) before the second line I get:

TypeError: Can't convert 'bytes' object to str implicitly

b'iVBORw0KGgoAAAANS






2013/9/2 Zolt?n V?r?s <zvoros at gmail.com>

>
> Hi Francesco,
>
>
>     import base64
>      pngs = []
>      for fn in images:  # loop over the input file names
>         with open(fn, "rb") as f:
>             pngs.append('data:image/png;base64,' +
> base64.b64encode(f.read()))
>
>  But I get the following error: on the last line:
> TypeError: Can't convert 'bytes' object to str implicitly
>
>  Is there a way to disable table lines in HTML?
>
> From this, it seems to me that you encounter the problem on the
> png-to-base64 conversion. Is that correct? Because afterwards the image is
> encoded in base64, so it's not binary anymore, and I don't see why html
> should fail at that point. But if it is an error in b64encode, then you
> probably don't have a valid input file.
>
> Cheers,
> Zolt?n
>
>
>
> 2013/9/2 Zolt?n V?r?s <zvoros at gmail.com>
>
>>  Hi Francesco,
>>
>> It might be a hack, but this works:
>>
>> with open("roche1.png", "rb") as image_file:
>>     im1 = 'data:image/png;base64,' + base64.b64encode(image_file.read())
>>
>> with open("roche2.png", "rb") as image_file:
>>     im2 = 'data:image/png;base64,' + base64.b64encode(image_file.read())
>>
>> s = """<table>
>> <tr>
>> <th><img src="%s"/></th>
>> <th><img src="%s"/></th>
>> </tr></table>"""%(im1, im2)
>> t=HTML(s)
>> display(t)
>>
>>
>> Cheers,
>> Zolt?n
>>
>>
>> On 02/09/13 15:32, Francesco Montesano wrote:
>>
>>  Dear List,
>>
>>  I have a script that produces some image and I'm creating a notebook
>> for testing/logging.
>>
>>  For comparison reasons, I would like to load to images in a cell and
>> show them side by side.
>>
>>  from IPython.display import Image
>>  constr = Image(filename="fig1.png")
>>  sigma = Image(filename="fig2.png")
>>  display(constr, sigma)
>>
>>
>>  put the figures one after the other. I've tried to play with the width
>> keyword, but that does not help.
>>
>>  Making an HTML table with the output of "Image" does not work (I don't
>> know if I should add "of course")
>>
>>  s = """<table>
>> <tr>
>> <th>{f1}</th>
>> <th>{f2}</th>
>> </tr>""".format(f1=constr, f2=sigma)
>> t=HTML(s)
>> display(t)
>>
>>  Is there any way to do what I want directly in the notebook? (I guess
>> that there is some python library to do it, but I think that the notebook
>> should be able to do it)
>>
>>  Cheers,
>>
>>  Fra
>>
>>
>>  _______________________________________________
>> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> _______________________________________________
> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/5635f4ef/attachment.html>

From franz.bergesund at gmail.com  Mon Sep  2 11:15:40 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 17:15:40 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBKJwFaA7TwHM2-h-Thq9fdC3EgzqjZxa2MW-aVPSAhRfBw@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<52249AC5.6060807@gmail.com>
	<CAOCdBKJm5sbh5nckDCZTwsCLSGqkq6+PyFjJUaTnGsDm0pfg+A@mail.gmail.com>
	<5224A75C.1010407@gmail.com>
	<CAOCdBKJwFaA7TwHM2-h-Thq9fdC3EgzqjZxa2MW-aVPSAhRfBw@mail.gmail.com>
Message-ID: <CAOCdBKJGz_X7HNSbrxqx80nZwbwWjvi7+Ta1XHyDyDMtZgeyCg@mail.gmail.com>

[ps: Sent by error]

So it's complaining about trying to concatenate a string and a 'bytes'
object

Do you know if/how to solve it?

Fra


2013/9/2 Francesco Montesano <franz.bergesund at gmail.com>

> Dear Zoltan,
>
> I've split the line with the append
>
>             fig_base64 = base64.b64encode(f.read())  # this works: so file
> read and encoded without problems
>             full_html = 'data:image/png;base64,' + fig_base64   #
> error TypeError: Can't convert 'bytes' object to str implicitly
>             pngs.append(full_html)
>
>  Now if I try to print(fig_base64) before the second line I get:
>
> TypeError: Can't convert 'bytes' object to str implicitly
>
> b'iVBORw0KGgoAAAANS
>
>
>
>
>
>
> 2013/9/2 Zolt?n V?r?s <zvoros at gmail.com>
>
>>
>> Hi Francesco,
>>
>>
>>     import base64
>>      pngs = []
>>      for fn in images:  # loop over the input file names
>>         with open(fn, "rb") as f:
>>             pngs.append('data:image/png;base64,' +
>> base64.b64encode(f.read()))
>>
>>  But I get the following error: on the last line:
>> TypeError: Can't convert 'bytes' object to str implicitly
>>
>>  Is there a way to disable table lines in HTML?
>>
>> From this, it seems to me that you encounter the problem on the
>> png-to-base64 conversion. Is that correct? Because afterwards the image is
>> encoded in base64, so it's not binary anymore, and I don't see why html
>> should fail at that point. But if it is an error in b64encode, then you
>> probably don't have a valid input file.
>>
>> Cheers,
>> Zolt?n
>>
>>
>>
>> 2013/9/2 Zolt?n V?r?s <zvoros at gmail.com>
>>
>>>  Hi Francesco,
>>>
>>> It might be a hack, but this works:
>>>
>>> with open("roche1.png", "rb") as image_file:
>>>     im1 = 'data:image/png;base64,' + base64.b64encode(image_file.read())
>>>
>>> with open("roche2.png", "rb") as image_file:
>>>     im2 = 'data:image/png;base64,' + base64.b64encode(image_file.read())
>>>
>>> s = """<table>
>>> <tr>
>>> <th><img src="%s"/></th>
>>> <th><img src="%s"/></th>
>>> </tr></table>"""%(im1, im2)
>>> t=HTML(s)
>>> display(t)
>>>
>>>
>>> Cheers,
>>> Zolt?n
>>>
>>>
>>> On 02/09/13 15:32, Francesco Montesano wrote:
>>>
>>>  Dear List,
>>>
>>>  I have a script that produces some image and I'm creating a notebook
>>> for testing/logging.
>>>
>>>  For comparison reasons, I would like to load to images in a cell and
>>> show them side by side.
>>>
>>>  from IPython.display import Image
>>>  constr = Image(filename="fig1.png")
>>>  sigma = Image(filename="fig2.png")
>>>  display(constr, sigma)
>>>
>>>
>>>  put the figures one after the other. I've tried to play with the width
>>> keyword, but that does not help.
>>>
>>>  Making an HTML table with the output of "Image" does not work (I don't
>>> know if I should add "of course")
>>>
>>>  s = """<table>
>>> <tr>
>>> <th>{f1}</th>
>>> <th>{f2}</th>
>>> </tr>""".format(f1=constr, f2=sigma)
>>> t=HTML(s)
>>> display(t)
>>>
>>>  Is there any way to do what I want directly in the notebook? (I guess
>>> that there is some python library to do it, but I think that the notebook
>>> should be able to do it)
>>>
>>>  Cheers,
>>>
>>>  Fra
>>>
>>>
>>>  _______________________________________________
>>> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/6e8a2b82/attachment.html>

From franz.bergesund at gmail.com  Mon Sep  2 11:18:19 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 17:18:19 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
Message-ID: <CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>

Dear Stefan,

thanks but I wouldn't know how to use it.
'Image' object have a _repr_html_ that returns nothing. The png is stored
in data or returned by _repr_png_.
And I don't see how that would help.

Fra


2013/9/2 St?fan van der Walt <stefan at sun.ac.za>

> On Mon, Sep 2, 2013 at 3:32 PM, Francesco Montesano
> <franz.bergesund at gmail.com> wrote:
> > For comparison reasons, I would like to load to images in a cell and show
> > them side by side.
>
> Perhaps something like this:
>
>
> http://nbviewer.ipython.org/urls/gist.github.com/stefanv/5013257/raw/5a5271a65371e87cced2182025df2d03751b780b/side_by_side.ipynb
>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/f1fac2c1/attachment.html>

From asmeurer at gmail.com  Mon Sep  2 12:47:53 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 2 Sep 2013 10:47:53 -0600
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
Message-ID: <3650923831373097372@unknownmsgid>

It would be useful if pandas data frames displayed objects with custom
display formatters in the HTML table. This has come up in other situations
as well. See http://stackoverflow.com/q/18503351/161801

Aaron Meurer

On Sep 2, 2013, at 9:18 AM, Francesco Montesano <franz.bergesund at gmail.com>
wrote:

Dear Stefan,

thanks but I wouldn't know how to use it.
'Image' object have a _repr_html_ that returns nothing. The png is stored
in data or returned by _repr_png_.
And I don't see how that would help.

Fra


2013/9/2 St?fan van der Walt <stefan at sun.ac.za>

> On Mon, Sep 2, 2013 at 3:32 PM, Francesco Montesano
> <franz.bergesund at gmail.com> wrote:
> > For comparison reasons, I would like to load to images in a cell and show
> > them side by side.
>
> Perhaps something like this:
>
>
> http://nbviewer.ipython.org/urls/gist.github.com/stefanv/5013257/raw/5a5271a65371e87cced2182025df2d03751b780b/side_by_side.ipynb
>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/4ca05956/attachment.html>

From zvoros at gmail.com  Mon Sep  2 13:15:13 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 02 Sep 2013 19:15:13 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
 side
In-Reply-To: <CAOCdBKJwFaA7TwHM2-h-Thq9fdC3EgzqjZxa2MW-aVPSAhRfBw@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<52249AC5.6060807@gmail.com>
	<CAOCdBKJm5sbh5nckDCZTwsCLSGqkq6+PyFjJUaTnGsDm0pfg+A@mail.gmail.com>
	<5224A75C.1010407@gmail.com>
	<CAOCdBKJwFaA7TwHM2-h-Thq9fdC3EgzqjZxa2MW-aVPSAhRfBw@mail.gmail.com>
Message-ID: <5224C7A1.2050000@gmail.com>

Hi Francesco,

I have tried your code in python 2.7, and I don't encounter the problem 
that you describe. However, it seems that there are some complications 
in python 3. See this thread: 
http://stackoverflow.com/questions/16740188/base64-b64encode-error

I hope this helps,

Zolt?n

On 02/09/13 17:14, Francesco Montesano wrote:
> Dear Zoltan,
>
> I've split the line with the append
>
>             fig_base64 = base64.b64encode(f.read())  # this works: so 
> file read and encoded without problems
>             full_html = 'data:image/png;base64,' + fig_base64   # 
> error TypeError: Can't convert 'bytes' object to str implicitly
>             pngs.append(full_html)
>
> Now if I try to print(fig_base64) before the second line I get:
>
> TypeError: Can't convert 'bytes' object to str implicitly
>
> b'iVBORw0KGgoAAAANS
>
>
>
>
>
>
> 2013/9/2 Zolt?n V?r?s <zvoros at gmail.com <mailto:zvoros at gmail.com>>
>
>
>     Hi Francesco,
>>
>>         import base64
>>         pngs = []
>>         for fn in images:  # loop over the input file names
>>             with open(fn, "rb") as f:
>>     pngs.append('data:image/png;base64,' + base64.b64encode(f.read()))
>>
>>     But I get the following error: on the last line:
>>     TypeError: Can't convert 'bytes' object to str implicitly
>>
>>     Is there a way to disable table lines in HTML?
>     From this, it seems to me that you encounter the problem on the
>     png-to-base64 conversion. Is that correct? Because afterwards the
>     image is encoded in base64, so it's not binary anymore, and I
>     don't see why html should fail at that point. But if it is an
>     error in b64encode, then you probably don't have a valid input file.
>
>     Cheers,
>     Zolt?n
>
>>
>>
>>     2013/9/2 Zolt?n V?r?s <zvoros at gmail.com <mailto:zvoros at gmail.com>>
>>
>>         Hi Francesco,
>>
>>         It might be a hack, but this works:
>>
>>         with open("roche1.png", "rb") as image_file:
>>             im1 = 'data:image/png;base64,' +
>>         base64.b64encode(image_file.read())
>>
>>         with open("roche2.png", "rb") as image_file:
>>             im2 = 'data:image/png;base64,' +
>>         base64.b64encode(image_file.read())
>>
>>         s = """<table>
>>         <tr>
>>         <th><img src="%s"/></th>
>>         <th><img src="%s"/></th>
>>         </tr></table>"""%(im1, im2)
>>         t=HTML(s)
>>         display(t)
>>
>>
>>         Cheers,
>>         Zolt?n
>>
>>
>>         On 02/09/13 15:32, Francesco Montesano wrote:
>>>         Dear List,
>>>
>>>         I have a script that produces some image and I'm creating a
>>>         notebook for testing/logging.
>>>
>>>         For comparison reasons, I would like to load to images in a
>>>         cell and show them side by side.
>>>
>>>             from IPython.display import Image
>>>             constr = Image(filename="fig1.png")
>>>             sigma = Image(filename="fig2.png")
>>>             display(constr, sigma)
>>>
>>>
>>>         put the figures one after the other. I've tried to play with
>>>         the width keyword, but that does not help.
>>>
>>>         Making an HTML table with the output of "Image" does not
>>>         work (I don't know if I should add "of course")
>>>
>>>         s = """<table>
>>>         <tr>
>>>         <th>{f1}</th>
>>>         <th>{f2}</th>
>>>         </tr>""".format(f1=constr, f2=sigma)
>>>         t=HTML(s)
>>>         display(t)
>>>
>>>         Is there any way to do what I want directly in the notebook?
>>>         (I guess that there is some python library to do it, but I
>>>         think that the notebook should be able to do it)
>>>
>>>         Cheers,
>>>
>>>         Fra
>>>
>>>
>>>         _______________________________________________
>>>         IPython-dev mailing list
>>>         IPython-dev at scipy.org  <mailto:IPython-dev at scipy.org>
>>>         http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>         _______________________________________________
>>         IPython-dev mailing list
>>         IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>>         http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>>
>>     _______________________________________________
>>     IPython-dev mailing list
>>     IPython-dev at scipy.org  <mailto:IPython-dev at scipy.org>
>>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/bafeaae6/attachment.html>

From stefan at sun.ac.za  Mon Sep  2 16:22:27 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 2 Sep 2013 22:22:27 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
Message-ID: <CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>

On Mon, Sep 2, 2013 at 5:18 PM, Francesco Montesano
<franz.bergesund at gmail.com> wrote:
> thanks but I wouldn't know how to use it.
> 'Image' object have a _repr_html_ that returns nothing. The png is stored in
> data or returned by _repr_png_.
> And I don't see how that would help.

Here's a full example:
http://nbviewer.ipython.org/urls/gist.github.com/stefanv/6416926/raw/96978d6ad6991c26ccce995327b1adb6aceeeb6e/side_by_side.ipynb

St?fan


From franz.bergesund at gmail.com  Mon Sep  2 16:31:54 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 20:31:54 +0000
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
Message-ID: <CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>

Dear Stefan,

thank you very much for the example. This should work wonderfully for
python 2.x., but, as Zoltan pointed out, it won't work on python 3.x, as
you can't concatenate string and bytes objects (returned by base64 methods).
I've sent some time to search for something that convert a b'...' object
into a string, but I'm not sure that I've found anything and I doubt that
it would work.

Cheers,
Fra


2013/9/2 St?fan van der Walt <stefan at sun.ac.za>

> On Mon, Sep 2, 2013 at 5:18 PM, Francesco Montesano
> <franz.bergesund at gmail.com> wrote:
> > thanks but I wouldn't know how to use it.
> > 'Image' object have a _repr_html_ that returns nothing. The png is
> stored in
> > data or returned by _repr_png_.
> > And I don't see how that would help.
>
> Here's a full example:
>
> http://nbviewer.ipython.org/urls/gist.github.com/stefanv/6416926/raw/96978d6ad6991c26ccce995327b1adb6aceeeb6e/side_by_side.ipynb
>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/42ca902f/attachment.html>

From stefan at sun.ac.za  Mon Sep  2 16:38:45 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 2 Sep 2013 22:38:45 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
	<CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
Message-ID: <CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>

On Mon, Sep 2, 2013 at 10:31 PM, Francesco Montesano
<franz.bergesund at gmail.com> wrote:
> thank you very much for the example. This should work wonderfully for python
> 2.x., but, as Zoltan pointed out, it won't work on python 3.x, as you can't
> concatenate string and bytes objects (returned by base64 methods).

That shouldn't be too much of a problem:

In [12]: bytes('asd'.encode('utf-8')) + base64.b64encode(b'asd')
Out[12]: b'asdYXNk'

In [13]: (bytes('asd'.encode('utf-8')) +
base64.b64encode(b'asd')).decode('utf-8')
Out[13]: 'asdYXNk'

Have a look at "numpy.compat" for some utility functions as well.

St?fan


From franz.bergesund at gmail.com  Mon Sep  2 16:42:26 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 2 Sep 2013 20:42:26 +0000
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
	<CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
	<CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>
Message-ID: <CAOCdBKKvdRdzkpJ3q2S8nhmvwbK0EQ4FEaZ7O-BbEAg0oudxyQ@mail.gmail.com>

Thanks,

I'll give a try tomorrow. not it's getting late
Do you know if HTML understands b'...'?

Fra



2013/9/2 St?fan van der Walt <stefan at sun.ac.za>

> On Mon, Sep 2, 2013 at 10:31 PM, Francesco Montesano
> <franz.bergesund at gmail.com> wrote:
> > thank you very much for the example. This should work wonderfully for
> python
> > 2.x., but, as Zoltan pointed out, it won't work on python 3.x, as you
> can't
> > concatenate string and bytes objects (returned by base64 methods).
>
> That shouldn't be too much of a problem:
>
> In [12]: bytes('asd'.encode('utf-8')) + base64.b64encode(b'asd')
> Out[12]: b'asdYXNk'
>
> In [13]: (bytes('asd'.encode('utf-8')) +
> base64.b64encode(b'asd')).decode('utf-8')
> Out[13]: 'asdYXNk'
>
> Have a look at "numpy.compat" for some utility functions as well.
>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130902/94d18263/attachment.html>

From dave.hirschfeld at gmail.com  Tue Sep  3 10:08:40 2013
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Tue, 3 Sep 2013 14:08:40 +0000 (UTC)
Subject: [IPython-dev] Invalid cp0 encoding
Message-ID: <loom.20130903T152627-288@post.gmane.org>

Just pinging the list to mention my attempt at a PR 
(https://github.com/ipython/ipython/pull/4163) to fix the error I was 
encountering below.

Whilst trying out rendering notebooks in a flask app under Apache on Windows 
I got the below error when simply trying to import `SlidesExporter`:

```
mod_wsgi (pid=6260): Exception occurred processing WSGI script 
'flask_test.wsgi'.
Traceback (most recent call last):
  File "flask_test.py", line 81, in render_notebook
    from IPython.nbconvert.exporters import SlidesExporter
  File "c:\\dev\\code\\ipython\\IPython\\__init__.py", line 47, in <module>
    from .terminal.embed import embed
  File "c:\\dev\\code\\ipython\\IPython\\terminal\\embed.py", line 32, in 
<module>
    from IPython.terminal.interactiveshell import TerminalInteractiveShell
  File "c:\\dev\\code\\ipython\\IPython\\terminal\\interactiveshell.py", 
line 25, in <module>
    from IPython.core.interactiveshell import InteractiveShell, 
InteractiveShellABC
  File "c:\\dev\\code\\ipython\\IPython\\core\\interactiveshell.py", line 
59, in <module>
    from IPython.core.prompts import PromptManager
  File "c:\\dev\\code\\ipython\\IPython\\core\\prompts.py", line 138, in 
<module>
    HOME = 
py3compat.str_to_unicode(os.environ.get("HOME","//////:::::ZZZZZ,,,~~~"))
  File "c:\\dev\\code\\ipython\\IPython\\utils\\py3compat.py", line 18, in 
decode
    return s.decode(encoding, "replace")
LookupError: unknown encoding: cp0
```

NB: Although it now works great I have no idea what the second argument 
returned by `SlidesExporter.from_filename` is or is for. Is there any 
documentation on this?

```
In [34]: html, resources = exporter.from_filename('testnotebook.ipynb')

In [35]: type(resources)
Out[35]: IPython.nbconvert.exporters.exporter.ResourcesDict

In [36]: resources.keys()
Out[36]: ['reveal', 'inlining', 'output_extension', 'metadata']

In [37]: print IPython.sys_info()
{'codename': 'An Afternoon Hack',
 'commit_hash': 'ee40094',
 'commit_source': 'repository',
 'default_encoding': 'cp1252',
 'ipython_path': 'c:\\dev\\code\\ipython\\IPython',
 'ipython_version': '2.0.0-dev',
 'os_name': 'nt',
 'platform': 'Windows-2008ServerR2-6.1.7601-SP1',
 'sys_executable': 'C:\\dev\\bin\\Python27\\python.exe',
 'sys_platform': 'win32',
 'sys_version': '2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit 
(AMD64)]'}
```

Thanks,
Dave




From patrickmarshwx at gmail.com  Tue Sep  3 12:36:58 2013
From: patrickmarshwx at gmail.com (Patrick Marsh)
Date: Tue, 3 Sep 2013 11:36:58 -0500
Subject: [IPython-dev] Basemap Segfault in IPython; Works in Python
Message-ID: <CAHUTaHEV6bTk6UrF11yEwJuiMy3Pi=Hfnm=gcGhhtBebwupruw@mail.gmail.com>

Hi, All,

I'm not sure what is going on, but the following fails in IPython (terminal
and notebook) but works just fine in regular Python.

I'm using OS X 10.8.4; git master for git master for Matplotlib (a091f6d),
IPython (9f92804), and Basemap (1d7664c); and geos version 3.4.2 (built by
Homebrew).

The following script will work from a Python prompt but fails (segfaults)
from the IPython prompt. After doing some digging it appears that the
segfault occurs when accessing some of the methods associated with the
generated geos module (_geoslib.so). I've rolled back to previous versions
of everything and I still have the same issues.

What I cannot figure out is why this would work with a pure Python
interpreter, but fail in the IPython console.


=====
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap

m = Basemap()    # This is where the segfault occurs

m.drawcoastlines()
plt.show()
=====


I originally posted this on the IPython Users list and got no response, so
I thought I would try here before posting a bug on both IPython and
Basemap's Github issue trackers.


Thanks for any help or ideas on fixing this issue, or at least on how to
track down this issue...


Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130903/86065f59/attachment.html>

From benjaminrk at gmail.com  Tue Sep  3 14:15:10 2013
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 3 Sep 2013 11:15:10 -0700
Subject: [IPython-dev] Basemap Segfault in IPython; Works in Python
In-Reply-To: <CAHUTaHEV6bTk6UrF11yEwJuiMy3Pi=Hfnm=gcGhhtBebwupruw@mail.gmail.com>
References: <CAHUTaHEV6bTk6UrF11yEwJuiMy3Pi=Hfnm=gcGhhtBebwupruw@mail.gmail.com>
Message-ID: <CAHNn8BXs1zTcY=qVNtWVQRYxdKAU-PZseTCWRVSdqXWTqPOnww@mail.gmail.com>

This is terminal IPython?

Is there a chance that the matplotlib backend is different in each case?
Have you enabled matplotlib eventloop integration in IPython (otherwise, it
will block when you draw a plot).


On Tue, Sep 3, 2013 at 9:36 AM, Patrick Marsh <patrickmarshwx at gmail.com>wrote:

> Hi, All,
>
> I'm not sure what is going on, but the following fails in IPython
> (terminal and notebook) but works just fine in regular Python.
>
> I'm using OS X 10.8.4; git master for git master for Matplotlib (a091f6d),
> IPython (9f92804), and Basemap (1d7664c); and geos version 3.4.2 (built by
> Homebrew).
>
> The following script will work from a Python prompt but fails (segfaults)
> from the IPython prompt. After doing some digging it appears that the
> segfault occurs when accessing some of the methods associated with the
> generated geos module (_geoslib.so). I've rolled back to previous versions
> of everything and I still have the same issues.
>
> What I cannot figure out is why this would work with a pure Python
> interpreter, but fail in the IPython console.
>
>
> =====
> import matplotlib.pyplot as plt
> from mpl_toolkits.basemap import Basemap
>
> m = Basemap()    # This is where the segfault occurs
>
> m.drawcoastlines()
> plt.show()
> =====
>
>
> I originally posted this on the IPython Users list and got no response, so
> I thought I would try here before posting a bug on both IPython and
> Basemap's Github issue trackers.
>
>
> Thanks for any help or ideas on fixing this issue, or at least on how to
> track down this issue...
>
>
> Patrick
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130903/fa3bcaf1/attachment.html>

From patrickmarshwx at gmail.com  Tue Sep  3 14:27:37 2013
From: patrickmarshwx at gmail.com (Patrick Marsh)
Date: Tue, 3 Sep 2013 13:27:37 -0500
Subject: [IPython-dev] Basemap Segfault in IPython; Works in Python
In-Reply-To: <CAHNn8BXs1zTcY=qVNtWVQRYxdKAU-PZseTCWRVSdqXWTqPOnww@mail.gmail.com>
References: <CAHUTaHEV6bTk6UrF11yEwJuiMy3Pi=Hfnm=gcGhhtBebwupruw@mail.gmail.com>
	<CAHNn8BXs1zTcY=qVNtWVQRYxdKAU-PZseTCWRVSdqXWTqPOnww@mail.gmail.com>
Message-ID: <CAHUTaHG_bDLgCYQiCZEgyOWqud84v47M7Uo+=8YxpL1dQ2XO0w@mail.gmail.com>

Sorry if this is a double post; used the wrong email in the first reply and
got at least one failure notice...

This is terminal IPython and IPython Notebook. (I noticed it first in the
IPython Notebook). The backends are the same in both Python and IPython
(and IPython Notebook). I have not changed any of the default IPython
settings.

I'm wondering if this is a path issue of some sorts?namely IPython is
picking up an old version of geos that I cannot seem to find. At least this
is the angle I'm currently trying to work on. I should add that I've remove
all my manually installed modules and "started over". I built Numpy,
Matplotlib, Basemap, and then IPython (in that order), and still have the
same segfault issue.


Patrick



On Tue, Sep 3, 2013 at 1:15 PM, MinRK <benjaminrk at gmail.com> wrote:

> This is terminal IPython?
>
> Is there a chance that the matplotlib backend is different in each case?
> Have you enabled matplotlib eventloop integration in IPython (otherwise, it
> will block when you draw a plot).
>
>
> On Tue, Sep 3, 2013 at 9:36 AM, Patrick Marsh <patrickmarshwx at gmail.com>wrote:
>
>> Hi, All,
>>
>> I'm not sure what is going on, but the following fails in IPython
>> (terminal and notebook) but works just fine in regular Python.
>>
>> I'm using OS X 10.8.4; git master for git master for Matplotlib
>> (a091f6d), IPython (9f92804), and Basemap (1d7664c); and geos version 3.4.2
>> (built by Homebrew).
>>
>> The following script will work from a Python prompt but fails (segfaults)
>> from the IPython prompt. After doing some digging it appears that the
>> segfault occurs when accessing some of the methods associated with the
>> generated geos module (_geoslib.so). I've rolled back to previous versions
>> of everything and I still have the same issues.
>>
>> What I cannot figure out is why this would work with a pure Python
>> interpreter, but fail in the IPython console.
>>
>>
>> =====
>> import matplotlib.pyplot as plt
>> from mpl_toolkits.basemap import Basemap
>>
>> m = Basemap()    # This is where the segfault occurs
>>
>> m.drawcoastlines()
>> plt.show()
>> =====
>>
>>
>> I originally posted this on the IPython Users list and got no response,
>> so I thought I would try here before posting a bug on both IPython and
>> Basemap's Github issue trackers.
>>
>>
>> Thanks for any help or ideas on fixing this issue, or at least on how to
>> track down this issue...
>>
>>
>> Patrick
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130903/3cc5efad/attachment.html>

From gvwilson at third-bit.com  Tue Sep  3 20:49:26 2013
From: gvwilson at third-bit.com (Greg Wilson)
Date: Tue, 03 Sep 2013 20:49:26 -0400
Subject: [IPython-dev] Curly braces stripped from magic input
In-Reply-To: <CAHAreOqed_LGifBrOT0waRYkuoxKHp+jpTwCzwV4ZfGY_GWd3Q@mail.gmail.com>
References: <CAMqnrZpxXpYr+7V1amL5XkkGP2QTxMjF15U2YaRqK=r0SiVZYg@mail.gmail.com>
	<CAOvn4qha2pTzq67EAUx2u8S181NDn4O+XXvFt3XtuNQyXAWQpA@mail.gmail.com>
	<CAMqnrZq3zSqq=Q0kzs8PQwx6hP01vpN6aEur+qVdY+nv0brwkw@mail.gmail.com>
	<CAOvn4qia9CvP42Te+6GTsecWXpAHgaK992sTS5oOxe-rpe=95Q@mail.gmail.com>
	<5222376C.7070609@third-bit.com>
	<CAHAreOqed_LGifBrOT0waRYkuoxKHp+jpTwCzwV4ZfGY_GWd3Q@mail.gmail.com>
Message-ID: <52268396.8060503@third-bit.com>

On 2013-09-02 12:29 AM, Fernando Perez wrote:
> Yes, it's weird and sub-optimal, and just a quirk of the historical
> path followed by the implementation.  While we've cleaned up a LOT of
> the magics machinery, there's always improvements to be done...
>
> The culprit here is this line of code:
>
> https://github.com/ipython/ipython/blob/master/IPython/core/interactiveshell.py#L2127
>
> where we call var_expand *unconditionally* outside the magic object
> itself.  So magics don't have a chance to know what the line before
> expansion looked like...
>
> I'd like to rewrite run_line_magic and run_cell_magic so they do even
> less and hand over the input as quickly as possible to the magics
> themselves. They can then decide whether to expand variables or not,
> and would have access to the original buffer.
>
> It's pretty straightforward to do, and would give us a clean and
> simple solution to situations like this.
>
> Fancy a PR, Greg? :)
>
Yup; I'll try to get something in tomorrow.
- G


From alimanfoo at googlemail.com  Wed Sep  4 07:55:15 2013
From: alimanfoo at googlemail.com (Alistair Miles)
Date: Wed, 4 Sep 2013 12:55:15 +0100
Subject: [IPython-dev] notebook container width
Message-ID: <CAMr-JwYEFXe5aFv4MsSa+rutzfMJGO6z-kr+Gn=GWNTXHuYPSQ@mail.gmail.com>

Hi all,

Apologies if this is the wrong place but I just wanted to mention that the
new version of the notebook is wonderful except for limiting the width of
the main container to a fixed number of pixels. I regularly make big plots
to look at on my wide screen and it's just a lot more convenient if the
main container width is 100% of the available screen. I can hack this in
the CSS but if others feel the same it might be worth making this a view or
configuration option?

Cheers,
Alistair
-- 
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health <http://cggh.org>
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Web: http://purl.org/net/aliman
Email: alimanfoo at gmail.com
Tel: +44 (0)1865 287721 ***new number***
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130904/572a2721/attachment.html>

From scopatz at gmail.com  Wed Sep  4 10:51:40 2013
From: scopatz at gmail.com (Anthony Scopatz)
Date: Wed, 4 Sep 2013 16:51:40 +0200
Subject: [IPython-dev] notebook container width
In-Reply-To: <CAMr-JwYEFXe5aFv4MsSa+rutzfMJGO6z-kr+Gn=GWNTXHuYPSQ@mail.gmail.com>
References: <CAMr-JwYEFXe5aFv4MsSa+rutzfMJGO6z-kr+Gn=GWNTXHuYPSQ@mail.gmail.com>
Message-ID: <CAPk-6T58oHVGXHqXRNJrFiSPd0VrNQRyNQPPjUSEWhKhZeryuA@mail.gmail.com>

Hello All,

I'd also like to see an option like this somewhere.

Be Well
Anthony


On Wed, Sep 4, 2013 at 1:55 PM, Alistair Miles <alimanfoo at googlemail.com>wrote:

> Hi all,
>
> Apologies if this is the wrong place but I just wanted to mention that the
> new version of the notebook is wonderful except for limiting the width of
> the main container to a fixed number of pixels. I regularly make big plots
> to look at on my wide screen and it's just a lot more convenient if the
> main container width is 100% of the available screen. I can hack this in
> the CSS but if others feel the same it might be worth making this a view or
> configuration option?
>
> Cheers,
> Alistair
> --
> Alistair Miles
> Head of Epidemiological Informatics
> Centre for Genomics and Global Health <http://cggh.org>
> The Wellcome Trust Centre for Human Genetics
> Roosevelt Drive
> Oxford
> OX3 7BN
> United Kingdom
> Web: http://purl.org/net/aliman
> Email: alimanfoo at gmail.com
> Tel: +44 (0)1865 287721 ***new number***
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130904/3aedaabf/attachment.html>

From takowl at gmail.com  Wed Sep  4 18:44:06 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 4 Sep 2013 15:44:06 -0700
Subject: [IPython-dev] Working on the test machinery
Message-ID: <CAOvn4qhO1bJdxY8SeP=P-PsZqXYToFeTs95Q7iFScKuykNtbbA@mail.gmail.com>

Hi all,

I'm going to be working on refactoring the IPython.testing machinery for a
while. If you could hold off on making any other major changes in that
package for a bit, we'll avoid conflicts. If fixes are necessary, apply
them and ping me so that I can make sure I don't cause a regression.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130904/8d4821b0/attachment.html>

From takowl at gmail.com  Fri Sep  6 12:22:19 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 6 Sep 2013 09:22:19 -0700
Subject: [IPython-dev] Another satisfied user
Message-ID: <CAOvn4qgNM5G4PF6B6D4r7Mn3E2YOw8_9YpbwLCUFM0cS33HtLw@mail.gmail.com>

I know we like to keep an eye on what people are saying, so here's another
'First impressions of IPython Notebook' post (from July, but I don't
remember seeing it before):

http://www.pgbovine.net/ipython-notebook-first-impressions.htm

As in most cases, he's enthusiastic but has plenty of suggestions for what
he'd like to see.

Discussion (also positive) on HN:
https://news.ycombinator.com/item?id=6339744
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130906/c14e613f/attachment.html>

From fsaldan1 at gmail.com  Sat Sep  7 08:20:24 2013
From: fsaldan1 at gmail.com (FernandoSaldanha)
Date: Sat, 7 Sep 2013 05:20:24 -0700 (PDT)
Subject: [IPython-dev] Width of pandas DataFrames in IPython
Message-ID: <1378556424068-5031058.post@n6.nabble.com>

I am exploring IPython under Spyder and I found that it prints pandas
DataFrames in nicely formatted tables. Much better than the normal printing,
which has the column names misaligned with the columns. Two questions:

1) When the number of columns is more than just a few IPython does not print
the contents, it just shows a summary. This happens even if there is plenty
of real estate to show more columns. Is there a way to tell IPython to
increase the number of columns to be displayed?

2) In Spyder one can choose to have IPython show graphics inline or in
separate windows. This is a very nice feature. Sometimes I like to have
graphs in separate windows so I don't have to scroll back to see them. Can
one do the same with the tables that display pandas DataFrames?

Thanks for the help.

FS



--
View this message in context: http://python.6.x6.nabble.com/Width-of-pandas-DataFrames-in-IPython-tp5031058.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From wirth.jason at gmail.com  Sat Sep  7 09:02:39 2013
From: wirth.jason at gmail.com (Jason Wirth)
Date: Sat, 7 Sep 2013 08:02:39 -0500
Subject: [IPython-dev] Width of pandas DataFrames in IPython
In-Reply-To: <1378556424068-5031058.post@n6.nabble.com>
References: <1378556424068-5031058.post@n6.nabble.com>
Message-ID: <CAEwvNMiZXt0G_4L4eWokpR9NyCZYnKTwM17AnARqhzPEvdt8dA@mail.gmail.com>

I think what you are looking for is the print options. Try setting the
max_columns for a higher number.

Example:

    pandas.set_option('display.max_columns', 50)


Documentation on set_option.
http://pandas.pydata.org/pandas-docs/stable/basics.html?highlight=set_option


Here's a similar StackOverflow question which also mentions the terminal
size. Note, they use a depricated method, `set_printoptions'.


-- 
Jason Wirth
    213.675.5294
    wirth.jason at gmail.com


On Sat, Sep 7, 2013 at 7:20 AM, FernandoSaldanha <fsaldan1 at gmail.com> wrote:

> I am exploring IPython under Spyder and I found that it prints pandas
> DataFrames in nicely formatted tables. Much better than the normal
> printing,
> which has the column names misaligned with the columns. Two questions:
>
> 1) When the number of columns is more than just a few IPython does not
> print
> the contents, it just shows a summary. This happens even if there is plenty
> of real estate to show more columns. Is there a way to tell IPython to
> increase the number of columns to be displayed?
>
> 2) In Spyder one can choose to have IPython show graphics inline or in
> separate windows. This is a very nice feature. Sometimes I like to have
> graphs in separate windows so I don't have to scroll back to see them. Can
> one do the same with the tables that display pandas DataFrames?
>
> Thanks for the help.
>
> FS
>
>
>
> --
> View this message in context:
> http://python.6.x6.nabble.com/Width-of-pandas-DataFrames-in-IPython-tp5031058.html
> Sent from the IPython - Development mailing list archive at Nabble.com.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130907/3ad47278/attachment.html>

From mattpap at gmail.com  Sun Sep  8 15:54:14 2013
From: mattpap at gmail.com (Mateusz Paprocki)
Date: Sun, 8 Sep 2013 21:54:14 +0200
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
Message-ID: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>

Hi,

I would like to announce IScala: a Scala-language backend for IPython,
see [1]. Scala is general purpose, object-functional, statically typed
programming language for JVM. IScala is an early work in progress, but
it's already fairly usable. All three types of frontends are
supported. Required dependencies are IPython 1.0+ and Java Runtime
Environment 1.6+. See README.md for installation and usage details.

Motivation for this project came when I saw IJulia announcement. The
hard part - actually "interpreting" Scala - was done in Scala REPL by
Scala developers. Scala is a compiled language, so interpreting it
means that source code is compiled (with some fancy wrappers),
resulting class files loaded into JVM and code run via Java
reflection. Interpreted code behaves exactly the same as compiled code
and it runs at full speed. There are a few issues with this, however.
Scala is a pure object-oriented language, so at top level you can put
only packages, traits, classes and singleton objects. No expressions
and other types of statements are allowed. Interpreter removes this
restriction, by wrapping source code in singleton objects, so that you
can simply write e.g. `val x = 1` to define a value, which is
otherwise illegal.

During development of IScala I also created a set of notebooks with
Scala code, see [2]. Those are based on a subset of Twitter's Scala
lessons [3]. I didn't encode notebooks directly in the JSON-based
format, but I invented an adhoc format based on Markdown with support
for cells. There is a simple converter that allows to generate
IPython-compatible notebooks.

[1] https://github.com/mattpap/IScala
[2] https://github.com/mattpap/scala_school_notebooks
[3] https://github.com/twitter/scala_school

Mateusz


From bussonniermatthias at gmail.com  Sun Sep  8 16:49:49 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 8 Sep 2013 22:49:49 +0200
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
Message-ID: <3BA872EE-D012-4789-9D0D-166D25AB381F@gmail.com>

Hi, 

Great to see that, happy that you were able to bind to IPython frontend.

You might be able to enable scala highlighting in notebook pretty easily
as code mirror support it. 
Ask if you ned help.

As you might know (or not) we will do some changes in the protocol/notebook 
format for next release, you might want to follow that, and give us your input. 

Especially if there are python-ism  that are annoying for you. 
(we'll try to ping you on relevant issues if we think of it.)

> I didn't encode notebooks directly in the JSON-based
> format, but I invented an adhoc format based on Markdown with support
> for cells.

Any reason for that ? think your were missing/being annoyed in json ?
Having  notebook as ipynb should allow you to render them 
on nbviewer/using nbconvert. 

We already have kind-of something alike that import .py files tagged with comments 
but without support four 'outputs' block as I think it does not make sense to manually edit 
output. It might be moved into nbconvert at some points.

Thanks, 

-- 
Matthias


Le 8 sept. 2013 ? 21:54, Mateusz Paprocki a ?crit :

> Hi,
> 
> I would like to announce IScala: a Scala-language backend for IPython,
> see [1]. Scala is general purpose, object-functional, statically typed
> programming language for JVM. IScala is an early work in progress, but
> it's already fairly usable. All three types of frontends are
> supported. Required dependencies are IPython 1.0+ and Java Runtime
> Environment 1.6+. See README.md for installation and usage details.
> 
> [...]
> 
> During development of IScala I also created a set of notebooks with
> Scala code, see [2]. Those are based on a subset of Twitter's Scala
> lessons [3]. I didn't encode notebooks directly in the JSON-based
> format, but I invented an adhoc format based on Markdown with support
> for cells. There is a simple converter that allows to generate
> IPython-compatible notebooks.
> 
> [1] https://github.com/mattpap/IScala
> [2] https://github.com/mattpap/scala_school_notebooks
> [3] https://github.com/twitter/scala_school
> 
> Mateusz
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From mattpap at gmail.com  Mon Sep  9 12:08:10 2013
From: mattpap at gmail.com (Mateusz Paprocki)
Date: Mon, 9 Sep 2013 18:08:10 +0200
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <3BA872EE-D012-4789-9D0D-166D25AB381F@gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
	<3BA872EE-D012-4789-9D0D-166D25AB381F@gmail.com>
Message-ID: <CAGBZUCboVjAH8nzz6C=mi+CNdjf=-eZ00P2N195jhFnmqg5XjA@mail.gmail.com>

Hi,

On 8 September 2013 22:49, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
> Hi,
>
> Great to see that, happy that you were able to bind to IPython frontend.
>
> You might be able to enable scala highlighting in notebook pretty easily
> as code mirror support it.
> Ask if you ned help.

The support in CodeMirror is rather limited (only clike mode with
Scala keywords), but sufficient for now. However, I don't see any
wiring in notebook, because it doesn't take advantage of information
sent in kernel_info message, unless I'm missing something.

> As you might know (or not) we will do some changes in the protocol/notebook
> format for next release, you might want to follow that, and give us your input.
>
> Especially if there are python-ism  that are annoying for you.

There are quite a few things to fix/improve. I will gradually start
issues for those (first I have to review collected material and see
what was already submitted by IJulia team, because there is obviously
a lot of overlap).

> (we'll try to ping you on relevant issues if we think of it.)

I'm watching this mailing list and issues so I shouldn't miss anything.

>> I didn't encode notebooks directly in the JSON-based
>> format, but I invented an adhoc format based on Markdown with support
>> for cells.
>
> Any reason for that ? think your were missing/being annoyed in json ?
> Having  notebook as ipynb should allow you to render them
> on nbviewer/using nbconvert.
>
> We already have kind-of something alike that import .py files tagged with comments
> but without support four 'outputs' block as I think it does not make sense to manually edit
> output. It might be moved into nbconvert at some points.

It's easier for me to work with a text editor (vim), but editing JSON
is far from being optimal. I also wanted to encode information about
different types of outputs, so that results, stdout and stderr weren't
mixed together. Finally, the original files were written in Textile
(just a few vim macros were needed to convert them to Markdown) with
source code embedded and I didn't want to rewrite it further (i.e.
having Scala files with text embedded in comments). Generated ipynb
files can be found on gh-pages branch so you can use nbviewer on them,
or just view generated html files.

> Thanks,
>
> --
> Matthias
>
>
> Le 8 sept. 2013 ? 21:54, Mateusz Paprocki a ?crit :
>
>> Hi,
>>
>> I would like to announce IScala: a Scala-language backend for IPython,
>> see [1]. Scala is general purpose, object-functional, statically typed
>> programming language for JVM. IScala is an early work in progress, but
>> it's already fairly usable. All three types of frontends are
>> supported. Required dependencies are IPython 1.0+ and Java Runtime
>> Environment 1.6+. See README.md for installation and usage details.
>>
>> [...]
>>
>> During development of IScala I also created a set of notebooks with
>> Scala code, see [2]. Those are based on a subset of Twitter's Scala
>> lessons [3]. I didn't encode notebooks directly in the JSON-based
>> format, but I invented an adhoc format based on Markdown with support
>> for cells. There is a simple converter that allows to generate
>> IPython-compatible notebooks.
>>
>> [1] https://github.com/mattpap/IScala
>> [2] https://github.com/mattpap/scala_school_notebooks
>> [3] https://github.com/twitter/scala_school
>>
>> Mateusz
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Mateusz


From bussonniermatthias at gmail.com  Mon Sep  9 12:27:21 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 9 Sep 2013 18:27:21 +0200
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <CAGBZUCboVjAH8nzz6C=mi+CNdjf=-eZ00P2N195jhFnmqg5XjA@mail.gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
	<3BA872EE-D012-4789-9D0D-166D25AB381F@gmail.com>
	<CAGBZUCboVjAH8nzz6C=mi+CNdjf=-eZ00P2N195jhFnmqg5XjA@mail.gmail.com>
Message-ID: <43610D9A-3712-4C4B-94DF-B29F9D187FAC@gmail.com>


Le 9 sept. 2013 ? 18:08, Mateusz Paprocki a ?crit :

> Hi,
> 
> On 8 September 2013 22:49, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> Hi,
>> 
>> Great to see that, happy that you were able to bind to IPython frontend.
>> 
>> You might be able to enable scala highlighting in notebook pretty easily
>> as code mirror support it.
>> Ask if you ned help.
> 
> The support in CodeMirror is rather limited (only clike mode with
> Scala keywords), but sufficient for now. However, I don't see any
> wiring in notebook, because it doesn't take advantage of information
> sent in kernel_info message, unless I'm missing something.

well when you create a profile you can inject javascript (custom.js) that modify this : 

https://github.com/ipython/ipython/blob/master/IPython/html/static/notebook/js/codecell.js#L94

IPython.CodeCell.options_default.cm_default.mode = 'scala' // or whatever

Load scala mode fro codemirror. (if it c-like it slightly more complicated, but CM doc is nice for that)
And optionally (but we should fix that), loop through already existing cells to set their mode to Scala
(otherwise hightailing will be off unit you click once on the cell). 

There are a few other things  that IJulia does in its profile like allowing bang (!) as valid char in tokens for tooltips.
as well as adding metadata to the notebook metadata field telling the the notebook is julia (or scala in your case)
for future v3 > v4 conversion where language will be a notebook top level key.

-- 
Matthias









-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130909/963d17e4/attachment.html>

From asmeurer at gmail.com  Mon Sep  9 18:01:30 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 9 Sep 2013 16:01:30 -0600
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <CAGBZUCboVjAH8nzz6C=mi+CNdjf=-eZ00P2N195jhFnmqg5XjA@mail.gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
	<3BA872EE-D012-4789-9D0D-166D25AB381F@gmail.com>
	<CAGBZUCboVjAH8nzz6C=mi+CNdjf=-eZ00P2N195jhFnmqg5XjA@mail.gmail.com>
Message-ID: <CAKgW=6+F7xCxrfqUFFQE6byzhHwgctm49v2Pshquq=UNjN5HMw@mail.gmail.com>

On Mon, Sep 9, 2013 at 10:08 AM, Mateusz Paprocki <mattpap at gmail.com> wrote:
> Hi,
>
> On 8 September 2013 22:49, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> Hi,
>>
>> Great to see that, happy that you were able to bind to IPython frontend.
>>
>> You might be able to enable scala highlighting in notebook pretty easily
>> as code mirror support it.
>> Ask if you ned help.
>
> The support in CodeMirror is rather limited (only clike mode with
> Scala keywords), but sufficient for now. However, I don't see any
> wiring in notebook, because it doesn't take advantage of information
> sent in kernel_info message, unless I'm missing something.
>
>> As you might know (or not) we will do some changes in the protocol/notebook
>> format for next release, you might want to follow that, and give us your input.
>>
>> Especially if there are python-ism  that are annoying for you.
>
> There are quite a few things to fix/improve. I will gradually start
> issues for those (first I have to review collected material and see
> what was already submitted by IJulia team, because there is obviously
> a lot of overlap).
>
>> (we'll try to ping you on relevant issues if we think of it.)
>
> I'm watching this mailing list and issues so I shouldn't miss anything.
>
>>> I didn't encode notebooks directly in the JSON-based
>>> format, but I invented an adhoc format based on Markdown with support
>>> for cells.
>>
>> Any reason for that ? think your were missing/being annoyed in json ?
>> Having  notebook as ipynb should allow you to render them
>> on nbviewer/using nbconvert.
>>
>> We already have kind-of something alike that import .py files tagged with comments
>> but without support four 'outputs' block as I think it does not make sense to manually edit
>> output. It might be moved into nbconvert at some points.
>
> It's easier for me to work with a text editor (vim), but editing JSON
> is far from being optimal. I also wanted to encode information about
> different types of outputs, so that results, stdout and stderr weren't
> mixed together. Finally, the original files were written in Textile
> (just a few vim macros were needed to convert them to Markdown) with
> source code embedded and I didn't want to rewrite it further (i.e.
> having Scala files with text embedded in comments). Generated ipynb
> files can be found on gh-pages branch so you can use nbviewer on them,
> or just view generated html files.

There's already an IPython notebook for Emacs. Someone just needs to
write one for vim. Then you can edit notebooks in vim natively.

Aaron Meurer

>
>> Thanks,
>>
>> --
>> Matthias
>>
>>
>> Le 8 sept. 2013 ? 21:54, Mateusz Paprocki a ?crit :
>>
>>> Hi,
>>>
>>> I would like to announce IScala: a Scala-language backend for IPython,
>>> see [1]. Scala is general purpose, object-functional, statically typed
>>> programming language for JVM. IScala is an early work in progress, but
>>> it's already fairly usable. All three types of frontends are
>>> supported. Required dependencies are IPython 1.0+ and Java Runtime
>>> Environment 1.6+. See README.md for installation and usage details.
>>>
>>> [...]
>>>
>>> During development of IScala I also created a set of notebooks with
>>> Scala code, see [2]. Those are based on a subset of Twitter's Scala
>>> lessons [3]. I didn't encode notebooks directly in the JSON-based
>>> format, but I invented an adhoc format based on Markdown with support
>>> for cells. There is a simple converter that allows to generate
>>> IPython-compatible notebooks.
>>>
>>> [1] https://github.com/mattpap/IScala
>>> [2] https://github.com/mattpap/scala_school_notebooks
>>> [3] https://github.com/twitter/scala_school
>>>
>>> Mateusz
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> Mateusz
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From benjaminrk at gmail.com  Mon Sep  9 19:14:28 2013
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 9 Sep 2013 16:14:28 -0700
Subject: [IPython-dev] [ANNOUNCE] IPython 1.1.0 release
Message-ID: <CAHNn8BWUYFc00-xhBXdmCkRZJBxBHqeoqnS5Sv1WmPfebrnKdg@mail.gmail.com>

Mostly bugfix release for IPython 1.0

Fixes various issues found in the first 30 days of the 1.0 release.
Upgrade is recommended for all users of 1.0.

The release on GitHub<https://github.com/ipython/ipython/releases/rel-1.1.0>
, PyPI <https://pypi.python.org/pypi/ipython/1.1.0>, and
archive.ipython.org<http://archive.ipython.org/release/1.1.0>
.

Check the docs<http://ipython.org/ipython-doc/stable/whatsnew/github-stats-1.0.html#issues-closed-in-1-1>to
see what changes have been backported from master.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130909/ef21e9ac/attachment.html>

From wstein at gmail.com  Tue Sep 10 12:11:50 2013
From: wstein at gmail.com (William Stein)
Date: Tue, 10 Sep 2013 09:11:50 -0700
Subject: [IPython-dev] ipython with sync
Message-ID: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>

Hi,

Somewhat by accident I spent the last two weeks implementing hosted
IPython notebooks with sync for https://cloud.sagemath.com.
Initially I had just plan to simplify the port forwarding setup, since
what Ondrej Certik was doing with multiple forward and reverse port
forwards seemed complicated.  But then I became concerned about
multiple users (or users with multiple browsers) overwriting each
other's notebooks, because cloud.sagemath projects are frequently
shared between multiple people, and everything else does realtime
sync.    I had planned just to add some very minimal merge-on-save
functionality to avoid major issues, but somehow got sucked into
realtime sync (even with the other person's cursor showing).

It would be enormously helpful to me if a couple of expert IPython
users were to try out what I implemented and just ask a bunch of
questions.

1. Go to https://cloud.sagemath.com and make an account; this is
completely free, and is hosted on computers at University of
Washington.

2. Create a new project.

3. Click +New, then click "IPython" (or paste in a link to an ipython
notebook, or upload a file).

4. An IPython notebook server will start, the given .ipynb file should
load in a same-domain iframe, and then some of the ipython notebook
code is and iframe contents are monkey patched, in order to support
sync and better integration with https://cloud.sagemath.com.

5. Open the ipynb file in multiple browsers, and see that changes in
one appear in the other, including moving cells around, creating new
cells, editing markdown (the rendered version appears elsewhere), etc.
  Anything that sets the notebook.dirty flag in IPython causes a sync
(evaluating a cell that creates no output doesn't set this flag, at
least in 1.0.0, which is a bug in IPython, I guess).

Since this is all very new and the first (I guess) realtime sync
implementation on top of IPython, there are probably a lot of issues.
 Note that if you click the "i" info button to the right, you'll get a
link to the standard IPython

The other thing of interest is a little Python script called
"ipython-notebook", which I wrote.  It basically makes it easy to run
an IPython notebook server as a daemon, get the port it is running on,
etc.  It's pretty simple but satisfies my use case, and has
JSON-output, to make it web friendly.    As I've written it, my script
passes several base_url options through by default, which are needed
for cloud.sagemath.  Anyway, I've attached it to this email (with a
BSD license) in case there is any interest.

Regarding the monkey patching in 4 above, the right thing to do would
be to explain exactly what hooks/changes in the IPython html client I
need in order to do sync, etc., make sure these makes sense to the
IPython devs, and send a pull request (or have a coding sprint in
Seattle or Berkeley?).  As an example, in order to do sync
*efficiently*, I have to be able to set a given cell from JSON -- it's
critical to do this in place when possible, since the overhead of
creating a new cell is huge (due probably to the overhead of creating
CodeMirror editors); however, the fromJSON method in IPython assumes
that the cell is brand new -- it would be nice to add an option to
make a cell fromJSON without assuming it is empty.

The ultimate outcome of this could be a clean well-defined way of
doing sync for IPython notebooks using any third-party sync
implementation.  IPython might provide their own sync service and
there are starting to be others available these days -- e.g., Google
has one: https://developers.google.com/drive/realtime/, and maybe
Guido van Rosum helped write one for Dropbox recently?

Subdirectories:   I noticed, incidentally, that the wakari version of
the IPython notebook server allows one to load ipynb files that are in
any subdirectory, whereas the standard IPython notebook server
doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
server for each directory that a user accesses files in right now.
This seems cludgy, so I'm interested in the situation regarding adding
support for subdirectories.

-- William



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-notebook
Type: application/octet-stream
Size: 6553 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130910/20225a2f/attachment.obj>

From takowl at gmail.com  Tue Sep 10 12:30:00 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 10 Sep 2013 09:30:00 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
Message-ID: <CAOvn4qjftpz8AL_E1q9__aTJ4a=aXPEqbg60B2zKi_Rpoc1Q9w@mail.gmail.com>

Hi William,

Thanks for your work on this. I'll leave better qualified people to answer
most of your questions, but I can respond to this one:

On 10 September 2013 09:11, William Stein <wstein at gmail.com> wrote:

> For cloud.sagemath, I just spawn a new IPython notebook
> server for each directory that a user accesses files in right now.
> This seems cludgy, so I'm interested in the situation regarding adding
> support for subdirectories.
>

It's a work in progress, but it requires quite a bit of re-architecting to
get it to work properly. It's still a few weeks away from being merged, but
we hope to ship it in 2.0 some time this winter. Zach's pull request for it
is here:

https://github.com/ipython/ipython/pull/3619

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130910/93d83376/attachment.html>

From fperez.net at gmail.com  Tue Sep 10 15:54:04 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 10 Sep 2013 12:54:04 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
Message-ID: <CAHAreOpHNeHX6UVBAA_sFaFhA-X9y5snT_4ZHrrUBDVTPvygWA@mail.gmail.com>

Hi William,

On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:

> 5. Open the ipynb file in multiple browsers, and see that changes in
> one appear in the other, including moving cells around, creating new
> cells, editing markdown (the rendered version appears elsewhere), etc.
>   Anything that sets the notebook.dirty flag in IPython causes a sync
> (evaluating a cell that creates no output doesn't set this flag, at
> least in 1.0.0, which is a bug in IPython, I guess).

Wow, very impressive!  At the very first cell I saw some glitches: I
simply typed '1' in the first empty cell and ran it; it produced '1'
as output but the input got deleted and copied to the next (new) cell.
Weird.  It didn't happen again.

Overall it seems to work, though for now I've only tested it very
lightly.  The startup problem I saw yesterday seems gone.

> Since this is all very new and the first (I guess) realtime sync
> implementation on top of IPython, there are probably a lot of issues.
>  Note that if you click the "i" info button to the right, you'll get a
> link to the standard IPython

My first question is: to what extent does this rely on your
surrounding node.js machinery and to what extent is it something
self-contained that could be used by the current IPython design?  This
is obviously something of high interest to us (and oft-requested by
users), so it would be good to understand what parts of the design
would make sense to upstream and what parts are deeply intertwined
with the entire sagemath architecture.

> The other thing of interest is a little Python script called
> "ipython-notebook", which I wrote.  It basically makes it easy to run
> an IPython notebook server as a daemon, get the port it is running on,
> etc.  It's pretty simple but satisfies my use case, and has
> JSON-output, to make it web friendly.    As I've written it, my script
> passes several base_url options through by default, which are needed
> for cloud.sagemath.  Anyway, I've attached it to this email (with a
> BSD license) in case there is any interest.

On that front, there are two related projects worth mentioning:

https://github.com/UnataInc/ipydra: along the lines of yours but more elaborate.

https://github.com/ptone/jiffylab: docker-based full containerization
of a notebook+shell.

> Regarding the monkey patching in 4 above, the right thing to do would
> be to explain exactly what hooks/changes in the IPython html client I
> need in order to do sync, etc., make sure these makes sense to the
> IPython devs, and send a pull request (or have a coding sprint in
> Seattle or Berkeley?).  As an example, in order to do sync
> *efficiently*, I have to be able to set a given cell from JSON -- it's
> critical to do this in place when possible, since the overhead of
> creating a new cell is huge (due probably to the overhead of creating
> CodeMirror editors); however, the fromJSON method in IPython assumes
> that the cell is brand new -- it would be nice to add an option to
> make a cell fromJSON without assuming it is empty.

Yup, this seems like a sensible idea and I can imagine us using it in
many other places, actually.

> The ultimate outcome of this could be a clean well-defined way of
> doing sync for IPython notebooks using any third-party sync
> implementation.  IPython might provide their own sync service and
> there are starting to be others available these days -- e.g., Google
> has one: https://developers.google.com/drive/realtime/, and maybe
> Guido van Rosum helped write one for Dropbox recently?

Thanks for that Google link, I hadn't seen this one.  I remember
they'd opened up an older implementation of their Wave stuff, but when
I looked at it, it was a huge Java monster so it didn't look like we
could reuse any of it.  It would be interesting to consider relying on
an external service for syncing as an option, but honestly it seems to
me that in the end we should put that into the IPython server itself.
If nothing else, for latency reasons in locally hosted servers I can
imagine it making a big difference.

> Subdirectories:   I noticed, incidentally, that the wakari version of
> the IPython notebook server allows one to load ipynb files that are in
> any subdirectory, whereas the standard IPython notebook server
> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
> server for each directory that a user accesses files in right now.
> This seems cludgy, so I'm interested in the situation regarding adding
> support for subdirectories.

See Thomas' response.  We hope to have this merged before too long,
and it will be a huge improvement for everyone.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 10 15:59:07 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 10 Sep 2013 12:59:07 -0700
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
Message-ID: <CAHAreOrrBtq+7qkn88JFfCbEFV97DH5bFA_2bt=j+RU_3Q2HaQ@mail.gmail.com>

Hi Mateusz,


On Sun, Sep 8, 2013 at 12:54 PM, Mateusz Paprocki <mattpap at gmail.com> wrote:
> Hi,
>
> I would like to announce IScala: a Scala-language backend for IPython,
> see [1]. Scala is general purpose, object-functional, statically typed
> programming language for JVM. IScala is an early work in progress, but
> it's already fairly usable. All three types of frontends are
> supported. Required dependencies are IPython 1.0+ and Java Runtime
> Environment 1.6+. See README.md for installation and usage details.

This is great! Just last week I was playing a bit with Scala, this
will make my life much easier :)

> Motivation for this project came when I saw IJulia announcement. The
> hard part - actually "interpreting" Scala - was done in Scala REPL by
> Scala developers. Scala is a compiled language, so interpreting it
> means that source code is compiled (with some fancy wrappers),
> resulting class files loaded into JVM and code run via Java
> reflection. Interpreted code behaves exactly the same as compiled code
> and it runs at full speed. There are a few issues with this, however.
> Scala is a pure object-oriented language, so at top level you can put
> only packages, traits, classes and singleton objects. No expressions
> and other types of statements are allowed. Interpreter removes this
> restriction, by wrapping source code in singleton objects, so that you
> can simply write e.g. `val x = 1` to define a value, which is
> otherwise illegal.

Very interesting: I've been thinking precisely about this problem in
the context of creating an IPython kernel for Go, and wondering what
it would take.  Go doesn't have the hardcore OO stance of the JVM, but
some similar issues regarding persistence of state and compilation
arise. The lessons you've learned here would likely be very useful.

> During development of IScala I also created a set of notebooks with
> Scala code, see [2]. Those are based on a subset of Twitter's Scala
> lessons [3]. I didn't encode notebooks directly in the JSON-based
> format, but I invented an adhoc format based on Markdown with support
> for cells. There is a simple converter that allows to generate
> IPython-compatible notebooks.

I would recommend using the 'real' IPython notebook format as your
default, so that you can benefit automatically from the entire rest of
our machinery (nbconvert, nbviewer, etc).

Our long term view is for the format to be completely language
agnostic, so we're willing to make any changes and fixes to the
underlying format to accomodate Scala cleanly.  We're already looking
at a bunch of such fixes for Julia, and having another language at
hand will help us correctly identify the right abstraction points and
how to leave generality in the base format while allowing each
language to represent its own information with all the detail and
precision desired.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 10 16:01:32 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 10 Sep 2013 13:01:32 -0700
Subject: [IPython-dev] Another satisfied user
In-Reply-To: <CAOvn4qgNM5G4PF6B6D4r7Mn3E2YOw8_9YpbwLCUFM0cS33HtLw@mail.gmail.com>
References: <CAOvn4qgNM5G4PF6B6D4r7Mn3E2YOw8_9YpbwLCUFM0cS33HtLw@mail.gmail.com>
Message-ID: <CAHAreOoMD+oNBKhDf--PH_+3RT+R_+LCXV_OGyKN=ULVAp0eCQ@mail.gmail.com>

Hi,

yes, Philip is a great and very thoughtful guy. I met him years ago
when he was a PhD student and our conversations on this topic started
back then; he creatively ran with some ideas from those conversations
and did great work. HIs recollections can be found in his little
memoir (p64 and elsewhere):

http://www.pgbovine.net/PhD-memoir/pguo-PhD-grind.pdf

I'm sure we'll continue collaborating with Philip in the years to come.

Cheers,

f

On Fri, Sep 6, 2013 at 9:22 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> I know we like to keep an eye on what people are saying, so here's another
> 'First impressions of IPython Notebook' post (from July, but I don't
> remember seeing it before):
>
> http://www.pgbovine.net/ipython-notebook-first-impressions.htm
>
> As in most cases, he's enthusiastic but has plenty of suggestions for what
> he'd like to see.
>
> Discussion (also positive) on HN:
> https://news.ycombinator.com/item?id=6339744
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From wstein at gmail.com  Tue Sep 10 16:09:01 2013
From: wstein at gmail.com (William Stein)
Date: Tue, 10 Sep 2013 13:09:01 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CAHAreOpHNeHX6UVBAA_sFaFhA-X9y5snT_4ZHrrUBDVTPvygWA@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
	<CAHAreOpHNeHX6UVBAA_sFaFhA-X9y5snT_4ZHrrUBDVTPvygWA@mail.gmail.com>
Message-ID: <CACLE5GCxV7+1iJWM_c0yzr1vookhrWyv_nqZDZ0khC6bBbudPw@mail.gmail.com>

On Tue, Sep 10, 2013 at 12:54 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi William,
>
> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>
>> 5. Open the ipynb file in multiple browsers, and see that changes in
>> one appear in the other, including moving cells around, creating new
>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>> (evaluating a cell that creates no output doesn't set this flag, at
>> least in 1.0.0, which is a bug in IPython, I guess).
>
> Wow, very impressive!  At the very first cell I saw some glitches: I
> simply typed '1' in the first empty cell and ran it; it produced '1'
> as output but the input got deleted and copied to the next (new) cell.
> Weird.  It didn't happen again.

You should have seen had bad the overall sync infrastructure was in
May... I've fixed a lot of issues with it, but it's still not perfect
in all cases, and I'm using some new features for the IPython stuff.
Hopefully the above sort of race condition should be very rare.  If
you can find anything reproducible, please let me know.

> Overall it seems to work, though for now I've only tested it very
> lightly.  The startup problem I saw yesterday seems gone.
>
>> Since this is all very new and the first (I guess) realtime sync
>> implementation on top of IPython, there are probably a lot of issues.
>>  Note that if you click the "i" info button to the right, you'll get a
>> link to the standard IPython
>
> My first question is: to what extent does this rely on your
> surrounding node.js machinery and to what extent is it something
> self-contained that could be used by the current IPython design?  This
> is obviously something of high interest to us (and oft-requested by
> users), so it would be good to understand what parts of the design
> would make sense to upstream and what parts are deeply intertwined
> with the entire sagemath architecture.

I just quickly wrote up a rough draft of a blog post, which I had
planned to post at http://www.blogger.com, but evidently that
(Google-owned) site is broken for login, for me at least.     So just
see the 1-page "How It Works" section here:

   http://wstein.org/tmp/2013-09-10-104600-ipython.html

>> The other thing of interest is a little Python script called
>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>> an IPython notebook server as a daemon, get the port it is running on,
>> etc.  It's pretty simple but satisfies my use case, and has
>> JSON-output, to make it web friendly.    As I've written it, my script
>> passes several base_url options through by default, which are needed
>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>> BSD license) in case there is any interest.
>
> On that front, there are two related projects worth mentioning:
>
> https://github.com/UnataInc/ipydra: along the lines of yours but more elaborate.
>
> https://github.com/ptone/jiffylab: docker-based full containerization
> of a notebook+shell.
>
>> Regarding the monkey patching in 4 above, the right thing to do would
>> be to explain exactly what hooks/changes in the IPython html client I
>> need in order to do sync, etc., make sure these makes sense to the
>> IPython devs, and send a pull request (or have a coding sprint in
>> Seattle or Berkeley?).  As an example, in order to do sync
>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>> critical to do this in place when possible, since the overhead of
>> creating a new cell is huge (due probably to the overhead of creating
>> CodeMirror editors); however, the fromJSON method in IPython assumes
>> that the cell is brand new -- it would be nice to add an option to
>> make a cell fromJSON without assuming it is empty.
>
> Yup, this seems like a sensible idea and I can imagine us using it in
> many other places, actually.
>
>> The ultimate outcome of this could be a clean well-defined way of
>> doing sync for IPython notebooks using any third-party sync
>> implementation.  IPython might provide their own sync service and
>> there are starting to be others available these days -- e.g., Google
>> has one: https://developers.google.com/drive/realtime/, and maybe
>> Guido van Rosum helped write one for Dropbox recently?
>
> Thanks for that Google link, I hadn't seen this one.  I remember
> they'd opened up an older implementation of their Wave stuff, but when
> I looked at it, it was a huge Java monster so it didn't look like we
> could reuse any of it.  It would be interesting to consider relying on
> an external service for syncing as an option, but honestly it seems to
> me that in the end we should put that into the IPython server itself.
> If nothing else, for latency reasons in locally hosted servers I can
> imagine it making a big difference.

It might be possible to do both, with the right design and hooks.   I
know you guys like to really think things through carefully and get
things right in a long-term general and flexible way.

>
>> Subdirectories:   I noticed, incidentally, that the wakari version of
>> the IPython notebook server allows one to load ipynb files that are in
>> any subdirectory, whereas the standard IPython notebook server
>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>> server for each directory that a user accesses files in right now.
>> This seems cludgy, so I'm interested in the situation regarding adding
>> support for subdirectories.
>
> See Thomas' response.  We hope to have this merged before too long,
> and it will be a huge improvement for everyone.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From fperez.net at gmail.com  Tue Sep 10 17:24:09 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 10 Sep 2013 14:24:09 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CACLE5GCxV7+1iJWM_c0yzr1vookhrWyv_nqZDZ0khC6bBbudPw@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
	<CAHAreOpHNeHX6UVBAA_sFaFhA-X9y5snT_4ZHrrUBDVTPvygWA@mail.gmail.com>
	<CACLE5GCxV7+1iJWM_c0yzr1vookhrWyv_nqZDZ0khC6bBbudPw@mail.gmail.com>
Message-ID: <CAHAreOpNGk0AnK+Qb08TwaHvP_CpmbBnFOwURf4PYVDnLyMdOw@mail.gmail.com>

On Tue, Sep 10, 2013 at 1:09 PM, William Stein <wstein at gmail.com> wrote:
> On Tue, Sep 10, 2013 at 12:54 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> You should have seen had bad the overall sync infrastructure was in
> May... I've fixed a lot of issues with it, but it's still not perfect
> in all cases, and I'm using some new features for the IPython stuff.
> Hopefully the above sort of race condition should be very rare.  If
> you can find anything reproducible, please let me know.

Will do. This looks great already, and I can see myself using it a lot.

> I just quickly wrote up a rough draft of a blog post, which I had
> planned to post at http://www.blogger.com, but evidently that
> (Google-owned) site is broken for login, for me at least.     So just
> see the 1-page "How It Works" section here:
>
>    http://wstein.org/tmp/2013-09-10-104600-ipython.html

Excellent, thanks for that info. Question: is your implementation pure
node.js?  And how interwoven is it with the rest of your server
architecture for cloudsage?

> It might be possible to do both, with the right design and hooks.   I
> know you guys like to really think things through carefully and get
> things right in a long-term general and flexible way.

Yes, for better or worse :)  But I'm very happy to be having this
conversation already. Even though real-time sync isn't on our
immediate todo path (we have our hands more than full with nbconvert
and our interactive JS api, obviously influenced by Sage's experiences
with @interact), it will be great to start thinking about this with
your experience and implementation already blazing the trail.

Cheers,

f


From abie at uw.edu  Tue Sep 10 20:15:23 2013
From: abie at uw.edu (Abraham D. Flaxman)
Date: Wed, 11 Sep 2013 00:15:23 +0000
Subject: [IPython-dev] Using IPython Cluster with SGE -- help needed
In-Reply-To: <51FFB46F.3060309@hilboll.de>
References: <51FE6282.7090104@hilboll.de>
	<CAHCaCkKWtSGa5mx4M9wnJArdAV00M6NFAb=MgU-NQeSDx_HxTg@mail.gmail.com>
	<51FFAC64.5070908@hilboll.de>
	<CAHCaCkKr0DKkbGF3-O_WffgqyJh2ecchV8BAK_yCKkJLoVRPWg@mail.gmail.com>
	<51FFB46F.3060309@hilboll.de>
Message-ID: <F723135E32F9984ABC5B614F28BF24910179FECF@UWIT-MBX06.exchange.washington.edu>

Have you had more success with this Andreas?  I've tried to use my SGE today as well, and had moderate success.  I started where you were:

    ipcluster start --profile=sge -n 12  # starts one engine, sometimes

which I can confirm via:
In [48]:  len(p.Client(profile='sge'))
Out[48]: 1

Then I qsub more engines with:

    for i in {1..100}; do qsub sge.engine.template; done  # starts 101 more

and this gives me more remote clients:

In [49]: len(p.Client(profile='sge'))
Out[49]: 101

To shut down the cluster:
qstat | awk {'print $1'} | xargs qdel

Also, it shut down on its own one time, which I appreciated, but perhaps was not intended.

Any tips on how I can make this work more smoothly?

--Abie


Abraham D. Flaxman
Assistant Professor
Institute for Health Metrics and Evaluation | University of Washington
2301 5th Avenue, Suite 600 | Seattle, WA 98121| USA
Tel: +1-206-897-2800 | Fax: +1-206-897-2899 UW
abie at uw.edu | http://healthmetricsandevaluation.org | http://healthyalgorithms.com



-----Original Message-----
From: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] On Behalf Of Andreas Hilboll
Sent: Monday, August 05, 2013 7:19 AM
To: Matthieu Brucher
Cc: IPython developers list
Subject: Re: [IPython-dev] Using IPython Cluster with SGE -- help needed

Thanks, Mathieu,

answers inline:

Am 05.08.2013 16:02, schrieb Matthieu Brucher:
> Hi,
> 
> I don't know why the registration was not complete. Is your home 
> folder the same on all nodes and on the login node?

Yes, it is. Could this be some firewall issue?

> You won't see 12 jobs. You asked for 12 engines, and they will all be 
> submitted in one job and the 12 engines will be started by mpiexec -n 
> 12. This is the standard way of using batch schedulers. Ask for some 
> cores, run an mpi application on these cores.

Well, then I guess our IT department doesn't like "the standard way". We have a multi-node cluster, comprising 12 nodes, one 'management' and 11 'computing' nodes. And we don't have/use mpi usually.

What I would need in order to use our multi-node cluster the way our sysadmins want us to, I'd need to submit a total of {n} ipengines via {n} calls to ``qsub``.

Any idea how I can accomplish this?

Thanks for your help!
Andreas.



From gmbecker at ucdavis.edu  Tue Sep 10 21:32:12 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Tue, 10 Sep 2013 18:32:12 -0700
Subject: [IPython-dev] Some new cell types for describing data analyses
 in IPy. Notebook
In-Reply-To: <CAH4pYpT4TBgHgqgZXLEN48+GWBk3Lk-_y+NY-QboFC+LH0t=rg@mail.gmail.com>
References: <CADwqtCNRb3EK7g6E80XXMSoY+QLrF1BaAdCm2yMhjWbr-Y-KzQ@mail.gmail.com>
	<CAHNn8BUZy5+0sMFiT9sFwnD4gUPKhZ-Mfv4W_6Q9jaOOHGBhwQ@mail.gmail.com>
	<CADwqtCN_NyjPBmny511LC8LkTTfo5cpAbBFQR5y_-wC9s6M=xw@mail.gmail.com>
	<CAHNn8BVP_0_o98ryMCeNEHjdPdq009zDu9majmrpXAbfhBrZgQ@mail.gmail.com>
	<F33ADD32-33D0-456A-9436-5FAE915FCA81@gmail.com>
	<CADwqtCM11DurMQFNd1PB+h_n_wMWHy1kR_d25V7VeJsQrY4LXw@mail.gmail.com>
	<CAH4pYpST2b81nVz5WJWMt6PpfqoGhRuOmw_wyKUDZFqGh=+7=Q@mail.gmail.com>
	<CADwqtCPQNrT95oZcTfXp=XG=avhWsj=fgjyzhHuc36w3nFESDA@mail.gmail.com>
	<CAH4pYpQjs_tTNLxD1c6fO7aYHgTYmgUpFPkGinQ8jOufHDzhYQ@mail.gmail.com>
	<CADwqtCP-cJxFqde_9YwsvzdOSRfx_tkrVcTxaE69VxU+Ni43Rw@mail.gmail.com>
	<CAH4pYpT4TBgHgqgZXLEN48+GWBk3Lk-_y+NY-QboFC+LH0t=rg@mail.gmail.com>
Message-ID: <CADwqtCPzJUTjYSkLQq-UGX+_CBZYvRr=1_0psgZE_yw0=eDWQQ@mail.gmail.com>

Brian et al,

Brian I hope your move/travel/etc was as pleasant as such things can be.


On Fri, Jul 12, 2013 at 9:21 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> Gabriel,
> <snip>
>
> Great, let's talk in Sept. to figure out a time that would work.
>

I'm still quite interested in meeting with you guys. Somewhere near the end
of the month would be best for me, but I'm pretty flexible.


> <snip>
>
>  > Branching/DAG notebooks allow a single document to encompass the
> research
> > you did, while providing easy access to various views corresponding to
> the
> > generation of intermediate, alternative, and final results.
> >
> > These more complex notebooks allow the viewer to ask and answer important
> > questions such as "What else did (s)he try here?" and potentially even
> "Why
> > did (s)he choose this particular analysis strategy?". These questions
> can be
> > answered in the text or external supplementary materials in a linear
> > notebook, but this is a significant barrier to reproducibility of the
> > research process (as opposed to the analysis results).
>
> I can see that, however, I think the pure alt cells lack a critical
> feature.  They treat all branches as being equally important.  In
> reality, the branch that is chosen as the "best" one will likely
> require further analysis and discussion that that other branches
> don't.  Putting the different branches side by side makes it a little
> like "choose your own adventure" - when in reality, the author of the
> research want to steer the reader along a very particular path.  The
> alternative paths maybe useful to have around, but they should be be
> given equal weight as the "best" one.  But, maybe it is just
> presentation and can be accounted for in descriptive text.
>

This is very true. My current thinking calls for both a "default"
designation and a "most recently selected/run" designation, which I believe
deals with the valid concern you raise above.

There are also other important designations for "branch types". The most
notable/easily explained of these is the concept of a "terminal" branch,
which is a branch that records important computations (and prose), and
which a viewer of the notebook  (be it the original author, a reviewer, a
student, or someone looking to extend the work) may want to look at or run,
but whose output is not compatible with the subsequent computations. This
arises most commonly when one analysis strategy is implemented and pursued,
but ultimately abandoned  (hopefully for good reasons, and with this we can
check!) in favor of a different final strategy which produces incompatible
output. The subsequent code then makes assumptions about the output which
are compatible with the final strategy computations, but not the original
strategy ones. A way to gracefully deal with this case is important for any
document/processing/rendering system attempting to pursue these concepts.

There are other cases that arise with these documents, but I will omit a
detailed discussion of them and what I think should be done to support them
here, as that would make this mail burdensomely long and it is not my
primary message.

I will note, though, that while I agree that the final/core/whathaveyou and
secondary/informative/archival branches should not be indistinguishable, it
is important for my usecase that they be easily accessible when the reader
wants to in both interactive (notebook) and headless (nbconvert) modes.


> <snip>
>
> > As a practical/UI standpoint unselected branches can be hidden almost
> > entirely (in theory, not currently in my PoC :p), resulting in a view
> > equivalent to (any) the only view offered by a linear notebook. This
> means
> > that from a viewer (and author since a straight line IS a DAG and nesting
> > isn't forced) standpoint, what I'm describing is in essense a strict
> > extension of what the notebook does now, rather than a change.
>
> I would be *more* interested in alt-cell approaches that present the
> notebook as a linear entity in all cases, but that has the alt-cell
> logic underneath.  For example, what about the following:
>
> * A user writes the different N alt cells in linear sequence
> * The result is a purely linear notebook where one of the N cells should
> be run.
> * We write a JavaScript plugin for the notebook that does a couple of
> things:
>
> 1. It provides a cell toolbar for marking those cells as members of an
> alt-set.  This would simple modify the cell level metadata and allow
> the author to provide titles of each alt-member.
>

What about branching that is 2 or more levels deep? That happens naturally
with my approach but sounds difficult/annoying to keep track of in the one
you are describing.


> 2. It provides the logic for building a UI for viewing one of the
> alt-set members at a time.  It could be as simple as injecting a drop
> down menu that shows one and hides the rest.
>

I have an ugly but functional version of this now in my implementation.


>
> * This plugin could simple walk the notebook cells and find all the
> alt-cell sets and build this supplementary UI.
> * This plugin could also have settings that allow the author to select
> the "best" member of the alt-set.
> * nbconvert Transformers could use the cell level metadata to export
> the notebook in different formats.
>
> As I write about this - I think this would be extremely nice, and it
> would not be difficult to write at all.  Because of how our JavaScript
> plugins work, it could be developed outside IPython initially.  The
> question of inclusion in the official code base could be handled
> later.  Honestly, this approach should be much easier than the work
> you have already done.
>

Well, editing the notebook once it exists in this form seems like it would
be much less fun, in terms of adding new cells.

What you're describing is also much more onerous for the author. With what
I have now, you declare a cell to be an altset or task and everything just
sort of works. New cells are inserted in the right places, cells trivially
know who their parents are, etc.

If I understand you correctly, the author would have to write all the
alternatives in a big linear document (not fun or easy to test, see
discussion below) and then click a bunch of buttons to manually select what
cells go in which alternate. That is a much larger cognitive burden on the
author (as well as probably being really annoying...).


>
> Best of all the resulting notebooks would remain standard linear
> notebooks that could be shared today on nbviewer, etc.  It would just
> work.
>

Respectfully, this is actually the fatal flaw of this approach IMO, both in
this case and in other cases where a JS plugin/extension uses the metadata
approach to fundamentally modify *behavior* (as opposed to aestethics/UI)
of the IPython Notebook.

The issue, stated in the context of the nesting/alts/etc cells extension,
is that a notebook that has branching/alternates *requires* that they be
understood as such, rather than simply benefiting from it.

The ability to distribute notebooks I write and have them work properly is
entirely core to my usecase for IPython. If I can't do so, what I
personally can get IPython or IPython notebooks to do on my own machine is
not something I have any real interest in. Now you may be thinking to
yourself "But Gabe, no one is using your fork so you can't do that now with
your implementation anyway". That is true, but if someone without my fork
installed manages to get their hands on a notebook which uses the nesting
features, it will break when they try to load it.

If I create an extension as you are describing, create a complex notebook
using it, and someone without the plugin installed finds it, downloads it,
and runs it, it will *run fine and happily give them incorrect results
without even noticing the extra bits I stuck in the metadata*.

The core issue here is that running a notebook with branching as a linear
notebook by executing each of the branches in sequence is actually
erroneous and will produce undefined, untrustworthy, and likely incorrect,
behavior and output. The reason for this is that branches/alternatives are
assumed to be mutually exclusive by the computational model, and can alter
objects in-place in manners that can have unintended cumulative effects.

As a very simple example consider branches which handle outliers in a
certain variable by modifying the variable in-place and trimming its
values  by .1, 1, 5, and 10%, respectively,  using quantiles and then
consider what would happen if these branches were all run in an arbitrary
order.

It is easy to see that the outcome from running all the branches (which is
what will silently happen if the notebook is treated as a standard linear
notebook because the plugin is not being used) does not reflect any of the
choices intended by the author and more complex situations could be
difficult to predict at all without sitting down and thinking about it.

As such, I would not be comfortable distributing branching notebooks using
the extension mechanism as I understand it to exist now because a) I feel
it indirectly damages the type of scientific reprodicibility and result
trustworthiness I seek to advance, and b) I don't want to spend all my time
fielding angry emails/bugreports from notebook authors who sent their
notebooks to collaborators who didn't have the plugin installed.



>
> <snip>
>
> > Consider the example of classifying new data based on a training set via
> > KNN, SVM, and GLM approaches. These approaches all need different sets of
> > parameters, return different types of objects as the output of the
> fitting
> > function, may have subtley different behaviour when being used for
> > prediction, etc.
>
> Yep, that is the big challenge with the branching idea in general.  It
> is not always true that the members of the alt sets can be swapped
> out.
>

And under the model I am envisioning, that is actually an informative  and
queriable feature, rather than a drawback. See my discussion above
regarding terminal branches.


>
> <snip>
>
> I hope you can see that I really like the general idea and think the
> usage cases you are describing are really important.  I think I can
> speak for the project in saying that we want the notebook to be useful
> for things like this.  But I think our abstractions are important
> enough that we make every attempt to see how we can do these while
> leveraging our existing abstractions.  This is partially a question
> about implementation, but also partly a question about how the new
> features are thought about.  The reason we don't like to break
> abstractions for new features is that we have found an interesting
> relationship between abstraction breaking and new features.  We have
> found that when a new feature/idea breaks a core abstraction that we
> have thought about very carefully, it is usually because the feature
> has not been fully understood.  Time and time again, we have found
> that when we take the time to fully understand the feature, it usually
> fits within our abstractions beautifully and is even much better that
> we ever imagined it could be.
>
> The plugin idea above is a perfect example of this.  By preserving the
> abstractions the new feature itself a multiplication of even new
> functionality:
>
> * The resulting notebooks can still be version controlled.  This means
> that the different alt-cell can be thrown into git and when we develop
> a visual diff tool for notebooks, they will *just work*.
>

I don't really understand this point. I have numerous fork-based non-linear
notebooks under version control.

Also, when you have a visual diff tool, it will successfully do
*something*when given a linear+metadata branching notebook, but
whether that something
would be to deliver the information required to understand changes to
non-linear notebooks  is less clear (and seems somewhat unlikely).


> * The notebooks can immediately leverage the abstractions we have put
> into place for converting notebooks to different formats.  You could
> write custom transformers to present the notebook in a reveal.js
> giving alt-cells special treatment.
>


I could write custom transformers, this is true, but the default behavior
would treat the notebook as if it actually were linear (instead of just
being stored that way) which is problematic.



> * All of this can be done, and into the hands of user, without going
> through those overly conservative IPython developers ;-)
> * It will just work with nbviewer as well.
>

Again, I disagree. It would *display* in nbviewer, but not work, in that
the display would be actively misleading regarding what the notebook would
do when executed properly.


>  * It provides a cleanly abstracted foundation for other people to build
> upon
>

I agree that this is important, but it is not clear to me that it would be
more true in the case that I created the extension via custom JS than it
would if nesting were supported in the actual ipynb format and core
notebook mechanisms.


>
> In summary, we are trying to build an architecture that allows a few
> simple abstractions (we actually don't have that many!) to combine in
> boundless ways to create features we never planned on, but that "just
> work".
>

I agree that the customjs + metadata extensions approach is very powerful
and almost infinitely versatile. I think it is great for extensions which
change appearance/rendering/UI details of how the notebook behaves.

As far as I can see, however,  it has some signficant problems with regard
to extensions which fundamentally change non-rendering behavior of
notebooks (please correct me if I'm wrong), namely:

   - There is no guarantee that notebooks authored using an extension which
   alters fundamental behaviors will work or visibly fail in the absence of
   that extension
   - There is no way for an individual notebook to require a particular
   extension
   - There is no way to ensure that two extensions are compatible with
   each-other
   - There is no standard/unified way for end-users to install extensions
   - There is no way for users to determine which extensions they have

The first point is not true of extensions which exclusively affect
rendering and UI, making the rest of the points minor nuisances rather than
critical issues.
Looking forward to hearing your (further) thoughts about this stuff and
hopefully meeting you in person soon.

~G

-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130910/8e576523/attachment.html>

From gvwilson at third-bit.com  Wed Sep 11 11:19:41 2013
From: gvwilson at third-bit.com (Greg Wilson)
Date: Wed, 11 Sep 2013 11:19:41 -0400
Subject: [IPython-dev] refactoring magics
Message-ID: <52308A0D.8000207@third-bit.com>

Hi,
Matt Davis and I have written a small magic that allows people to enter 
a regular expression and a few lines of text, and displays the matches 
in that text.  For example, if a user enters:

     %%regex ab+
     There are about three matches
     in all of this text: xab and xabbbbbyz are two of them

then the 'ab' in 'about', the 'ab' in 'xab', and the 'abbbbb' in 
'xabbbbbyz' are highlighted in the output.  (see 
https://github.com/gvwilson/regexmagic for the magic).  We'd like to use 
this to teach regular expressions in Software Carpentry in place of 
http://regexpal.com, but there's a problem.  Right now, this regular 
expression:

     \d{4}

doesn't work, because IPython interprets '{4}' as "expand the macro 
named '4'", which results in an empty string.  We can get around it by 
doubling up the parens and writing it as:

     \d{{4}}

but our main reason for creating this tool in the first place is to 
reduce the number of things we need to explain away when teaching novices.

An alternative is to refactor the code that runs magics so that they 
have both the original text (without macro expansion) and either the 
expanded text or a way to get it.  The key pieces of code are 
run_line_magic and run_cell_magic in IPython/core/interactiveshell.py.  
It's easy enough to change these to use the raw line instead of the 
expanded line, but I'd welcome suggestions on how to allow magic authors 
to get the expansions. Options I see are:

1. Leave it completely in the magic writer's hands.  I don't like this, 
because it requires magic writers to know that they need to go down 
exactly three stack frames in order to do expansion properly. (Right 
now, both run_line_magic and run_cell_magic need to go exactly two 
frames, no more and no less, to expand variables properly.)

2. Do the expansion in run_line_magic and run_cell_magic and pass in 
both the raw line and the expanded line.  This is the smallest change to 
interactiveshell.py, but requires changes to the signatures of magic 
functions (they need an extra parameter).

3. Provide an API function that users can call inside their magic that 
takes the raw line and returns the expanded version if they want it.  
The problem is, this function would then have to somehow know how far up 
the stack to go to get to the right frame (unless we def'd it in the 
calling code and passed it into the magic, in which case we might as 
well use option #2).

I'd welcome suggestions on which route people think is best, or whether 
there's a better way that I'm missing.
Thanks,
Greg


From bussonniermatthias at gmail.com  Wed Sep 11 14:16:22 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Wed, 11 Sep 2013 20:16:22 +0200
Subject: [IPython-dev] refactoring magics
In-Reply-To: <52308A0D.8000207@third-bit.com>
References: <52308A0D.8000207@third-bit.com>
Message-ID: <39289215-9654-47BB-9A57-B7EDDCBF1A1D@gmail.com>




> 1. Leave it completely in the magic writer's hands. 
> 
> 2. Do the expansion in run_line_magic and run_cell_magic and pass in 
> both the raw line and the expanded line. ....
> 
> 3. Provide an API function that users can call inside their magic...
> 
> I'd welcome suggestions on which route people think is best, or whether 
> there's a better way that I'm missing.
> Thanks,
> Greg


What about a decorator ? It does not give author much choice.

Or have the register_magics function take an kwarg which tell it wether or not the string will be expanded.
This one might allow us not to change the current api. 
-- 
M

From takowl at gmail.com  Wed Sep 11 14:49:08 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 11 Sep 2013 11:49:08 -0700
Subject: [IPython-dev] refactoring magics
In-Reply-To: <39289215-9654-47BB-9A57-B7EDDCBF1A1D@gmail.com>
References: <52308A0D.8000207@third-bit.com>
	<39289215-9654-47BB-9A57-B7EDDCBF1A1D@gmail.com>
Message-ID: <CAOvn4qhs9uo-wNpLdnROD3kmJQiGZtUaBwzf3AEJR1ArKf3ngg@mail.gmail.com>

On 11 September 2013 11:16, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> What about a decorator ? It does not give author much choice.


I think the best way is to use the @needs_local_scope decorator, and change
the api of var_expand a bit. The reason that var_expand needs a stack depth
is so that it can find the local scope, if the magic is called within a
function (which can't happen for a cell magic, in any case). We can easily
get a reference to the global user namespace.

With the @needs_local_scope decorator, the magic function gets a keyword
argument local_ns with the namespace of the function from which it was
called. It could then pass that to var_expand() instead of a stack depth.
We're already using that decorator, e.g. for %time and %Rpush, so this
seems like the simplest approach.

Have we thought about which magic functions should continue to expand
variables?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130911/7ea7dbce/attachment.html>

From jason-sage at creativetrax.com  Thu Sep 12 10:54:15 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 12 Sep 2013 09:54:15 -0500
Subject: [IPython-dev] notebook code editor
Message-ID: <5231D597.2050507@creativetrax.com>

Is there a way to edit a plain python file from the IPython notebook?

I was talking to another person about using the IPython notebook to 
collaborate on a code library (using git, github, pull requests, etc). 
The notebook format doesn't work here because git merges can mess it up, 
etc.  I think they'd like to use the notebook to interact with the code 
(i.e., run functions, draw plots, etc.).  It would be cool if they could 
stay in the notebook to actually edit the library code.  What I'm 
imagining is that the initial notebook screen lists ipynb files and .py 
files.  Opening a .py file just opens a huge CodeMirror instance with 
the file and a save button at the top.

Thoughts?

Thanks,

Jason



From franz.bergesund at gmail.com  Thu Sep 12 11:27:01 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Thu, 12 Sep 2013 17:27:01 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBKKvdRdzkpJ3q2S8nhmvwbK0EQ4FEaZ7O-BbEAg0oudxyQ@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
	<CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
	<CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>
	<CAOCdBKKvdRdzkpJ3q2S8nhmvwbK0EQ4FEaZ7O-BbEAg0oudxyQ@mail.gmail.com>
Message-ID: <CAOCdBK+niuxCJ_C2K-piZdGUYw_WtVYb1kFRcTsQoiFJECkP+Q@mail.gmail.com>

Dear Stefan,

Sorry for not replying before. It does work now.

Thanks,

Francesco


2013/9/2 Francesco Montesano <franz.bergesund at gmail.com>

> Thanks,
>
> I'll give a try tomorrow. not it's getting late
> Do you know if HTML understands b'...'?
>
> Fra
>
>
>
> 2013/9/2 St?fan van der Walt <stefan at sun.ac.za>
>
>> On Mon, Sep 2, 2013 at 10:31 PM, Francesco Montesano
>> <franz.bergesund at gmail.com> wrote:
>> > thank you very much for the example. This should work wonderfully for
>> python
>> > 2.x., but, as Zoltan pointed out, it won't work on python 3.x, as you
>> can't
>> > concatenate string and bytes objects (returned by base64 methods).
>>
>> That shouldn't be too much of a problem:
>>
>> In [12]: bytes('asd'.encode('utf-8')) + base64.b64encode(b'asd')
>> Out[12]: b'asdYXNk'
>>
>> In [13]: (bytes('asd'.encode('utf-8')) +
>> base64.b64encode(b'asd')).decode('utf-8')
>> Out[13]: 'asdYXNk'
>>
>> Have a look at "numpy.compat" for some utility functions as well.
>>
>> St?fan
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130912/e69f7cc1/attachment.html>

From rgbkrk at gmail.com  Thu Sep 12 11:37:47 2013
From: rgbkrk at gmail.com (Kyle Kelley)
Date: Thu, 12 Sep 2013 10:37:47 -0500
Subject: [IPython-dev] notebook code editor
In-Reply-To: <5231D597.2050507@creativetrax.com>
References: <5231D597.2050507@creativetrax.com>
Message-ID: <CA+tbMaXjBGbzv8n1V8+9pr34NViP3g9-7m0Q9jJrrpUco2jh=Q@mail.gmail.com>

Sounds cool, but as far as I know that functionality doesn't exist (yet).

Would be pretty rad though.


On Thu, Sep 12, 2013 at 9:54 AM, Jason Grout <jason-sage at creativetrax.com>wrote:

> Is there a way to edit a plain python file from the IPython notebook?
>
> I was talking to another person about using the IPython notebook to
> collaborate on a code library (using git, github, pull requests, etc).
> The notebook format doesn't work here because git merges can mess it up,
> etc.  I think they'd like to use the notebook to interact with the code
> (i.e., run functions, draw plots, etc.).  It would be cool if they could
> stay in the notebook to actually edit the library code.  What I'm
> imagining is that the initial notebook screen lists ipynb files and .py
> files.  Opening a .py file just opens a huge CodeMirror instance with
> the file and a save button at the top.
>
> Thoughts?
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130912/7ab0a2d4/attachment.html>

From franz.bergesund at gmail.com  Thu Sep 12 11:53:21 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Thu, 12 Sep 2013 17:53:21 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <CAOCdBK+niuxCJ_C2K-piZdGUYw_WtVYb1kFRcTsQoiFJECkP+Q@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
	<CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
	<CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>
	<CAOCdBKKvdRdzkpJ3q2S8nhmvwbK0EQ4FEaZ7O-BbEAg0oudxyQ@mail.gmail.com>
	<CAOCdBK+niuxCJ_C2K-piZdGUYw_WtVYb1kFRcTsQoiFJECkP+Q@mail.gmail.com>
Message-ID: <CAOCdBKJ4NF3Hi28bLT0LesDBkPE8y8Etzg1y6tyR+rLdB1-=Cw@mail.gmail.com>

ehm.
It does work with python 2. With python 3.3 I get in the notebook the
symbols of brocken images and the string of the png printed in the console

Fra


2013/9/12 Francesco Montesano <franz.bergesund at gmail.com>

> Dear Stefan,
>
> Sorry for not replying before. It does work now.
>
> Thanks,
>
> Francesco
>
>
> 2013/9/2 Francesco Montesano <franz.bergesund at gmail.com>
>
>> Thanks,
>>
>> I'll give a try tomorrow. not it's getting late
>> Do you know if HTML understands b'...'?
>>
>> Fra
>>
>>
>>
>> 2013/9/2 St?fan van der Walt <stefan at sun.ac.za>
>>
>>> On Mon, Sep 2, 2013 at 10:31 PM, Francesco Montesano
>>> <franz.bergesund at gmail.com> wrote:
>>> > thank you very much for the example. This should work wonderfully for
>>> python
>>> > 2.x., but, as Zoltan pointed out, it won't work on python 3.x, as you
>>> can't
>>> > concatenate string and bytes objects (returned by base64 methods).
>>>
>>> That shouldn't be too much of a problem:
>>>
>>> In [12]: bytes('asd'.encode('utf-8')) + base64.b64encode(b'asd')
>>> Out[12]: b'asdYXNk'
>>>
>>> In [13]: (bytes('asd'.encode('utf-8')) +
>>> base64.b64encode(b'asd')).decode('utf-8')
>>> Out[13]: 'asdYXNk'
>>>
>>> Have a look at "numpy.compat" for some utility functions as well.
>>>
>>> St?fan
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130912/da995c2a/attachment.html>

From benjaminrk at gmail.com  Thu Sep 12 12:31:17 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 12 Sep 2013 09:31:17 -0700
Subject: [IPython-dev] notebook code editor
In-Reply-To: <CA+tbMaXjBGbzv8n1V8+9pr34NViP3g9-7m0Q9jJrrpUco2jh=Q@mail.gmail.com>
References: <5231D597.2050507@creativetrax.com>
	<CA+tbMaXjBGbzv8n1V8+9pr34NViP3g9-7m0Q9jJrrpUco2jh=Q@mail.gmail.com>
Message-ID: <CAHNn8BUjUuROPXbotB=td-R0oGpcZKg62_DbJCaKi6gGM4b_Xw@mail.gmail.com>

Yes, we have for quite some time planned a simple single-page CodeMirror
view for a file.
It requires the url-scheme reorganization in IPEP 16.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130912/8b5d2211/attachment.html>

From zvoros at gmail.com  Thu Sep 12 12:58:17 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Thu, 12 Sep 2013 18:58:17 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
 side
In-Reply-To: <CAOCdBKJ4NF3Hi28bLT0LesDBkPE8y8Etzg1y6tyR+rLdB1-=Cw@mail.gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
	<CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
	<CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>
	<CAOCdBKKvdRdzkpJ3q2S8nhmvwbK0EQ4FEaZ7O-BbEAg0oudxyQ@mail.gmail.com>
	<CAOCdBK+niuxCJ_C2K-piZdGUYw_WtVYb1kFRcTsQoiFJECkP+Q@mail.gmail.com>
	<CAOCdBKJ4NF3Hi28bLT0LesDBkPE8y8Etzg1y6tyR+rLdB1-=Cw@mail.gmail.com>
Message-ID: <5231F2A9.40601@gmail.com>

But I have told you that there is an issue in python 3.3. I sent you a 
link in one of my earlier e-mails. In that link you should also find the 
fix.

Cheers,
Zolt?n

On 12/09/13 17:53, Francesco Montesano wrote:
> ehm.
> It does work with python 2. With python 3.3 I get in the notebook the 
> symbols of brocken images and the string of the png printed in the 
> console
>
> Fra
>
>
> 2013/9/12 Francesco Montesano <franz.bergesund at gmail.com 
> <mailto:franz.bergesund at gmail.com>>
>
>     Dear Stefan,
>
>     Sorry for not replying before. It does work now.
>
>     Thanks,
>
>     Francesco
>
>
>     2013/9/2 Francesco Montesano <franz.bergesund at gmail.com
>     <mailto:franz.bergesund at gmail.com>>
>
>         Thanks,
>
>         I'll give a try tomorrow. not it's getting late
>         Do you know if HTML understands b'...'?
>
>         Fra
>
>
>
>         2013/9/2 St?fan van der Walt <stefan at sun.ac.za
>         <mailto:stefan at sun.ac.za>>
>
>             On Mon, Sep 2, 2013 at 10:31 PM, Francesco Montesano
>             <franz.bergesund at gmail.com
>             <mailto:franz.bergesund at gmail.com>> wrote:
>             > thank you very much for the example. This should work
>             wonderfully for python
>             > 2.x., but, as Zoltan pointed out, it won't work on
>             python 3.x, as you can't
>             > concatenate string and bytes objects (returned by base64
>             methods).
>
>             That shouldn't be too much of a problem:
>
>             In [12]: bytes('asd'.encode('utf-8')) +
>             base64.b64encode(b'asd')
>             Out[12]: b'asdYXNk'
>
>             In [13]: (bytes('asd'.encode('utf-8')) +
>             base64.b64encode(b'asd')).decode('utf-8')
>             Out[13]: 'asdYXNk'
>
>             Have a look at "numpy.compat" for some utility functions
>             as well.
>
>             St?fan
>             _______________________________________________
>             IPython-dev mailing list
>             IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>             http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130912/7626c600/attachment.html>

From ondrej.certik at gmail.com  Thu Sep 12 14:25:21 2013
From: ondrej.certik at gmail.com (=?UTF-8?B?T25kxZllaiDEjGVydMOtaw==?=)
Date: Thu, 12 Sep 2013 12:25:21 -0600
Subject: [IPython-dev] Fortran Magic for IPython
Message-ID: <CADDwiVBin-RnKtn=vC9+6B6=SYWu0J2FT9CG23Mbdg=GAK7N4A@mail.gmail.com>

Hi,

I developed Fortran magic for IPython:

https://github.com/certik/ipython_fortran

Follow the link in the README for an example notebook. Travis
suggested this idea to me yesterday in a pub. ;)

You should be able to run everything from:

http://fortran90.org/src/rosetta.html
http://fortran90.org/src/best-practices.html


Some notes:

* the syntax highlighting is wrong for Fortran, both in code cells an
markdown cells

* it'd be nice to automatically wrap via Cython some subroutines to
Python with %%fwrap magic, currently I just execute the code with the
%%fortran magic

* it'd be nice to allow to link my Fortran library in, so that I can
interactively test subroutines in it. So I use my current build system
for my library, and just tell the %%fortran magic to link it + paths
to *.mod files, so that I can import modules. That should be quite
robust/simple to do. The only requirement is to use the same compiler.

* what is the best way to setup different compilers and compiler
options in the %%fortran magic? The code is here:
https://github.com/certik/ipython_fortran/blob/master/fortranmagic.py,
it's really simple so far.

* all cells are currently independent. I don't think it makes sense to
try to somehow append and execute code together from multiple cells.
But it might make sense to reuse modules defined in different cells
somehow.

* Eventually I'd love to contribute a PR against IPython notebook with
this extension, but for now I am just polishing it in the above
repository.

I would appreciate any feedback.

Ondrej


From damianavila at gmail.com  Thu Sep 12 14:35:35 2013
From: damianavila at gmail.com (=?UTF-8?B?RGFtacOhbiBBdmlsYQ==?=)
Date: Thu, 12 Sep 2013 15:35:35 -0300
Subject: [IPython-dev] Fortran Magic for IPython
In-Reply-To: <CADDwiVBin-RnKtn=vC9+6B6=SYWu0J2FT9CG23Mbdg=GAK7N4A@mail.gmail.com>
References: <CADDwiVBin-RnKtn=vC9+6B6=SYWu0J2FT9CG23Mbdg=GAK7N4A@mail.gmail.com>
Message-ID: <52320977.6090802@gmail.com>

El 12/09/13 15:25, Ond?ej ?ert?k escribi?:
> Hi,
>
> I developed Fortran magic for IPython:
>
> https://github.com/certik/ipython_fortran
>
> Follow the link in the README for an example notebook. Travis
> suggested this idea to me yesterday in a pub. ;)
>
> You should be able to run everything from:
>
> http://fortran90.org/src/rosetta.html
> http://fortran90.org/src/best-practices.html
>
>
> Some notes:
>
> * the syntax highlighting is wrong for Fortran, both in code cells an
> markdown cells
>
> * it'd be nice to automatically wrap via Cython some subroutines to
> Python with %%fwrap magic, currently I just execute the code with the
> %%fortran magic
>
> * it'd be nice to allow to link my Fortran library in, so that I can
> interactively test subroutines in it. So I use my current build system
> for my library, and just tell the %%fortran magic to link it + paths
> to *.mod files, so that I can import modules. That should be quite
> robust/simple to do. The only requirement is to use the same compiler.
>
> * what is the best way to setup different compilers and compiler
> options in the %%fortran magic? The code is here:
> https://github.com/certik/ipython_fortran/blob/master/fortranmagic.py,
> it's really simple so far.
>
> * all cells are currently independent. I don't think it makes sense to
> try to somehow append and execute code together from multiple cells.
> But it might make sense to reuse modules defined in different cells
> somehow.
>
> * Eventually I'd love to contribute a PR against IPython notebook with
> this extension, but for now I am just polishing it in the above
> repository.
>
> I would appreciate any feedback.
>
> Ondrej
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Ondrej, some days ago Martin publish a fortran magic, he also makes a PR 
to give codemirror support for Fortran, maybe both can join forces... ;-)

https://github.com/mgaitan/fortran_magic

Regards.

Dami?n.




From wstein at gmail.com  Thu Sep 12 14:38:48 2013
From: wstein at gmail.com (William Stein)
Date: Thu, 12 Sep 2013 11:38:48 -0700
Subject: [IPython-dev] Fortran Magic for IPython
In-Reply-To: <52320977.6090802@gmail.com>
References: <CADDwiVBin-RnKtn=vC9+6B6=SYWu0J2FT9CG23Mbdg=GAK7N4A@mail.gmail.com>
	<52320977.6090802@gmail.com>
Message-ID: <CACLE5GBOOQ6foQ3m1+E9HZKnccmE+c1V+8yTOEB_86c6tbmxHQ@mail.gmail.com>

On Thu, Sep 12, 2013 at 11:35 AM, Dami?n Avila <damianavila at gmail.com> wrote:
> El 12/09/13 15:25, Ond?ej ?ert?k escribi?:
>> Hi,
>>
>> I developed Fortran magic for IPython:
>>
>> https://github.com/certik/ipython_fortran
>>
>> Follow the link in the README for an example notebook. Travis
>> suggested this idea to me yesterday in a pub. ;)
>>
>> You should be able to run everything from:
>>
>> http://fortran90.org/src/rosetta.html
>> http://fortran90.org/src/best-practices.html
>>
>>
>> Some notes:
>>
>> * the syntax highlighting is wrong for Fortran, both in code cells an
>> markdown cells
>>
>> * it'd be nice to automatically wrap via Cython some subroutines to
>> Python with %%fwrap magic, currently I just execute the code with the
>> %%fortran magic
>>
>> * it'd be nice to allow to link my Fortran library in, so that I can
>> interactively test subroutines in it. So I use my current build system
>> for my library, and just tell the %%fortran magic to link it + paths
>> to *.mod files, so that I can import modules. That should be quite
>> robust/simple to do. The only requirement is to use the same compiler.
>>
>> * what is the best way to setup different compilers and compiler
>> options in the %%fortran magic? The code is here:
>> https://github.com/certik/ipython_fortran/blob/master/fortranmagic.py,
>> it's really simple so far.
>>
>> * all cells are currently independent. I don't think it makes sense to
>> try to somehow append and execute code together from multiple cells.
>> But it might make sense to reuse modules defined in different cells
>> somehow.
>>
>> * Eventually I'd love to contribute a PR against IPython notebook with
>> this extension, but for now I am just polishing it in the above
>> repository.
>>
>> I would appreciate any feedback.
>>
>> Ondrej
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> Ondrej, some days ago Martin publish a fortran magic, he also makes a PR
> to give codemirror support for Fortran, maybe both can join forces... ;-)
>
> https://github.com/mgaitan/fortran_magic

And here's a direct link to his fortran mode for CodeMirror:

  https://github.com/mgaitan/CodeMirror/tree/be73b866e7381da6336b258f4aa75fb455623338/mode/fortran

>
> Regards.
>
> Dami?n.
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From ellisonbg at gmail.com  Thu Sep 12 17:53:19 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 12 Sep 2013 14:53:19 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
Message-ID: <CAH4pYpS0rEychGGcmu_zmdM-xtpJPqU+DmFtyc_8QP7938SjAw@mail.gmail.com>

William,

Thanks for notes.  This is really exciting and we are very interested
in getting live sync to work with IPython itself.  I am traveling this
week, but will try to have a look soon.  Is the code available
somewhere to look at?

Cheers,

Brian

On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
> Hi,
>
> Somewhat by accident I spent the last two weeks implementing hosted
> IPython notebooks with sync for https://cloud.sagemath.com.
> Initially I had just plan to simplify the port forwarding setup, since
> what Ondrej Certik was doing with multiple forward and reverse port
> forwards seemed complicated.  But then I became concerned about
> multiple users (or users with multiple browsers) overwriting each
> other's notebooks, because cloud.sagemath projects are frequently
> shared between multiple people, and everything else does realtime
> sync.    I had planned just to add some very minimal merge-on-save
> functionality to avoid major issues, but somehow got sucked into
> realtime sync (even with the other person's cursor showing).
>
> It would be enormously helpful to me if a couple of expert IPython
> users were to try out what I implemented and just ask a bunch of
> questions.
>
> 1. Go to https://cloud.sagemath.com and make an account; this is
> completely free, and is hosted on computers at University of
> Washington.
>
> 2. Create a new project.
>
> 3. Click +New, then click "IPython" (or paste in a link to an ipython
> notebook, or upload a file).
>
> 4. An IPython notebook server will start, the given .ipynb file should
> load in a same-domain iframe, and then some of the ipython notebook
> code is and iframe contents are monkey patched, in order to support
> sync and better integration with https://cloud.sagemath.com.
>
> 5. Open the ipynb file in multiple browsers, and see that changes in
> one appear in the other, including moving cells around, creating new
> cells, editing markdown (the rendered version appears elsewhere), etc.
>   Anything that sets the notebook.dirty flag in IPython causes a sync
> (evaluating a cell that creates no output doesn't set this flag, at
> least in 1.0.0, which is a bug in IPython, I guess).
>
> Since this is all very new and the first (I guess) realtime sync
> implementation on top of IPython, there are probably a lot of issues.
>  Note that if you click the "i" info button to the right, you'll get a
> link to the standard IPython
>
> The other thing of interest is a little Python script called
> "ipython-notebook", which I wrote.  It basically makes it easy to run
> an IPython notebook server as a daemon, get the port it is running on,
> etc.  It's pretty simple but satisfies my use case, and has
> JSON-output, to make it web friendly.    As I've written it, my script
> passes several base_url options through by default, which are needed
> for cloud.sagemath.  Anyway, I've attached it to this email (with a
> BSD license) in case there is any interest.
>
> Regarding the monkey patching in 4 above, the right thing to do would
> be to explain exactly what hooks/changes in the IPython html client I
> need in order to do sync, etc., make sure these makes sense to the
> IPython devs, and send a pull request (or have a coding sprint in
> Seattle or Berkeley?).  As an example, in order to do sync
> *efficiently*, I have to be able to set a given cell from JSON -- it's
> critical to do this in place when possible, since the overhead of
> creating a new cell is huge (due probably to the overhead of creating
> CodeMirror editors); however, the fromJSON method in IPython assumes
> that the cell is brand new -- it would be nice to add an option to
> make a cell fromJSON without assuming it is empty.
>
> The ultimate outcome of this could be a clean well-defined way of
> doing sync for IPython notebooks using any third-party sync
> implementation.  IPython might provide their own sync service and
> there are starting to be others available these days -- e.g., Google
> has one: https://developers.google.com/drive/realtime/, and maybe
> Guido van Rosum helped write one for Dropbox recently?
>
> Subdirectories:   I noticed, incidentally, that the wakari version of
> the IPython notebook server allows one to load ipynb files that are in
> any subdirectory, whereas the standard IPython notebook server
> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
> server for each directory that a user accesses files in right now.
> This seems cludgy, so I'm interested in the situation regarding adding
> support for subdirectories.
>
> -- William
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From wstein at gmail.com  Thu Sep 12 19:04:26 2013
From: wstein at gmail.com (William Stein)
Date: Thu, 12 Sep 2013 16:04:26 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CAH4pYpS0rEychGGcmu_zmdM-xtpJPqU+DmFtyc_8QP7938SjAw@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
	<CAH4pYpS0rEychGGcmu_zmdM-xtpJPqU+DmFtyc_8QP7938SjAw@mail.gmail.com>
Message-ID: <CACLE5GCM4LAdnGXGXhnE=ssSfU6_4ZNeQ+y0scBYB0gbZrT=Tg@mail.gmail.com>

On Thu, Sep 12, 2013 at 2:53 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> William,
>
> Thanks for notes.  This is really exciting and we are very interested
> in getting live sync to work with IPython itself.  I am traveling this
> week, but will try to have a look soon.  Is the code available
> somewhere to look at?

Unfortunately, this implementation is very entangled as part of
something I'm building as part of a startup company with UW's Center
for commercialization, and I can't just open source everything.  But
the algorithm is here:

  https://neil.fraser.name/writing/sync/

and the author of that paper has an open source Python implementation,
I think...


>
> Cheers,
>
> Brian
>
> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>> Hi,
>>
>> Somewhat by accident I spent the last two weeks implementing hosted
>> IPython notebooks with sync for https://cloud.sagemath.com.
>> Initially I had just plan to simplify the port forwarding setup, since
>> what Ondrej Certik was doing with multiple forward and reverse port
>> forwards seemed complicated.  But then I became concerned about
>> multiple users (or users with multiple browsers) overwriting each
>> other's notebooks, because cloud.sagemath projects are frequently
>> shared between multiple people, and everything else does realtime
>> sync.    I had planned just to add some very minimal merge-on-save
>> functionality to avoid major issues, but somehow got sucked into
>> realtime sync (even with the other person's cursor showing).
>>
>> It would be enormously helpful to me if a couple of expert IPython
>> users were to try out what I implemented and just ask a bunch of
>> questions.
>>
>> 1. Go to https://cloud.sagemath.com and make an account; this is
>> completely free, and is hosted on computers at University of
>> Washington.
>>
>> 2. Create a new project.
>>
>> 3. Click +New, then click "IPython" (or paste in a link to an ipython
>> notebook, or upload a file).
>>
>> 4. An IPython notebook server will start, the given .ipynb file should
>> load in a same-domain iframe, and then some of the ipython notebook
>> code is and iframe contents are monkey patched, in order to support
>> sync and better integration with https://cloud.sagemath.com.
>>
>> 5. Open the ipynb file in multiple browsers, and see that changes in
>> one appear in the other, including moving cells around, creating new
>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>> (evaluating a cell that creates no output doesn't set this flag, at
>> least in 1.0.0, which is a bug in IPython, I guess).
>>
>> Since this is all very new and the first (I guess) realtime sync
>> implementation on top of IPython, there are probably a lot of issues.
>>  Note that if you click the "i" info button to the right, you'll get a
>> link to the standard IPython
>>
>> The other thing of interest is a little Python script called
>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>> an IPython notebook server as a daemon, get the port it is running on,
>> etc.  It's pretty simple but satisfies my use case, and has
>> JSON-output, to make it web friendly.    As I've written it, my script
>> passes several base_url options through by default, which are needed
>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>> BSD license) in case there is any interest.
>>
>> Regarding the monkey patching in 4 above, the right thing to do would
>> be to explain exactly what hooks/changes in the IPython html client I
>> need in order to do sync, etc., make sure these makes sense to the
>> IPython devs, and send a pull request (or have a coding sprint in
>> Seattle or Berkeley?).  As an example, in order to do sync
>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>> critical to do this in place when possible, since the overhead of
>> creating a new cell is huge (due probably to the overhead of creating
>> CodeMirror editors); however, the fromJSON method in IPython assumes
>> that the cell is brand new -- it would be nice to add an option to
>> make a cell fromJSON without assuming it is empty.
>>
>> The ultimate outcome of this could be a clean well-defined way of
>> doing sync for IPython notebooks using any third-party sync
>> implementation.  IPython might provide their own sync service and
>> there are starting to be others available these days -- e.g., Google
>> has one: https://developers.google.com/drive/realtime/, and maybe
>> Guido van Rosum helped write one for Dropbox recently?
>>
>> Subdirectories:   I noticed, incidentally, that the wakari version of
>> the IPython notebook server allows one to load ipynb files that are in
>> any subdirectory, whereas the standard IPython notebook server
>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>> server for each directory that a user accesses files in right now.
>> This seems cludgy, so I'm interested in the situation regarding adding
>> support for subdirectories.
>>
>> -- William
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From jamesresearching at gmail.com  Fri Sep 13 06:38:30 2013
From: jamesresearching at gmail.com (James)
Date: Fri, 13 Sep 2013 19:38:30 +0900
Subject: [IPython-dev] Ipython parallel and PBS
Message-ID: <CAJtW7t=8SzOof6_oOqPS5==U7VZSH7ErC52e9ZPUrjAZ4LaqgQ@mail.gmail.com>

Dear all,

I'm having a lot of trouble setting up IPython parallel on a PBS cluster,
and I would really appreciate any help.

The architecture is a standard PBS cluster - a head node with slave nodes.
I connect to the head node from my laptop over ssh.

The client (laptop) -> Head node connection seems simple enough. The
problem is the engines.

Ignoring the laptop for a moment, I'll just focus on running ipython on the
head node, with the engines on a slave node. I assume this is a correct
method of working?

I did the following on the head node, following instructions at
http://ipython.org/ipython-doc/stable/parallel/parallel_process.html#using-ipcluster-in-pbs-mode:

$ ipython profile create --parallel --profile=pbs

Files are as follows:

$cat ipcluster_config.py
c = get_config()
c.IPClusterStart.controller_launcher_class = 'PBSControllerLauncher'
c.IPClusterEngines.engine_launcher_class = 'PBSEngineSetLauncher'
c.PBSLauncher.queue = 'long'
c.IPClusterEngines.n = 2 # Run 2 cores on 1 node or 2 nodes with all cores?
Not sure.

$ cat ipengine_config.py
c = get_config()

Then execute on the head node:
$ ipcluster start --profile=pbs -n 2
2013-09-10 15:02:46,771.771 [IPClusterStart] Using existing profile dir:
u'/home/username/.ipython/profile_pbs'
2013-09-10 15:02:46.777 [IPClusterStart] Starting ipcluster with
[daemon=False]
2013-09-10 15:02:46.778 [IPClusterStart] Creating pid file:
/home/username/.ipython/profile_pbs/pid/ipcluster.pid
2013-09-10 15:02:46.778 [IPClusterStart] Starting Controller with
PBSControllerLauncher
2013-09-10 15:02:46.792 [IPClusterStart] Job submitted with job id: '2830'
2013-09-10 15:02:47.793 [IPClusterStart] Starting 2 Engines with
PBSEngineSetLauncher
2013-09-10 15:02:47.808 [IPClusterStart] Job submitted with job id: '2831'

Then the queue shows
$ qstat
Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
2830[].master              ipcontroller     username              0 Q
long
2831[].master              ipengine         username              0 Q long

And they just hang there, queued forever. I assume the engines at least
should be running? Full information through "qstat -f" doesn't give the
reason for the queuing. Normally it would do. There are more than 4 nodes
available.

$qstat -f
Job Id: 2831[].master.domain
    Job_Name = ipengine
    Job_Owner = username at master.domain
    job_state = Q
    queue = long
    server = [head node's domain address]
    Checkpoint = u
    ctime = Tue Sep 10 15:02:47 2013
    Error_Path = master.domain:/home/username/
ipengine.e2831
    Hold_Types = n
    Join_Path = n
    Keep_Files = n
    Mail_Points = a
    mtime = Tue Sep 10 15:02:47 2013
    Output_Path = master.domain:/home/username/ipengine.o2831
    Priority = 0
    qtime = Tue Sep 10 15:02:47 2013
    Rerunable = True
    [...]
    etime = Tue Sep 10 15:02:47 2013
    submit_args = ./pbs_engines
    job_array_request = 1-2
    fault_tolerant = False
    submit_host = master.domain
    init_work_dir = /home/username

It also seems strange that the ipcontroller is launched through PBS. I
thought this should be on the head node, so I changed
'PBSControllerLauncher' to 'LocalControllerLauncher'. Then it doesn't
queue, but I don't know if what I'm doing is correct.

Any help would be really greatly appreciated.

Thank you.

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130913/501f82e6/attachment.html>

From franz.bergesund at gmail.com  Fri Sep 13 10:04:25 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Fri, 13 Sep 2013 16:04:25 +0200
Subject: [IPython-dev] [Notebook 1.0.0] Display two png images side by
	side
In-Reply-To: <5231F2A9.40601@gmail.com>
References: <CAOCdBKKSr1LCfmZ+SFLNTM8_YNit1qo9JmdKWuw556pFeAGe9w@mail.gmail.com>
	<CABDkGQ=Nhp0d9+ykroyZk5mvVEJF1jrneiAH3_-jgJgbgVvKOg@mail.gmail.com>
	<CAOCdBKLkPFueK=Qj9G3DG_wXuJ4Oc6AP55dZRm89EeAQ84S3ww@mail.gmail.com>
	<CABDkGQnsstGWDoWekBaBoijvW23rPz4Ssbj8k-XcsWyrrza07A@mail.gmail.com>
	<CAOCdBKLqAaB3Vd-53bVme4t0RJFZZNtqk1bucht+kz2kFK-8=w@mail.gmail.com>
	<CABDkGQmyLRb-aFqUfW_AyksBxGh-BDUkeQ9Wxw6_W-sjFUb8gg@mail.gmail.com>
	<CAOCdBKKvdRdzkpJ3q2S8nhmvwbK0EQ4FEaZ7O-BbEAg0oudxyQ@mail.gmail.com>
	<CAOCdBK+niuxCJ_C2K-piZdGUYw_WtVYb1kFRcTsQoiFJECkP+Q@mail.gmail.com>
	<CAOCdBKJ4NF3Hi28bLT0LesDBkPE8y8Etzg1y6tyR+rLdB1-=Cw@mail.gmail.com>
	<5231F2A9.40601@gmail.com>
Message-ID: <CAOCdBK+NGYiTt5Z85QyS9jWqmWyXa2BjvJ+g_JVZNCPO8B1UMg@mail.gmail.com>

Dear Zoltan,

I forgot about the link. Anyway, the last problem wasn't about merging a
base64 object and a string (the suggestion in Stefan mail worked). I was
able to create the html table and display it, but the images where broken.

But looking around, I found the solution on the wikipedia:
http://en.wikipedia.org/wiki/Data_URI_scheme#Python_3

thanks everyone again.

Francesco




2013/9/12 Zolt?n V?r?s <zvoros at gmail.com>

>  But I have told you that there is an issue in python 3.3. I sent you a
> link in one of my earlier e-mails. In that link you should also find the
> fix.
>
> Cheers,
> Zolt?n
>
>
> On 12/09/13 17:53, Francesco Montesano wrote:
>
> ehm.
> It does work with python 2. With python 3.3 I get in the notebook the
> symbols of brocken images and the string of the png printed in the console
>
>  Fra
>
>
> 2013/9/12 Francesco Montesano <franz.bergesund at gmail.com>
>
>> Dear Stefan,
>>
>>  Sorry for not replying before. It does work now.
>>
>>  Thanks,
>>
>>  Francesco
>>
>>
>>  2013/9/2 Francesco Montesano <franz.bergesund at gmail.com>
>>
>>>  Thanks,
>>>
>>>  I'll give a try tomorrow. not it's getting late
>>> Do you know if HTML understands b'...'?
>>>
>>>  Fra
>>>
>>>
>>>
>>>  2013/9/2 St?fan van der Walt <stefan at sun.ac.za>
>>>
>>>> On Mon, Sep 2, 2013 at 10:31 PM, Francesco Montesano
>>>> <franz.bergesund at gmail.com> wrote:
>>>> > thank you very much for the example. This should work wonderfully for
>>>> python
>>>> > 2.x., but, as Zoltan pointed out, it won't work on python 3.x, as you
>>>> can't
>>>> > concatenate string and bytes objects (returned by base64 methods).
>>>>
>>>>  That shouldn't be too much of a problem:
>>>>
>>>> In [12]: bytes('asd'.encode('utf-8')) + base64.b64encode(b'asd')
>>>> Out[12]: b'asdYXNk'
>>>>
>>>> In [13]: (bytes('asd'.encode('utf-8')) +
>>>> base64.b64encode(b'asd')).decode('utf-8')
>>>> Out[13]: 'asdYXNk'
>>>>
>>>> Have a look at "numpy.compat" for some utility functions as well.
>>>>
>>>> St?fan
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>
>
>
> _______________________________________________
> IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130913/01df5aff/attachment.html>

From mattpap at gmail.com  Fri Sep 13 13:34:12 2013
From: mattpap at gmail.com (Mateusz Paprocki)
Date: Fri, 13 Sep 2013 19:34:12 +0200
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <CAHAreOrrBtq+7qkn88JFfCbEFV97DH5bFA_2bt=j+RU_3Q2HaQ@mail.gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
	<CAHAreOrrBtq+7qkn88JFfCbEFV97DH5bFA_2bt=j+RU_3Q2HaQ@mail.gmail.com>
Message-ID: <CAGBZUCb7JJqrXgUM=5fBMbqHKmuaVac8YA8w3GyFcpGRt-u8RA@mail.gmail.com>

Hi,

On 10 September 2013 21:59, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi Mateusz,
>
>
> On Sun, Sep 8, 2013 at 12:54 PM, Mateusz Paprocki <mattpap at gmail.com> wrote:
>> Hi,
>>
>> I would like to announce IScala: a Scala-language backend for IPython,
>> see [1]. Scala is general purpose, object-functional, statically typed
>> programming language for JVM. IScala is an early work in progress, but
>> it's already fairly usable. All three types of frontends are
>> supported. Required dependencies are IPython 1.0+ and Java Runtime
>> Environment 1.6+. See README.md for installation and usage details.
>
> This is great! Just last week I was playing a bit with Scala, this
> will make my life much easier :)

Thanks. Let me know if you need any advice.

>> Motivation for this project came when I saw IJulia announcement. The
>> hard part - actually "interpreting" Scala - was done in Scala REPL by
>> Scala developers. Scala is a compiled language, so interpreting it
>> means that source code is compiled (with some fancy wrappers),
>> resulting class files loaded into JVM and code run via Java
>> reflection. Interpreted code behaves exactly the same as compiled code
>> and it runs at full speed. There are a few issues with this, however.
>> Scala is a pure object-oriented language, so at top level you can put
>> only packages, traits, classes and singleton objects. No expressions
>> and other types of statements are allowed. Interpreter removes this
>> restriction, by wrapping source code in singleton objects, so that you
>> can simply write e.g. `val x = 1` to define a value, which is
>> otherwise illegal.
>
> Very interesting: I've been thinking precisely about this problem in
> the context of creating an IPython kernel for Go, and wondering what
> it would take.  Go doesn't have the hardcore OO stance of the JVM, but
> some similar issues regarding persistence of state and compilation
> arise. The lessons you've learned here would likely be very useful.

I don't have any experience with Go, so unfortunately can't help here.

>> During development of IScala I also created a set of notebooks with
>> Scala code, see [2]. Those are based on a subset of Twitter's Scala
>> lessons [3]. I didn't encode notebooks directly in the JSON-based
>> format, but I invented an adhoc format based on Markdown with support
>> for cells. There is a simple converter that allows to generate
>> IPython-compatible notebooks.
>
> I would recommend using the 'real' IPython notebook format as your
> default, so that you can benefit automatically from the entire rest of
> our machinery (nbconvert, nbviewer, etc).

I actually benefit from those tools, because I use nbcovert to
generate html from generated ipynb files and ipynb files are stored
along with html on gh-pages branch, so you can use nbviewer on them.

> Our long term view is for the format to be completely language
> agnostic, so we're willing to make any changes and fixes to the
> underlying format to accomodate Scala cleanly.  We're already looking
> at a bunch of such fixes for Julia, and having another language at
> hand will help us correctly identify the right abstraction points and
> how to leave generality in the base format while allowing each
> language to represent its own information with all the detail and
> precision desired.

I think the IPython's notebook format is already quite language
independent. It's the interpretation of the format that has to be
improved (e.g. adding syntax highlight for other languages). I would
invent my adhoc format (xipynb) even if I was creating notebooks with
Python code, because editing JSON manually is not an option for me
(for quite obvious reasons). Neither is using Notebook app for major
editorial work (although, for running code is OK), at least until
vim-like key bindings are supported on cell and intra-cell level (the
other is equally important, because writing code/text vim style and
switching to emacs style for cell manipulation would be very
inefficient) (vim is too deeply wired in my mind). However, I may not
like JSON, but it's good you chose an encoding of the format that has
a (stable) specification and allows for easy automatic processing. I
think that there should be a format that can be easily processed,
converted etc. and understood by Notebook app, but which doesn't have
to be easily editable without specialized tools, and then, there could
be formats that aren't that easy to process, but allow for easy
editing (etc.). Then, scripts for converting to/from those formats
could be plugins to nbconvert. I don't expect Notebook app or html
generator to understand those extra formats directly.

Speaking about JSON, if you have a moment, you may want to look into
[1]. I implemented message specification there in an object-oriented
and type safe way (there is room for improvements). Originally I just
copied text from [2] and filled in "blanks", so the structure is
similar and comments are in place. It may be advantageous to abstract
the specification over JSON+Python's types. So, if there's a `list`,
then write what kind of list it should be, e.g. `list<int>` (here
Java/C++ style), the same for dictionaries/maps and tuples. If there
is "MIME type" involved, then define it, because I can assume it's a
string, but I could imagine it being a 2-tuple. Also use enumerations
instead of strings. In my opinion this will make the specification
much more formal and understandable and allow for removal of
substantial number of comments, which are otherwise unavoidable.

But those little issues when looking at big picture. In general my
experience have been pleasant and finally I have an environment for
interactive computing with Scala.

[1] https://github.com/mattpap/IScala/blob/master/src/main/scala/Msg.scala
[2] http://ipython.org/ipython-doc/dev/development/messaging.html

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Mateusz


From ellisonbg at gmail.com  Fri Sep 13 13:40:23 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 13 Sep 2013 10:40:23 -0700
Subject: [IPython-dev] IScala: a Scala-language backend for IPython
In-Reply-To: <CAGBZUCb7JJqrXgUM=5fBMbqHKmuaVac8YA8w3GyFcpGRt-u8RA@mail.gmail.com>
References: <CAGBZUCYtqC24s0fWCqigv9NxaEJ-KzrbWQ-k9V7_8hHjZjEoiQ@mail.gmail.com>
	<CAHAreOrrBtq+7qkn88JFfCbEFV97DH5bFA_2bt=j+RU_3Q2HaQ@mail.gmail.com>
	<CAGBZUCb7JJqrXgUM=5fBMbqHKmuaVac8YA8w3GyFcpGRt-u8RA@mail.gmail.com>
Message-ID: <CAH4pYpQLHehSUfUFrS3dNczF5U6woe2Yqqmsa768UfhSU7M1Sw@mail.gmail.com>

> I think the IPython's notebook format is already quite language
> independent. It's the interpretation of the format that has to be
> improved (e.g. adding syntax highlight for other languages).

I think you are confusing what the notebook format handles.  It knows
nothing about syntax highlighting.  Each code cell has a language
field that can be set.  The problem right now is with the frontend,
which doesn't set that field and switch syntax highlighting
appropriately.

> I would
> invent my adhoc format (xipynb) even if I was creating notebooks with
> Python code, because editing JSON manually is not an option for me
> (for quite obvious reasons). Neither is using Notebook app for major
> editorial work (although, for running code is OK), at least until
> vim-like key bindings are supported on cell and intra-cell level (the
> other is equally important, because writing code/text vim style and
> switching to emacs style for cell manipulation would be very
> inefficient) (vim is too deeply wired in my mind). However, I may not
> like JSON, but it's good you chose an encoding of the format that has
> a (stable) specification and allows for easy automatic processing. I
> think that there should be a format that can be easily processed,
> converted etc. and understood by Notebook app, but which doesn't have
> to be easily editable without specialized tools, and then, there could
> be formats that aren't that easy to process, but allow for easy
> editing (etc.). Then, scripts for converting to/from those formats
> could be plugins to nbconvert. I don't expect Notebook app or html
> generator to understand those extra formats directly.

I can understand why you would create another format if you want to
easily be able to edit the notebook documents by hand though.  JSON is
horrlble for that.

Cheers,

Brian

> Speaking about JSON, if you have a moment, you may want to look into
> [1]. I implemented message specification there in an object-oriented
> and type safe way (there is room for improvements). Originally I just
> copied text from [2] and filled in "blanks", so the structure is
> similar and comments are in place. It may be advantageous to abstract
> the specification over JSON+Python's types. So, if there's a `list`,
> then write what kind of list it should be, e.g. `list<int>` (here
> Java/C++ style), the same for dictionaries/maps and tuples. If there
> is "MIME type" involved, then define it, because I can assume it's a
> string, but I could imagine it being a 2-tuple. Also use enumerations
> instead of strings. In my opinion this will make the specification
> much more formal and understandable and allow for removal of
> substantial number of comments, which are otherwise unavoidable.
>
> But those little issues when looking at big picture. In general my
> experience have been pleasant and finally I have an environment for
> interactive computing with Scala.
>
> [1] https://github.com/mattpap/IScala/blob/master/src/main/scala/Msg.scala
> [2] http://ipython.org/ipython-doc/dev/development/messaging.html
>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> Mateusz
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Fri Sep 13 13:44:05 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 13 Sep 2013 10:44:05 -0700
Subject: [IPython-dev] ipython with sync
In-Reply-To: <CACLE5GCM4LAdnGXGXhnE=ssSfU6_4ZNeQ+y0scBYB0gbZrT=Tg@mail.gmail.com>
References: <CACLE5GDoXVOeEbEKZOJBzVbHAS=71RwhXbLfAkgfw0+U3rtC-w@mail.gmail.com>
	<CAH4pYpS0rEychGGcmu_zmdM-xtpJPqU+DmFtyc_8QP7938SjAw@mail.gmail.com>
	<CACLE5GCM4LAdnGXGXhnE=ssSfU6_4ZNeQ+y0scBYB0gbZrT=Tg@mail.gmail.com>
Message-ID: <CAH4pYpQSKj83qi7Td7Z8xbpT5nBLqcfWch-9gVZ_gjggy5DgEQ@mail.gmail.com>

Thanks for this link, that is helpful.

Cheers,

Brian

On Thu, Sep 12, 2013 at 4:04 PM, William Stein <wstein at gmail.com> wrote:
> On Thu, Sep 12, 2013 at 2:53 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> William,
>>
>> Thanks for notes.  This is really exciting and we are very interested
>> in getting live sync to work with IPython itself.  I am traveling this
>> week, but will try to have a look soon.  Is the code available
>> somewhere to look at?
>
> Unfortunately, this implementation is very entangled as part of
> something I'm building as part of a startup company with UW's Center
> for commercialization, and I can't just open source everything.  But
> the algorithm is here:
>
>   https://neil.fraser.name/writing/sync/
>
> and the author of that paper has an open source Python implementation,
> I think...
>
>
>>
>> Cheers,
>>
>> Brian
>>
>> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>>> Hi,
>>>
>>> Somewhat by accident I spent the last two weeks implementing hosted
>>> IPython notebooks with sync for https://cloud.sagemath.com.
>>> Initially I had just plan to simplify the port forwarding setup, since
>>> what Ondrej Certik was doing with multiple forward and reverse port
>>> forwards seemed complicated.  But then I became concerned about
>>> multiple users (or users with multiple browsers) overwriting each
>>> other's notebooks, because cloud.sagemath projects are frequently
>>> shared between multiple people, and everything else does realtime
>>> sync.    I had planned just to add some very minimal merge-on-save
>>> functionality to avoid major issues, but somehow got sucked into
>>> realtime sync (even with the other person's cursor showing).
>>>
>>> It would be enormously helpful to me if a couple of expert IPython
>>> users were to try out what I implemented and just ask a bunch of
>>> questions.
>>>
>>> 1. Go to https://cloud.sagemath.com and make an account; this is
>>> completely free, and is hosted on computers at University of
>>> Washington.
>>>
>>> 2. Create a new project.
>>>
>>> 3. Click +New, then click "IPython" (or paste in a link to an ipython
>>> notebook, or upload a file).
>>>
>>> 4. An IPython notebook server will start, the given .ipynb file should
>>> load in a same-domain iframe, and then some of the ipython notebook
>>> code is and iframe contents are monkey patched, in order to support
>>> sync and better integration with https://cloud.sagemath.com.
>>>
>>> 5. Open the ipynb file in multiple browsers, and see that changes in
>>> one appear in the other, including moving cells around, creating new
>>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>>> (evaluating a cell that creates no output doesn't set this flag, at
>>> least in 1.0.0, which is a bug in IPython, I guess).
>>>
>>> Since this is all very new and the first (I guess) realtime sync
>>> implementation on top of IPython, there are probably a lot of issues.
>>>  Note that if you click the "i" info button to the right, you'll get a
>>> link to the standard IPython
>>>
>>> The other thing of interest is a little Python script called
>>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>>> an IPython notebook server as a daemon, get the port it is running on,
>>> etc.  It's pretty simple but satisfies my use case, and has
>>> JSON-output, to make it web friendly.    As I've written it, my script
>>> passes several base_url options through by default, which are needed
>>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>>> BSD license) in case there is any interest.
>>>
>>> Regarding the monkey patching in 4 above, the right thing to do would
>>> be to explain exactly what hooks/changes in the IPython html client I
>>> need in order to do sync, etc., make sure these makes sense to the
>>> IPython devs, and send a pull request (or have a coding sprint in
>>> Seattle or Berkeley?).  As an example, in order to do sync
>>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>>> critical to do this in place when possible, since the overhead of
>>> creating a new cell is huge (due probably to the overhead of creating
>>> CodeMirror editors); however, the fromJSON method in IPython assumes
>>> that the cell is brand new -- it would be nice to add an option to
>>> make a cell fromJSON without assuming it is empty.
>>>
>>> The ultimate outcome of this could be a clean well-defined way of
>>> doing sync for IPython notebooks using any third-party sync
>>> implementation.  IPython might provide their own sync service and
>>> there are starting to be others available these days -- e.g., Google
>>> has one: https://developers.google.com/drive/realtime/, and maybe
>>> Guido van Rosum helped write one for Dropbox recently?
>>>
>>> Subdirectories:   I noticed, incidentally, that the wakari version of
>>> the IPython notebook server allows one to load ipynb files that are in
>>> any subdirectory, whereas the standard IPython notebook server
>>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>>> server for each directory that a user accesses files in right now.
>>> This seems cludgy, so I'm interested in the situation regarding adding
>>> support for subdirectories.
>>>
>>> -- William
>>>
>>>
>>>
>>> --
>>> William Stein
>>> Professor of Mathematics
>>> University of Washington
>>> http://wstein.org
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From benjaminrk at gmail.com  Fri Sep 13 14:14:17 2013
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 13 Sep 2013 11:14:17 -0700
Subject: [IPython-dev] Ipython parallel and PBS
In-Reply-To: <CAJtW7t=8SzOof6_oOqPS5==U7VZSH7ErC52e9ZPUrjAZ4LaqgQ@mail.gmail.com>
References: <CAJtW7t=8SzOof6_oOqPS5==U7VZSH7ErC52e9ZPUrjAZ4LaqgQ@mail.gmail.com>
Message-ID: <CAHNn8BWhb_jLV9U7YC6t9qJxdZu4LfZc_ox2LHHHhyX4Tzqesg@mail.gmail.com>

Can you inspect the pbs_engines template, and see if anything looks wrong?
Can you submit it manually, with qsub ./pbs_engines?


On Fri, Sep 13, 2013 at 3:38 AM, James <jamesresearching at gmail.com> wrote:

> Dear all,
>
> I'm having a lot of trouble setting up IPython parallel on a PBS cluster,
> and I would really appreciate any help.
>
> The architecture is a standard PBS cluster - a head node with slave nodes.
> I connect to the head node from my laptop over ssh.
>
> The client (laptop) -> Head node connection seems simple enough. The
> problem is the engines.
>
> Ignoring the laptop for a moment, I'll just focus on running ipython on
> the head node, with the engines on a slave node. I assume this is a correct
> method of working?
>
> I did the following on the head node, following instructions at
> http://ipython.org/ipython-doc/stable/parallel/parallel_process.html#using-ipcluster-in-pbs-mode:
>
> $ ipython profile create --parallel --profile=pbs
>
> Files are as follows:
>
> $cat ipcluster_config.py
> c = get_config()
> c.IPClusterStart.controller_launcher_class = 'PBSControllerLauncher'
> c.IPClusterEngines.engine_launcher_class = 'PBSEngineSetLauncher'
> c.PBSLauncher.queue = 'long'
> c.IPClusterEngines.n = 2 # Run 2 cores on 1 node or 2 nodes with all
> cores? Not sure.
>
> $ cat ipengine_config.py
> c = get_config()
>
> Then execute on the head node:
> $ ipcluster start --profile=pbs -n 2
> 2013-09-10 15:02:46,771.771 [IPClusterStart] Using existing profile dir:
> u'/home/username/.ipython/profile_pbs'
> 2013-09-10 15:02:46.777 [IPClusterStart] Starting ipcluster with
> [daemon=False]
> 2013-09-10 15:02:46.778 [IPClusterStart] Creating pid file:
> /home/username/.ipython/profile_pbs/pid/ipcluster.pid
> 2013-09-10 15:02:46.778 [IPClusterStart] Starting Controller with
> PBSControllerLauncher
> 2013-09-10 15:02:46.792 [IPClusterStart] Job submitted with job id: '2830'
> 2013-09-10 15:02:47.793 [IPClusterStart] Starting 2 Engines with
> PBSEngineSetLauncher
> 2013-09-10 15:02:47.808 [IPClusterStart] Job submitted with job id: '2831'
>
> Then the queue shows
> $ qstat
> Job id                    Name             User            Time Use S Queue
> ------------------------- ---------------- --------------- -------- - -----
> 2830[].master              ipcontroller     username              0 Q
> long
> 2831[].master              ipengine         username              0 Q
> long
>
> And they just hang there, queued forever. I assume the engines at least
> should be running? Full information through "qstat -f" doesn't give the
> reason for the queuing. Normally it would do. There are more than 4 nodes
> available.
>
> $qstat -f
> Job Id: 2831[].master.domain
>     Job_Name = ipengine
>     Job_Owner = username at master.domain
>     job_state = Q
>     queue = long
>     server = [head node's domain address]
>     Checkpoint = u
>     ctime = Tue Sep 10 15:02:47 2013
>     Error_Path = master.domain:/home/username/
> ipengine.e2831
>     Hold_Types = n
>     Join_Path = n
>     Keep_Files = n
>     Mail_Points = a
>     mtime = Tue Sep 10 15:02:47 2013
>     Output_Path = master.domain:/home/username/ipengine.o2831
>     Priority = 0
>     qtime = Tue Sep 10 15:02:47 2013
>     Rerunable = True
>     [...]
>     etime = Tue Sep 10 15:02:47 2013
>     submit_args = ./pbs_engines
>     job_array_request = 1-2
>     fault_tolerant = False
>     submit_host = master.domain
>     init_work_dir = /home/username
>
> It also seems strange that the ipcontroller is launched through PBS. I
> thought this should be on the head node, so I changed
> 'PBSControllerLauncher' to 'LocalControllerLauncher'. Then it doesn't
> queue, but I don't know if what I'm doing is correct.
>
> Any help would be really greatly appreciated.
>
> Thank you.
>
> James
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130913/cd410fa6/attachment.html>

From mikhail.ksenzov at gmail.com  Fri Sep 13 18:59:50 2013
From: mikhail.ksenzov at gmail.com (mksenzov)
Date: Fri, 13 Sep 2013 15:59:50 -0700 (PDT)
Subject: [IPython-dev] %install_ext question
In-Reply-To: <CAPSkGgek=xeNYku3ymJt4fD7688qM3CY_DnhRXb-bzx8v57QMQ@mail.gmail.com>
References: <CAPSkGgeNmcyvADKFo1Sjh3pgan8qyya7ePUSt7KE=Hseke_Xew@mail.gmail.com>
	<CAOvn4qgWNSKykyAsazjvFyOkKNg+VbvKu2Tm6z4o0gJSpBjynA@mail.gmail.com>
	<CAPSkGgei=AhYTZ88Si1yqy5j2iv257=_sXdaxniX6L265Ny+dQ@mail.gmail.com>
	<CAOvn4qiZc+zMAtn2RjWjG=0B9v3o0djN4s9h=eTCdtfFvua1pw@mail.gmail.com>
	<CAPSkGgek=xeNYku3ymJt4fD7688qM3CY_DnhRXb-bzx8v57QMQ@mail.gmail.com>
Message-ID: <1379113190038-5032167.post@n6.nabble.com>

I ran into exactly the same problem as you, found this thread and ended up
creating an extension that installs the zip file and then does all the
necessary manipulations with sys.path:
https://github.com/mksenzov/ipython_zip_extensions 

It has a magic function that can optionally replace the standard IPython's
%install_ext if needed.





--
View this message in context: http://python.6.x6.nabble.com/install-ext-question-tp5012904p5032167.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From eric at depagne.org  Sat Sep 14 07:02:14 2013
From: eric at depagne.org (=?ISO-8859-1?Q?=C9ric?= Depagne)
Date: Sat, 14 Sep 2013 13:02:14 +0200
Subject: [IPython-dev] Ipython output uses the previous version of the file
	to show wrong code line.
Message-ID: <5364828.hxnvyCpGHo@localhost.localdomain>

Hi all.

This is probably something I'm doing wrong, but I cannot figure out what.
When I run a script in ipython (started with ipython qtconsole --pylab=qt, ipython 
version 1.1.0, python 2.7.5), ipython warns me of errors, and tells me what line to look 
at. My problem is that when ipython outputs which line to look at, it will parse the 
previous version of the file, instead of the current one, albeit indicating me the correct 
line number.

Here is a short example of what happens (I have added the line number to the output 
of the !more command to make things easier to follow. The Numbers from 1 to 5 are 
not part of the python file).

In [*1*]: cd ~/Programmes/Python/Crosstalk/
/home/usertest/Programmes/Python/Crosstalk

In [*2*]: run test.py
  File "/home/usertest/Programmes/Python/Crosstalk/test.py", line 3
    b = a2/
           ^
SyntaxError: invalid syntax


In [*3*]: !more test.py
1 #!/usr/bin/python
2 a = 2
3 b = a2/
4 print b


In [*4*]: !more test.py
1 #!/usr/bin/python
2 a = 2
3 
4 b = a2/
5 print b


In [*5*]: run test.py
  File "/home/usertest/Programmes/Python/Crosstalk/test.py", line 4
    print b
           ^
SyntaxError: invalid syntax

So the first pass detects that the line 3 is wrong. Then I modify the file to add a blank 
line before. Thus now, it tells me line 4 is wrong. But then it show the content of line 
for in the previous version of my file, instead of the current one (thus the "print b" 
instead of the "b = a2/" ).

I've already removed my .config/.ipython directory (in fact, I've done this test with a 
newly created testuser, and it still shows the same behaviour.

Of course, when the faulty line is correct, everything works, but it does not change the 
problem : if I "uncorrect" the line to make it wrong, the same behaviour reoccurs.

I must be doing something wrong, but can't figure out what. Any idea welcome!

Thx.
?ric.


-- 
Un clavier azerty en vaut deux
----------------------------------------------------------
?ric Depagne                            eric at depagne.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130914/dc6a8bf1/attachment.html>

From benjaminrk at gmail.com  Sat Sep 14 15:31:20 2013
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 14 Sep 2013 12:31:20 -0700
Subject: [IPython-dev] Ipython output uses the previous version of the
 file to show wrong code line.
In-Reply-To: <5364828.hxnvyCpGHo@localhost.localdomain>
References: <5364828.hxnvyCpGHo@localhost.localdomain>
Message-ID: <CAHNn8BX7osLvQsBWq2tsLkGwdzwic301_PG4Qs_rovfQ0fpAvA@mail.gmail.com>

Something is going wrong with caching the contents of the file for use when
displaying the traceback. The cache isn't being updated, even though the
file has changed.

Perhaps someone with more intimate knowledge of the inner workings of %run
and/or the traceback renderer can shed more light on it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130914/3bddceb5/attachment.html>

From takowl at gmail.com  Sat Sep 14 17:52:36 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 14 Sep 2013 14:52:36 -0700
Subject: [IPython-dev] Ipython output uses the previous version of the
 file to show wrong code line.
In-Reply-To: <CAHNn8BX7osLvQsBWq2tsLkGwdzwic301_PG4Qs_rovfQ0fpAvA@mail.gmail.com>
References: <5364828.hxnvyCpGHo@localhost.localdomain>
	<CAHNn8BX7osLvQsBWq2tsLkGwdzwic301_PG4Qs_rovfQ0fpAvA@mail.gmail.com>
Message-ID: <CAOvn4qi9e1RCd3zsMoJfo6Bx9LWyS0J-omEtB4q4OpASAn+KDg@mail.gmail.com>

On 14 September 2013 12:31, MinRK <benjaminrk at gmail.com> wrote:

> Something is going wrong with caching the contents of the file for use
> when displaying the traceback. The cache isn't being updated, even though
> the file has changed.
>
> Perhaps someone with more intimate knowledge of the inner workings of %run
> and/or the traceback renderer can shed more light on it.
>

The relevant modules are linecache (in the standard library) and our
wrapper, ulinecache. I'll dig into this more on Monday.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130914/1d46e6cf/attachment.html>

From paulgb at gmail.com  Sun Sep 15 12:29:45 2013
From: paulgb at gmail.com (Paul Butler)
Date: Sun, 15 Sep 2013 12:29:45 -0400
Subject: [IPython-dev] Kernel race condition on Mac OS
Message-ID: <CAKBnqpAXkdwXVGuEOAvQysah2AE3qhfT_whTSzQReUTU-OSwPQ@mail.gmail.com>

Hi,

I'm developing with the kernel APIs and I've encountered a race condition.
It seems to be a Mac-specific thing; I haven't been able to reproduce it on
Ubuntu or CentOS.

The following code demonstrates the problem. It's also available as a
gist<https://gist.github.com/paulgb/6572133>.
The call to iopub.get_msg will (usually) fail unless sleep(1) is
uncommented.

from IPython.kernel import KernelManager
from time import sleep


def racecondition():
    km = KernelManager()
    km.start_kernel()


    kc = km.client()
    kc.start_channels()


    iopub = kc.iopub_channel
    shell = kc.shell_channel

    #sleep(1)


    shell.execute('print "hello world"')
    print shell.get_msg(timeout=1)
    print iopub.get_msg(timeout=1)

if __name__ == '__main__':
    racecondition()

By experimenting with where the sleep(1) call can go, it seems that
the kernel subprocess takes a few milliseconds to be ready to accept
input.


Is there any call to check if the kernel is ready?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130915/cfcaee8c/attachment.html>

From arenold at ischool.berkeley.edu  Mon Sep 16 03:40:17 2013
From: arenold at ischool.berkeley.edu (AJ Renold)
Date: Mon, 16 Sep 2013 00:40:17 -0700
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
Message-ID: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>

Hi,

I'm a graduate student at UC Berkeley School of Information and have worked
with IPython in various classes and promoted it in others. As an
enthusiastic user and a student of Peer
Production<http://www.ischool.berkeley.edu/courses/i290m-ocpp>,
I'd like to offer my time to contribute to IPython and have a question
about setting up a development environment.

Specifically I'd like to work on the open issue here (
https://github.com/ipython/ipython/issues/3653) to extend the Completer
class to handle auto-completing in the case of "from module import <tab>".

Getting started on this issue, I am having trouble setting up my
development environment inside of virualenv to work on editing
completer.js. I have tried the Development
Instructions<https://github.com/ipython/ipython#development-installation>
on
Github and on IPython.org<http://ipython.org/ipython-doc/stable/install/install.html#installing-the-development-version>,
but both methods don't seem to work when I modify the file and then create
a new ipython notebook server. Is there a build process or something
similar that I am missing? If there is another resource on setting up the
correct development environment that I am missing please link me. I would
also be happy to add to the development wiki if these instructions are
buried somewhere obscure.

After getting this set, I plan to follow up that github issue with a Pull
Request.

Thanks!
AJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/9991da43/attachment.html>

From bussonniermatthias at gmail.com  Mon Sep 16 04:47:00 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 16 Sep 2013 10:47:00 +0200
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
Message-ID: <46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>

Hi there, 


Le 16 sept. 2013 ? 09:40, AJ Renold a ?crit :

> Hi,
> 
> I'm a graduate student at UC Berkeley School of Information and have worked with IPython in various classes and promoted it in others. As an enthusiastic user and a student of Peer Production, I'd like to offer my time to contribute to IPython and have a question about setting up a development environment.

Nice to meet you and see that you are motivated.

> 
> Specifically I'd like to work on the open issue here (https://github.com/ipython/ipython/issues/3653) to extend the Completer class to handle auto-completing in the case of "from module import <tab>".
> 
> Getting started on this issue, I am having trouble setting up my development environment inside of virualenv to work on editing completer.js. I have tried the Development Instructions on Github and on IPython.org, but both methods don't seem to work when I modify the file and then create a new ipython notebook server.

What do you mean by "do not seem to work" ? 
If javascript seem not to change, try to force-refresh. 
Usually if you modify a JS file you do not need to restart the server but just refresh the page.
Using a window with private browsing/incognito mode/... also help in making sure this is not due to browser caching.

import IPython
print IPython.sys_info()

will also help you to diagnoses in which folder IPython is looking for resources.


> Is there a build process or something similar that I am missing? If there is another resource on setting up the correct development environment that I am missing please link me. I would also be happy to add to the development wiki if these instructions are buried somewhere obscure.

You can also try 
$ python setup.py develop


> After getting this set, I plan to follow up that github issue with a Pull Request.

I would also suggest you get a look at IPython Ehencement Proposal 11 About completion.

https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor

If at the same time you could try to make the API for the completer configurable, that would be great. 
For example, current code in completer.js does not assume `!` is a char that can be part of a token, whereas it can be in julia.

Don't hesitate to ask question directly on the relevant issue. 
If you really have problems, most of the core dev are near berkley, so meeting in person will be the fasttest if you are really stuck. 
-- 
Matthias

> 
> Thanks!
> AJ



> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/035dca09/attachment.html>

From franz.bergesund at gmail.com  Mon Sep 16 12:03:12 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 16 Sep 2013 18:03:12 +0200
Subject: [IPython-dev] [parallel] issue when executing 'import pandas as pd'
	on engine
Message-ID: <CAOCdBK+HGWQQ977ZjSG91BsJVf3CVrMEaOs1Fr+dZm6NU4o79w@mail.gmail.com>

Dear all,

I'm having an issue with the parallel (v1.1.0, under python 2.7).

Some time ago I did build a number of python codes to manipulate
catalogues.
I can have either thousands of small file or few possibly huge file.
So I've written my codes such that I can chose from command whether to use
any of the two.

Typically my codes have the following structure:

import numpy as np
> import pandas as pd
> def parse(...):  #argparse
>     ....
> def to_do(fname,...):  #function(s) that do what I need
>     ....
> if __name__=='__main__':
>     args = parse(...)
>     if args.paralell == False:
>          for fn in file_name_list:
>               to_do(fn, ...)

     else:  #execute in parallel

         parallel_env = Lbv()       # custom class that init a load
> ballance view

         imports = ['import numpy as np', 'import pandas as pd']

         parallel_env.exec_on_engine(imports) # execute the above strings
> on all engines (direct view)

         #execute in parallel

         runs = [parallel_env.apply(to_do, os.path.abspath(fn), ...) for fn
> in file_name_list]



I build the whole parallel dispatching against ipython 0.13 and everything
worked fine.
But today I've tried to run one of my scripts enabling parallel and got the
following error


>   File "code.py", line XXX, in <module>
>     parallel_env.exec_on_engine(imports)
>   File "XXX/ipython_parallel.py", line 86, in exec_on_engine
>     e.raise_exception()
>   File "XXX.local/lib/python2.7/site-packages/IPython/parallel/error.py",
> line 199, in raise_exception
>     raise RemoteError(en, ev, etb, ei)
> RemoteError: NameError(name 'plt' is not defined)


The only thing that uses matplotlib is pandas, and modifying

 imports = ['import numpy as np', 'import matplotlib.pyplot as plt',
> 'import pandas as pd']


seems to solve the problem (although at least in one case the first call of
my code crashed with the error and the second went through).

If I run my code without requiring the ipy parallel I don't have any
problem with 'plt'

I guess that this is a bug. But I still haven't understood how to debug
what happens on the engines, so I can't give more details.
Any clues?

If needed I can load my 'ipython_parallel.py' module in gist/github

Cheers,

Fra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/669599eb/attachment.html>

From takowl at gmail.com  Mon Sep 16 13:23:20 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 16 Sep 2013 10:23:20 -0700
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
	<46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
Message-ID: <CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>

On 16 September 2013 01:47, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> I would also suggest you get a look at IPython Ehencement Proposal 11
> About completion.
>
>
> https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor<https://github.com/ipython/ipython/wiki/IPEP-11:-Tab-Completion-System-Refactor>
>
> If at the same time you could try to make the API for the completer
> configurable, that would be great.
>

I think that IPEP is mostly about the Python side, whereas issue #3653 is
probably something in the notebook javascript, because the same completion
does work in the terminal.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/9cfdc774/attachment.html>

From gmbecker at ucdavis.edu  Mon Sep 16 13:49:28 2013
From: gmbecker at ucdavis.edu (Gabriel Becker)
Date: Mon, 16 Sep 2013 10:49:28 -0700
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
	<46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
	<CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>
Message-ID: <CADwqtCNd+peq34AgegCfx3O_f8nb5i-YrVPjZ4DO45r_K9sNTg@mail.gmail.com>

AJ,

I've never had any problem modifying javascript files and having the
changes show up when running the following from the root directory of a
clone of the ipython repository (on a system with all the dependencies
available):

python -m IPython notebook

I'm not a python guru so I may be doing something silly here but it has
always worked for me (and I have spent quite a bit of time modifying the
notebook javascript machinery).

~G


On Mon, Sep 16, 2013 at 10:23 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 16 September 2013 01:47, Matthias BUSSONNIER <
> bussonniermatthias at gmail.com> wrote:
>
>> I would also suggest you get a look at IPython Ehencement Proposal 11
>> About completion.
>>
>>
>> https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor<https://github.com/ipython/ipython/wiki/IPEP-11:-Tab-Completion-System-Refactor>
>>
>> If at the same time you could try to make the API for the completer
>> configurable, that would be great.
>>
>
> I think that IPEP is mostly about the Python side, whereas issue #3653 is
> probably something in the notebook javascript, because the same completion
> does work in the terminal.
>
> Thomas
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Gabriel Becker
Graduate Student
Statistics Department
University of California, Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/a411cb72/attachment.html>

From benjaminrk at gmail.com  Mon Sep 16 13:53:34 2013
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 16 Sep 2013 10:53:34 -0700
Subject: [IPython-dev] [parallel] issue when executing 'import pandas as
 pd' on engine
In-Reply-To: <CAOCdBK+HGWQQ977ZjSG91BsJVf3CVrMEaOs1Fr+dZm6NU4o79w@mail.gmail.com>
References: <CAOCdBK+HGWQQ977ZjSG91BsJVf3CVrMEaOs1Fr+dZm6NU4o79w@mail.gmail.com>
Message-ID: <CAHNn8BVNe21ojvEuGO39zhpxytppFewjF_6tv8tvqmpWB9T87A@mail.gmail.com>

Cn you please share your whole code?


On Mon, Sep 16, 2013 at 9:03 AM, Francesco Montesano <
franz.bergesund at gmail.com> wrote:

> Dear all,
>
> I'm having an issue with the parallel (v1.1.0, under python 2.7).
>
> Some time ago I did build a number of python codes to manipulate
> catalogues.
> I can have either thousands of small file or few possibly huge file.
> So I've written my codes such that I can chose from command whether to use
> any of the two.
>
> Typically my codes have the following structure:
>
> import numpy as np
>> import pandas as pd
>> def parse(...):  #argparse
>>     ....
>> def to_do(fname,...):  #function(s) that do what I need
>>     ....
>> if __name__=='__main__':
>>     args = parse(...)
>>     if args.paralell == False:
>>          for fn in file_name_list:
>>               to_do(fn, ...)
>
>      else:  #execute in parallel
>
>          parallel_env = Lbv()       # custom class that init a load
>> ballance view
>
>          imports = ['import numpy as np', 'import pandas as pd']
>
>          parallel_env.exec_on_engine(imports) # execute the above strings
>> on all engines (direct view)
>
>          #execute in parallel
>
>          runs = [parallel_env.apply(to_do, os.path.abspath(fn), ...) for
>> fn in file_name_list]
>
>
>
> I build the whole parallel dispatching against ipython 0.13 and everything
> worked fine.
> But today I've tried to run one of my scripts enabling parallel and got
> the following error
>
>
>>   File "code.py", line XXX, in <module>
>>     parallel_env.exec_on_engine(imports)
>>   File "XXX/ipython_parallel.py", line 86, in exec_on_engine
>>     e.raise_exception()
>>   File "XXX.local/lib/python2.7/site-packages/IPython/parallel/error.py",
>> line 199, in raise_exception
>>     raise RemoteError(en, ev, etb, ei)
>> RemoteError: NameError(name 'plt' is not defined)
>
>
> The only thing that uses matplotlib is pandas, and modifying
>
>  imports = ['import numpy as np', 'import matplotlib.pyplot as plt',
>> 'import pandas as pd']
>
>
> seems to solve the problem (although at least in one case the first call
> of my code crashed with the error and the second went through).
>
> If I run my code without requiring the ipy parallel I don't have any
> problem with 'plt'
>
> I guess that this is a bug. But I still haven't understood how to debug
> what happens on the engines, so I can't give more details.
> Any clues?
>
> If needed I can load my 'ipython_parallel.py' module in gist/github
>
> Cheers,
>
> Fra
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/2fef25d7/attachment.html>

From arenold at ischool.berkeley.edu  Mon Sep 16 14:28:10 2013
From: arenold at ischool.berkeley.edu (AJ Renold)
Date: Mon, 16 Sep 2013 11:28:10 -0700
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <CADwqtCNd+peq34AgegCfx3O_f8nb5i-YrVPjZ4DO45r_K9sNTg@mail.gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
	<46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
	<CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>
	<CADwqtCNd+peq34AgegCfx3O_f8nb5i-YrVPjZ4DO45r_K9sNTg@mail.gmail.com>
Message-ID: <CABY8EU4d0F3vqEnnS07OTHhAKVUswaO2=b+uEyVDXm4bh-gmUg@mail.gmail.com>

Thanks everyone. It seems Gabriel's method of running IPython works. I
forgot to mention this is on a system with IPython already installed
system-wide. Here is the difference running from the root directory of the
repository with ipython installed in a virtualenv:

Output of "print IPython.sys_info()"

after running IPython with $ python -m IPython notebook

{'codename': 'An Afternoon Hack',
 'commit_hash': '2dd6b4d',
 'commit_source': 'repository',
 'default_encoding': 'UTF-8',
 'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
 'ipython_version': '2.0.0-dev',
 'os_name': 'posix',
 'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
 'sys_executable': '/Users/ajrenold/Dropbox/Code/ipython/venv/bin/python',
 'sys_platform': 'darwin',
 'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1
Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}


and after running with $ ipython notebook

{'codename': 'An Afternoon Hack',
 'commit_hash': '2dd6b4d',
 'commit_source': 'repository',
 'default_encoding': 'UTF-8',
 'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
 'ipython_version': '2.0.0-dev',
 'os_name': 'posix',
 'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
 'sys_executable': '/usr/bin/python',
 'sys_platform': 'darwin',
 'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1
Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}


Looks like I'm already out of date compared to today's most recent commit!

Also, thank you for all of the comments on the completion system. I was
planning to contribute to completer.js and was not aware of IPEP 11: Tab
Completion System
Refactor<https://github.com/ipython/ipython/wiki/IPEP-11:-Tab-Completion-System-Refactor>.
I'll take a look at IPEP 11 in more detail to see if it is something I can
contribute to.

AJ

On Mon, Sep 16, 2013 at 10:49 AM, Gabriel Becker <gmbecker at ucdavis.edu>wrote:

> AJ,
>
> I've never had any problem modifying javascript files and having the
> changes show up when running the following from the root directory of a
> clone of the ipython repository (on a system with all the dependencies
> available):
>
> python -m IPython notebook
>
> I'm not a python guru so I may be doing something silly here but it has
> always worked for me (and I have spent quite a bit of time modifying the
> notebook javascript machinery).
>
> ~G
>
>
> On Mon, Sep 16, 2013 at 10:23 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>
>> On 16 September 2013 01:47, Matthias BUSSONNIER <
>> bussonniermatthias at gmail.com> wrote:
>>
>>> I would also suggest you get a look at IPython Ehencement Proposal 11
>>> About completion.
>>>
>>>
>>> https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor<https://github.com/ipython/ipython/wiki/IPEP-11:-Tab-Completion-System-Refactor>
>>>
>>> If at the same time you could try to make the API for the completer
>>> configurable, that would be great.
>>>
>>
>> I think that IPEP is mostly about the Python side, whereas issue #3653 is
>> probably something in the notebook javascript, because the same completion
>> does work in the terminal.
>>
>> Thomas
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
>
> --
> Gabriel Becker
> Graduate Student
> Statistics Department
> University of California, Davis
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/1abf9aa3/attachment.html>

From bussonniermatthias at gmail.com  Mon Sep 16 14:37:51 2013
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 16 Sep 2013 20:37:51 +0200
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <CABY8EU4d0F3vqEnnS07OTHhAKVUswaO2=b+uEyVDXm4bh-gmUg@mail.gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
	<46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
	<CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>
	<CADwqtCNd+peq34AgegCfx3O_f8nb5i-YrVPjZ4DO45r_K9sNTg@mail.gmail.com>
	<CABY8EU4d0F3vqEnnS07OTHhAKVUswaO2=b+uEyVDXm4bh-gmUg@mail.gmail.com>
Message-ID: <A6F96291-800B-42B2-B2A7-C444F14E34BB@gmail.com>

The venv IPython seem to use the Dropbox IPython folder, is this what you expect ?

Note that my comment about IPep 11 (even if it only touch python side) might be relevant to the refactor of completer.js to try to prepare it for potential refactor. (Yes I know completer.js should be re-written from scratch using CM hint helpers for example).

Envoy? de mon iPhone

Le 16 sept. 2013 ? 20:28, AJ Renold <arenold at ischool.berkeley.edu> a ?crit :

> Thanks everyone. It seems Gabriel's method of running IPython works. I forgot to mention this is on a system with IPython already installed system-wide. Here is the difference running from the root directory of the repository with ipython installed in a virtualenv:
> 
> Output of "print IPython.sys_info()"
> 
> after running IPython with $ python -m IPython notebook
> 
> {'codename': 'An Afternoon Hack',
>  'commit_hash': '2dd6b4d',
>  'commit_source': 'repository',
>  'default_encoding': 'UTF-8',
>  'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
>  'ipython_version': '2.0.0-dev',
>  'os_name': 'posix',
>  'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
>  'sys_executable': '/Users/ajrenold/Dropbox/Code/ipython/venv/bin/python',
>  'sys_platform': 'darwin',
>  'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}
> 
> and after running with $ ipython notebook
> 
> {'codename': 'An Afternoon Hack',
>  'commit_hash': '2dd6b4d',
>  'commit_source': 'repository',
>  'default_encoding': 'UTF-8',
>  'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
>  'ipython_version': '2.0.0-dev',
>  'os_name': 'posix',
>  'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
>  'sys_executable': '/usr/bin/python',
>  'sys_platform': 'darwin',
>  'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}
> 
> Looks like I'm already out of date compared to today's most recent commit!
> 
> Also, thank you for all of the comments on the completion system. I was planning to contribute to completer.js and was not aware of IPEP 11: Tab Completion System Refactor. I'll take a look at IPEP 11 in more detail to see if it is something I can contribute to.
> 
> AJ
> 
> On Mon, Sep 16, 2013 at 10:49 AM, Gabriel Becker <gmbecker at ucdavis.edu> wrote:
>> AJ,
>> 
>> I've never had any problem modifying javascript files and having the changes show up when running the following from the root directory of a clone of the ipython repository (on a system with all the dependencies available):
>> 
>> python -m IPython notebook
>> 
>> I'm not a python guru so I may be doing something silly here but it has always worked for me (and I have spent quite a bit of time modifying the notebook javascript machinery).
>> 
>> ~G
>> 
>> 
>> On Mon, Sep 16, 2013 at 10:23 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>>> On 16 September 2013 01:47, Matthias BUSSONNIER <bussonniermatthias at gmail.com> wrote:
>>>> I would also suggest you get a look at IPython Ehencement Proposal 11 About completion.
>>>> 
>>>> https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor
>>>> 
>>>> If at the same time you could try to make the API for the completer configurable, that would be great. 
>>> 
>>> I think that IPEP is mostly about the Python side, whereas issue #3653 is probably something in the notebook javascript, because the same completion does work in the terminal.
>>> 
>>> Thomas
>>> 
>>> 
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> 
>> 
>> -- 
>> Gabriel Becker
>> Graduate Student
>> Statistics Department
>> University of California, Davis
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/747dc291/attachment.html>

From franz.bergesund at gmail.com  Mon Sep 16 15:28:49 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Mon, 16 Sep 2013 21:28:49 +0200
Subject: [IPython-dev] [parallel] issue when executing 'import pandas as
 pd' on engine
In-Reply-To: <CAHNn8BVNe21ojvEuGO39zhpxytppFewjF_6tv8tvqmpWB9T87A@mail.gmail.com>
References: <CAOCdBK+HGWQQ977ZjSG91BsJVf3CVrMEaOs1Fr+dZm6NU4o79w@mail.gmail.com>
	<CAHNn8BVNe21ojvEuGO39zhpxytppFewjF_6tv8tvqmpWB9T87A@mail.gmail.com>
Message-ID: <CAOCdBK+FqW9eBQ7yVBqfWBBkYLrFhvu65TJTH+=E4=8WFbGX1Q@mail.gmail.com>

These codes of mine can be few hundred lines and depend upon other 2 or 3
modules that I have put together.

If the problem is where I think it is, I might be able to write a
standalone program (maybe in a notebook). Stay tuned :D

Fra





2013/9/16 MinRK <benjaminrk at gmail.com>

> Cn you please share your whole code?
>
>
> On Mon, Sep 16, 2013 at 9:03 AM, Francesco Montesano <
> franz.bergesund at gmail.com> wrote:
>
>> Dear all,
>>
>> I'm having an issue with the parallel (v1.1.0, under python 2.7).
>>
>> Some time ago I did build a number of python codes to manipulate
>> catalogues.
>> I can have either thousands of small file or few possibly huge file.
>> So I've written my codes such that I can chose from command whether to
>> use any of the two.
>>
>> Typically my codes have the following structure:
>>
>> import numpy as np
>>> import pandas as pd
>>> def parse(...):  #argparse
>>>     ....
>>> def to_do(fname,...):  #function(s) that do what I need
>>>     ....
>>> if __name__=='__main__':
>>>     args = parse(...)
>>>     if args.paralell == False:
>>>          for fn in file_name_list:
>>>               to_do(fn, ...)
>>
>>      else:  #execute in parallel
>>
>>          parallel_env = Lbv()       # custom class that init a load
>>> ballance view
>>
>>          imports = ['import numpy as np', 'import pandas as pd']
>>
>>          parallel_env.exec_on_engine(imports) # execute the above strings
>>> on all engines (direct view)
>>
>>          #execute in parallel
>>
>>          runs = [parallel_env.apply(to_do, os.path.abspath(fn), ...) for
>>> fn in file_name_list]
>>
>>
>>
>> I build the whole parallel dispatching against ipython 0.13 and
>> everything worked fine.
>> But today I've tried to run one of my scripts enabling parallel and got
>> the following error
>>
>>
>>>   File "code.py", line XXX, in <module>
>>>     parallel_env.exec_on_engine(imports)
>>>   File "XXX/ipython_parallel.py", line 86, in exec_on_engine
>>>     e.raise_exception()
>>>   File
>>> "XXX.local/lib/python2.7/site-packages/IPython/parallel/error.py", line
>>> 199, in raise_exception
>>>     raise RemoteError(en, ev, etb, ei)
>>> RemoteError: NameError(name 'plt' is not defined)
>>
>>
>> The only thing that uses matplotlib is pandas, and modifying
>>
>>  imports = ['import numpy as np', 'import matplotlib.pyplot as plt',
>>> 'import pandas as pd']
>>
>>
>> seems to solve the problem (although at least in one case the first call
>> of my code crashed with the error and the second went through).
>>
>> If I run my code without requiring the ipy parallel I don't have any
>> problem with 'plt'
>>
>> I guess that this is a bug. But I still haven't understood how to debug
>> what happens on the engines, so I can't give more details.
>> Any clues?
>>
>> If needed I can load my 'ipython_parallel.py' module in gist/github
>>
>> Cheers,
>>
>> Fra
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/ada8de74/attachment.html>

From takowl at gmail.com  Mon Sep 16 15:55:49 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 16 Sep 2013 12:55:49 -0700
Subject: [IPython-dev] Ipython output uses the previous version of the
 file to show wrong code line.
In-Reply-To: <CAOvn4qi9e1RCd3zsMoJfo6Bx9LWyS0J-omEtB4q4OpASAn+KDg@mail.gmail.com>
References: <5364828.hxnvyCpGHo@localhost.localdomain>
	<CAHNn8BX7osLvQsBWq2tsLkGwdzwic301_PG4Qs_rovfQ0fpAvA@mail.gmail.com>
	<CAOvn4qi9e1RCd3zsMoJfo6Bx9LWyS0J-omEtB4q4OpASAn+KDg@mail.gmail.com>
Message-ID: <CAOvn4qhBq8VZb7hmvhpx7HCGW7ubGU7dxYMdNX5xNcr45wUvKg@mail.gmail.com>

On 14 September 2013 14:52, Thomas Kluyver <takowl at gmail.com> wrote:

> The relevant modules are linecache (in the standard library) and our
> wrapper, ulinecache. I'll dig into this more on Monday.


It appears to be specific to SyntaxErrors, because they use a different
pathway to get the line from the file. I've put together a fix for this in
PR 4218.

https://github.com/ipython/ipython/pull/4218

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/b970d552/attachment.html>

From arenold at ischool.berkeley.edu  Mon Sep 16 20:30:00 2013
From: arenold at ischool.berkeley.edu (AJ Renold)
Date: Mon, 16 Sep 2013 17:30:00 -0700
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <A6F96291-800B-42B2-B2A7-C444F14E34BB@gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
	<46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
	<CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>
	<CADwqtCNd+peq34AgegCfx3O_f8nb5i-YrVPjZ4DO45r_K9sNTg@mail.gmail.com>
	<CABY8EU4d0F3vqEnnS07OTHhAKVUswaO2=b+uEyVDXm4bh-gmUg@mail.gmail.com>
	<A6F96291-800B-42B2-B2A7-C444F14E34BB@gmail.com>
Message-ID: <CABY8EU4OJFC1HVXP8xtZ_9=+NQtemCNto3yxw0KgTuGNQ3hrYg@mail.gmail.com>

Yes, the directory of the venv IPython is what I expected. I have the habit
of creating all of my work inside of a Dropbox folder for redundancy
backing up any work.

I'll create a new message thread to discuss completer.js and IPEP 11 when
I've had a chance to better understand the current implementation and the
changes proposed.

Thanks for the welcome support!



On Mon, Sep 16, 2013 at 11:37 AM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> The venv IPython seem to use the Dropbox IPython folder, is this what you
> expect ?
>
> Note that my comment about IPep 11 (even if it only touch python side)
> might be relevant to the refactor of completer.js to try to prepare it for
> potential refactor. (Yes I know completer.js should be re-written from
> scratch using CM hint helpers for example).
>
> Envoy? de mon iPhone
>
> Le 16 sept. 2013 ? 20:28, AJ Renold <arenold at ischool.berkeley.edu> a
> ?crit :
>
> Thanks everyone. It seems Gabriel's method of running IPython works. I
> forgot to mention this is on a system with IPython already installed
> system-wide. Here is the difference running from the root directory of the
> repository with ipython installed in a virtualenv:
>
> Output of "print IPython.sys_info()"
>
> after running IPython with $ python -m IPython notebook
>
> {'codename': 'An Afternoon Hack',
>  'commit_hash': '2dd6b4d',
>  'commit_source': 'repository',
>  'default_encoding': 'UTF-8',
>  'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
>  'ipython_version': '2.0.0-dev',
>  'os_name': 'posix',
>  'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
>  'sys_executable': '/Users/ajrenold/Dropbox/Code/ipython/venv/bin/python',
>  'sys_platform': 'darwin',
>  'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}
>
>
> and after running with $ ipython notebook
>
> {'codename': 'An Afternoon Hack',
>  'commit_hash': '2dd6b4d',
>  'commit_source': 'repository',
>  'default_encoding': 'UTF-8',
>  'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
>  'ipython_version': '2.0.0-dev',
>  'os_name': 'posix',
>  'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
>  'sys_executable': '/usr/bin/python',
>  'sys_platform': 'darwin',
>  'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}
>
>
> Looks like I'm already out of date compared to today's most recent commit!
>
> Also, thank you for all of the comments on the completion system. I was
> planning to contribute to completer.js and was not aware of IPEP 11: Tab
> Completion System Refactor<https://github.com/ipython/ipython/wiki/IPEP-11:-Tab-Completion-System-Refactor>.
> I'll take a look at IPEP 11 in more detail to see if it is something I can
> contribute to.
>
> AJ
>
> On Mon, Sep 16, 2013 at 10:49 AM, Gabriel Becker <gmbecker at ucdavis.edu>wrote:
>
>> AJ,
>>
>> I've never had any problem modifying javascript files and having the
>> changes show up when running the following from the root directory of a
>> clone of the ipython repository (on a system with all the dependencies
>> available):
>>
>> python -m IPython notebook
>>
>> I'm not a python guru so I may be doing something silly here but it has
>> always worked for me (and I have spent quite a bit of time modifying the
>> notebook javascript machinery).
>>
>> ~G
>>
>>
>> On Mon, Sep 16, 2013 at 10:23 AM, Thomas Kluyver <takowl at gmail.com>wrote:
>>
>>> On 16 September 2013 01:47, Matthias BUSSONNIER <
>>> bussonniermatthias at gmail.com> wrote:
>>>
>>>> I would also suggest you get a look at IPython Ehencement Proposal 11
>>>> About completion.
>>>>
>>>>
>>>> https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor<https://github.com/ipython/ipython/wiki/IPEP-11:-Tab-Completion-System-Refactor>
>>>>
>>>> If at the same time you could try to make the API for the completer
>>>> configurable, that would be great.
>>>>
>>>
>>> I think that IPEP is mostly about the Python side, whereas issue #3653
>>> is probably something in the notebook javascript, because the same
>>> completion does work in the terminal.
>>>
>>> Thomas
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>>
>> --
>> Gabriel Becker
>> Graduate Student
>> Statistics Department
>> University of California, Davis
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130916/dc9f3687/attachment.html>

From asmeurer at gmail.com  Mon Sep 16 23:59:14 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 16 Sep 2013 21:59:14 -0600
Subject: [IPython-dev] New IPython Contributor,
	help with development environment?
In-Reply-To: <CABY8EU4OJFC1HVXP8xtZ_9=+NQtemCNto3yxw0KgTuGNQ3hrYg@mail.gmail.com>
References: <CABY8EU4pMYtDVmPfF55Yacud8cOxcmeiZOeEh-VriFu-h-kDFw@mail.gmail.com>
	<46FF1B2E-A151-470A-B49C-78905B3D0978@gmail.com>
	<CAOvn4qj2ZiX6AnFzzLQ1GX3yL=h5jbwRHNXsWougbmEM-S8hXg@mail.gmail.com>
	<CADwqtCNd+peq34AgegCfx3O_f8nb5i-YrVPjZ4DO45r_K9sNTg@mail.gmail.com>
	<CABY8EU4d0F3vqEnnS07OTHhAKVUswaO2=b+uEyVDXm4bh-gmUg@mail.gmail.com>
	<A6F96291-800B-42B2-B2A7-C444F14E34BB@gmail.com>
	<CABY8EU4OJFC1HVXP8xtZ_9=+NQtemCNto3yxw0KgTuGNQ3hrYg@mail.gmail.com>
Message-ID: <CAKgW=6+RKL9krAV9mkfK=fjMiaJt2KOsCn3_KHpZ5O=Rt3nD-Q@mail.gmail.com>

Just a warning, if the Dropbox merge algorithm ever kicks in (i.e.,
you do stuff on two computers before it gets a chance to sync
everything), it will completely bork up the .git directory of any git
repository. If you don't use multiple computers, it won't be an issue.

Aaron Meurer

On Mon, Sep 16, 2013 at 6:30 PM, AJ Renold <arenold at ischool.berkeley.edu> wrote:
> Yes, the directory of the venv IPython is what I expected. I have the habit
> of creating all of my work inside of a Dropbox folder for redundancy backing
> up any work.
>
> I'll create a new message thread to discuss completer.js and IPEP 11 when
> I've had a chance to better understand the current implementation and the
> changes proposed.
>
> Thanks for the welcome support!
>
>
>
> On Mon, Sep 16, 2013 at 11:37 AM, Matthias Bussonnier
> <bussonniermatthias at gmail.com> wrote:
>>
>> The venv IPython seem to use the Dropbox IPython folder, is this what you
>> expect ?
>>
>> Note that my comment about IPep 11 (even if it only touch python side)
>> might be relevant to the refactor of completer.js to try to prepare it for
>> potential refactor. (Yes I know completer.js should be re-written from
>> scratch using CM hint helpers for example).
>>
>> Envoy? de mon iPhone
>>
>> Le 16 sept. 2013 ? 20:28, AJ Renold <arenold at ischool.berkeley.edu> a ?crit
>> :
>>
>> Thanks everyone. It seems Gabriel's method of running IPython works. I
>> forgot to mention this is on a system with IPython already installed
>> system-wide. Here is the difference running from the root directory of the
>> repository with ipython installed in a virtualenv:
>>
>> Output of "print IPython.sys_info()"
>>
>> after running IPython with $ python -m IPython notebook
>>
>> {'codename': 'An Afternoon Hack',
>>  'commit_hash': '2dd6b4d',
>>  'commit_source': 'repository',
>>  'default_encoding': 'UTF-8',
>>  'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
>>  'ipython_version': '2.0.0-dev',
>>  'os_name': 'posix',
>>  'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
>>  'sys_executable': '/Users/ajrenold/Dropbox/Code/ipython/venv/bin/python',
>>  'sys_platform': 'darwin',
>>  'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1
>> Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}
>>
>>
>> and after running with $ ipython notebook
>>
>>
>>
>> {'codename': 'An Afternoon Hack',
>>  'commit_hash': '2dd6b4d',
>>  'commit_source': 'repository',
>>  'default_encoding': 'UTF-8',
>>  'ipython_path': '/Users/ajrenold/Dropbox/Code/ipython/IPython',
>>  'ipython_version': '2.0.0-dev',
>>  'os_name': 'posix',
>>  'platform': 'Darwin-12.4.0-x86_64-i386-64bit',
>>  'sys_executable': '/usr/bin/python',
>>  'sys_platform': 'darwin',
>>  'sys_version': '2.7.2 (default, Oct 11 2012, 20:14:37) \n[GCC 4.2.1
>> Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)]'}
>>
>>
>> Looks like I'm already out of date compared to today's most recent commit!
>>
>> Also, thank you for all of the comments on the completion system. I was
>> planning to contribute to completer.js and was not aware of IPEP 11: Tab
>> Completion System Refactor. I'll take a look at IPEP 11 in more detail to
>> see if it is something I can contribute to.
>>
>> AJ
>>
>> On Mon, Sep 16, 2013 at 10:49 AM, Gabriel Becker <gmbecker at ucdavis.edu>
>> wrote:
>>>
>>> AJ,
>>>
>>> I've never had any problem modifying javascript files and having the
>>> changes show up when running the following from the root directory of a
>>> clone of the ipython repository (on a system with all the dependencies
>>> available):
>>>
>>> python -m IPython notebook
>>>
>>> I'm not a python guru so I may be doing something silly here but it has
>>> always worked for me (and I have spent quite a bit of time modifying the
>>> notebook javascript machinery).
>>>
>>> ~G
>>>
>>>
>>> On Mon, Sep 16, 2013 at 10:23 AM, Thomas Kluyver <takowl at gmail.com>
>>> wrote:
>>>>
>>>> On 16 September 2013 01:47, Matthias BUSSONNIER
>>>> <bussonniermatthias at gmail.com> wrote:
>>>>>
>>>>> I would also suggest you get a look at IPython Ehencement Proposal 11
>>>>> About completion.
>>>>>
>>>>>
>>>>> https://github.com/ipython/ipython/wiki/IPEP-11%3A-Tab-Completion-System-Refactor
>>>>>
>>>>> If at the same time you could try to make the API for the completer
>>>>> configurable, that would be great.
>>>>
>>>>
>>>> I think that IPEP is mostly about the Python side, whereas issue #3653
>>>> is probably something in the notebook javascript, because the same
>>>> completion does work in the terminal.
>>>>
>>>> Thomas
>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>>
>>> --
>>> Gabriel Becker
>>> Graduate Student
>>> Statistics Department
>>> University of California, Davis
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From franz.bergesund at gmail.com  Tue Sep 17 05:26:43 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Tue, 17 Sep 2013 11:26:43 +0200
Subject: [IPython-dev] [parallel] issue when executing 'import pandas as
 pd' on engine
In-Reply-To: <CAOCdBK+FqW9eBQ7yVBqfWBBkYLrFhvu65TJTH+=E4=8WFbGX1Q@mail.gmail.com>
References: <CAOCdBK+HGWQQ977ZjSG91BsJVf3CVrMEaOs1Fr+dZm6NU4o79w@mail.gmail.com>
	<CAHNn8BVNe21ojvEuGO39zhpxytppFewjF_6tv8tvqmpWB9T87A@mail.gmail.com>
	<CAOCdBK+FqW9eBQ7yVBqfWBBkYLrFhvu65TJTH+=E4=8WFbGX1Q@mail.gmail.com>
Message-ID: <CAOCdBKKxRQahB126dTGu53wyO=0DPMhrZLgcnLSHy5h_Vae_qw@mail.gmail.com>

Dear Min,

Thanks to the notebook I've found the problem:
the engines (at a certain point) execute the only startup script that I
have (where I call `plt.ion()` see the notebook) but apparently ignores the
ipython_configure.py.

Here is a notebook that show the problem with some more explanation:
http://nbviewer.ipython.org/6592051

I did use the same config files and startup scripts in ipy v0.13, with the
same parallel code and never had the issue.
When I start a console ipy is initialised correctly and `plt.ion()` called
without problems.

If you need anything else let me know,

Fra


2013/9/16 Francesco Montesano <franz.bergesund at gmail.com>

> These codes of mine can be few hundred lines and depend upon other 2 or 3
> modules that I have put together.
>
> If the problem is where I think it is, I might be able to write a
> standalone program (maybe in a notebook). Stay tuned :D
>
> Fra
>
>
>
>
>
> 2013/9/16 MinRK <benjaminrk at gmail.com>
>
>> Cn you please share your whole code?
>>
>>
>> On Mon, Sep 16, 2013 at 9:03 AM, Francesco Montesano <
>> franz.bergesund at gmail.com> wrote:
>>
>>> Dear all,
>>>
>>> I'm having an issue with the parallel (v1.1.0, under python 2.7).
>>>
>>> Some time ago I did build a number of python codes to manipulate
>>> catalogues.
>>> I can have either thousands of small file or few possibly huge file.
>>> So I've written my codes such that I can chose from command whether to
>>> use any of the two.
>>>
>>> Typically my codes have the following structure:
>>>
>>> import numpy as np
>>>> import pandas as pd
>>>> def parse(...):  #argparse
>>>>     ....
>>>> def to_do(fname,...):  #function(s) that do what I need
>>>>     ....
>>>> if __name__=='__main__':
>>>>     args = parse(...)
>>>>     if args.paralell == False:
>>>>          for fn in file_name_list:
>>>>               to_do(fn, ...)
>>>
>>>      else:  #execute in parallel
>>>
>>>          parallel_env = Lbv()       # custom class that init a load
>>>> ballance view
>>>
>>>          imports = ['import numpy as np', 'import pandas as pd']
>>>
>>>          parallel_env.exec_on_engine(imports) # execute the above
>>>> strings on all engines (direct view)
>>>
>>>          #execute in parallel
>>>
>>>          runs = [parallel_env.apply(to_do, os.path.abspath(fn), ...) for
>>>> fn in file_name_list]
>>>
>>>
>>>
>>> I build the whole parallel dispatching against ipython 0.13 and
>>> everything worked fine.
>>> But today I've tried to run one of my scripts enabling parallel and got
>>> the following error
>>>
>>>
>>>>   File "code.py", line XXX, in <module>
>>>>     parallel_env.exec_on_engine(imports)
>>>>   File "XXX/ipython_parallel.py", line 86, in exec_on_engine
>>>>     e.raise_exception()
>>>>   File
>>>> "XXX.local/lib/python2.7/site-packages/IPython/parallel/error.py", line
>>>> 199, in raise_exception
>>>>     raise RemoteError(en, ev, etb, ei)
>>>> RemoteError: NameError(name 'plt' is not defined)
>>>
>>>
>>> The only thing that uses matplotlib is pandas, and modifying
>>>
>>>  imports = ['import numpy as np', 'import matplotlib.pyplot as plt',
>>>> 'import pandas as pd']
>>>
>>>
>>> seems to solve the problem (although at least in one case the first call
>>> of my code crashed with the error and the second went through).
>>>
>>> If I run my code without requiring the ipy parallel I don't have any
>>> problem with 'plt'
>>>
>>> I guess that this is a bug. But I still haven't understood how to debug
>>> what happens on the engines, so I can't give more details.
>>> Any clues?
>>>
>>> If needed I can load my 'ipython_parallel.py' module in gist/github
>>>
>>> Cheers,
>>>
>>> Fra
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130917/fee185bf/attachment.html>

From tmrlvi at gmail.com  Tue Sep 17 17:54:05 2013
From: tmrlvi at gmail.com (Tomer Levi)
Date: Wed, 18 Sep 2013 00:54:05 +0300
Subject: [IPython-dev] IPython Notebook Memory Usage Feature
Message-ID: <CAHtwKSh8WH+RAFekjGaja26u8AZpzFEU9OLLZB_xs0SdE8ad3Q@mail.gmail.com>

Hi everyone,

Recently I had a problem remembering which of my IPython Notebook kernels
were using a large portion of my computer memory.
So, I've created my a quick patch that added the KernelManager a method
that can extract it, and displayed it on the tree webpage of the Notebook.
(It depends on the module *psutil *to do so...)

I'd appreciate any comments. :)
Tomer

P.S:
The patch is in the attached file, as instructed in
http://ipython.org/ipython-doc/stable/development/gitwash/patching.html
If it isn't the way to introduce a patch, I'll be glad to know and fix it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130918/af01812d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-notebook-memory-usage.patch
Type: application/octet-stream
Size: 4069 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130918/af01812d/attachment.obj>

From zvoros at gmail.com  Wed Sep 18 08:10:18 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Wed, 18 Sep 2013 14:10:18 +0200
Subject: [IPython-dev] help menu triggered on executed cell
Message-ID: <5239982A.3000204@gmail.com>

Hi all,

I run the latest ipython from github, and I noticed that the pop-up help 
in the HTML notebook is displayed, even if the cell has already been 
executed.

Example:



Note that the cursor is already in the next line. I was wondering, 
whether this is the intended behaviour (my recollection is that in the 
past this didn't happen), and whether others can reproduce it, if it is 
indeed a bug.

Cheers,
Zolt?n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130918/1e954c2d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: djbbicfj.png
Type: image/png
Size: 17396 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130918/1e954c2d/attachment.png>

From sirinath at sakrio.com  Wed Sep 18 23:42:37 2013
From: sirinath at sakrio.com (Suminda Dharmasena)
Date: Thu, 19 Sep 2013 09:12:37 +0530
Subject: [IPython-dev] Org Mode and DADiSP like Functionality
Message-ID: <CAAfFNYFfRhAhWuesTt1QdF+SA_+_aY+-XdZ7n=pMJ-MLxYhj3Q@mail.gmail.com>

Hi,

Is it possible to provide Org Mode (http://orgmode.org/ - the website says
editing and planning but there is a lot more applications than that -
http://orgmode.org/manual/index.html, Scientific Papaers -
http://orgmode.org/worg/org-papers.html, Guide - http://orgmode.org/guide/,
FAQ - http://orgmode.org/worg/org-faq.html, http://orgmode.org/worg/doc.html)
and DADiSP (http://www.dadisp.com/) like functionality along with the
current functionality. Also perhaps define general workflows / scientific
workflows.

You can have richer task lists with scripting ability as well as
spreadsheet like functionality. Also there are other Org Mode like projects
also which you can get a idea from. Scripting can be restricted to JVM
language and perhaps the community can add extra functionality like Org
Mode (http://orgmode.org/worg/org-tools/index.html).

Also Processing has a JS version called Processingjs that can help better
visualisations. Also D3.

Suminda
--
Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.) Lond.,
P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*: 6G ? 1st
Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*
: +94-(0)711007945 ? *Office*: +94-(0) 11 2 199766 / 5 875614 ?
*Tele*: +94-(0)11-5
864614 / 2 825908 ? *Web*: http://www.sakrio.com ?

This email is subjected to the email Terms of Use and Disclaimer:
http://www.sakrio.com/email-legal. Please read this first.
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/9b4f66dc/attachment.html>

From sirinath at sakrio.com  Thu Sep 19 00:06:29 2013
From: sirinath at sakrio.com (Suminda Dharmasena)
Date: Thu, 19 Sep 2013 09:36:29 +0530
Subject: [IPython-dev] Security
Message-ID: <CAAfFNYHHsThvFgAFv8ePDxbJ8cr9+P511BF9AZAUvV6_NwprGQ@mail.gmail.com>

Hi,

What is the level of security and sand boxing built in notebook so that the
Python scripts write cannot evoke any dangerous applications. E.g. use
kernel or system functions, write or read to disk, make network
connections, etc.

If notebook is to be used on a web facing front end project accessible to
everyone this will need some thinking about.

Suminda

--
Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.) Lond.,
P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*: 6G ? 1st
Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*
: +94-(0)711007945 ? *Office*: +94-(0) 11 2 199766 / 5 875614 ?
*Tele*: +94-(0)11-5
864614 / 2 825908 ? *Web*: http://www.sakrio.com ?

This email is subjected to the email Terms of Use and Disclaimer:
http://www.sakrio.com/email-legal. Please read this first.
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/0b16097e/attachment.html>

From benjaminrk at gmail.com  Thu Sep 19 00:12:33 2013
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 18 Sep 2013 21:12:33 -0700
Subject: [IPython-dev] Security
In-Reply-To: <CAAfFNYHHsThvFgAFv8ePDxbJ8cr9+P511BF9AZAUvV6_NwprGQ@mail.gmail.com>
References: <CAAfFNYHHsThvFgAFv8ePDxbJ8cr9+P511BF9AZAUvV6_NwprGQ@mail.gmail.com>
Message-ID: <CAHNn8BX4S7rf0oxSN12WB2OU-AUd1Qfoz-mnr9aFQ=nmPd-8rg@mail.gmail.com>

On Wed, Sep 18, 2013 at 9:06 PM, Suminda Dharmasena <sirinath at sakrio.com>wrote:

> Hi,
>
> What is the level of security and sand boxing built in notebook so that
> the Python scripts write cannot evoke any dangerous applications. E.g. use
> kernel or system functions, write or read to disk, make network
> connections, etc.
>

There is none at all - a logged in notebook has full access to do anything
they want as you on the system.


>
> If notebook is to be used on a web facing front end project accessible to
> everyone this will need some thinking about.
>

That's correct.


>
> Suminda
>
> --
> Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.)
> Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM, CEO Sakr??! ? *Address*:
> 6G ? 1st Lane ? Pagoda Road ? Nugegoda 10250 ? Sri Lanka. ? *Mobile*:
> +94-(0)711007945 ? *Office*: +94-(0) 11 2 199766 / 5 875614 ? *Tele*: +94-(0)11-5
> 864614 / 2 825908 ? *Web*: http://www.sakrio.com ?
>
> This email is subjected to the email Terms of Use and Disclaimer:
> http://www.sakrio.com/email-legal. Please read this first.
> --
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130918/1c3695c8/attachment.html>

From pi at berkeley.edu  Thu Sep 19 00:30:01 2013
From: pi at berkeley.edu (Paul Ivanov)
Date: Wed, 18 Sep 2013 21:30:01 -0700
Subject: [IPython-dev] Org Mode and DADiSP like Functionality
In-Reply-To: <CAAfFNYFfRhAhWuesTt1QdF+SA_+_aY+-XdZ7n=pMJ-MLxYhj3Q@mail.gmail.com>
References: <CAAfFNYFfRhAhWuesTt1QdF+SA_+_aY+-XdZ7n=pMJ-MLxYhj3Q@mail.gmail.com>
Message-ID: <20130919043001.GK10020@HbI-OTOH.berkeley.edu>

Suminda Dharmasena, on 2013-09-19 09:12,  wrote:
> Is it possible to provide Org Mode (http://orgmode.org/ - the website says
> editing and planning but there is a lot more applications than that -
> http://orgmode.org/manual/index.html, Scientific Papaers -
> http://orgmode.org/worg/org-papers.html, Guide - http://orgmode.org/guide/,
> FAQ - http://orgmode.org/worg/org-faq.html, http://orgmode.org/worg/doc.html)
> and DADiSP (http://www.dadisp.com/) like functionality along with the
> current functionality. Also perhaps define general workflows / scientific
> workflows.
> 
> You can have richer task lists with scripting ability as well as
> spreadsheet like functionality. Also there are other Org Mode like projects
> also which you can get a idea from. Scripting can be restricted to JVM
> language and perhaps the community can add extra functionality like Org
> Mode (http://orgmode.org/worg/org-tools/index.html).

I'm not sure what the question is here, but at SciPy this year,
John Kitchin from CMU gave a talk in the Reproducible Research
track you might be interested in that was titled "Emacs +
org-mode + python in reproducible research". John does not use
IPython notebook, however, this is just Emacs and plain old
Python. You can find the talk abstract, a link to the youtube
video and other resources here:

http://conference.scipy.org/scipy2013/presentation_detail.php?id=178

Perhaps there's a way of integrating EIN - Emacs IPython Notebook
client (https://github.com/tkf/emacs-ipython-notebook) with
orgmode, so maybe that's what you're looking for? I do not use
Emacs, so I don't know what that would entail, or if that even
makes sense, but it seems relevant to mention on this list given
the content of your email.

best,
-- 
                   _
                  / \
                A*   \^   -
             ,./   _.`\\ / \
            / ,--.S    \/   \
           /  `"~,_     \    \
     __o           ?
   _ \<,_         /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
http://pirsquared.org


From wstein at gmail.com  Thu Sep 19 02:30:55 2013
From: wstein at gmail.com (William Stein)
Date: Wed, 18 Sep 2013 23:30:55 -0700
Subject: [IPython-dev] Security
In-Reply-To: <CAHNn8BX4S7rf0oxSN12WB2OU-AUd1Qfoz-mnr9aFQ=nmPd-8rg@mail.gmail.com>
References: <CAAfFNYHHsThvFgAFv8ePDxbJ8cr9+P511BF9AZAUvV6_NwprGQ@mail.gmail.com>
	<CAHNn8BX4S7rf0oxSN12WB2OU-AUd1Qfoz-mnr9aFQ=nmPd-8rg@mail.gmail.com>
Message-ID: <CACLE5GAVztBghVLvec3=ohD-VZwx-R9P9=Wh4LjFqE3oPU7+Wg@mail.gmail.com>

On Wed, Sep 18, 2013 at 9:12 PM, MinRK <benjaminrk at gmail.com> wrote:
> On Wed, Sep 18, 2013 at 9:06 PM, Suminda Dharmasena <sirinath at sakrio.com>
> wrote:
>> What is the level of security and sand boxing built in notebook so that
>> the Python scripts write cannot evoke any dangerous applications. E.g. use
>> kernel or system functions, write or read to disk, make network connections,
>> etc.
>
> There is none at all - a logged in notebook has full access to do anything
> they want as you on the system.
>
>> If notebook is to be used on a web facing front end project accessible to
>> everyone this will need some thinking about.

I thought about this problem recently [1].     In case anybody is
interested, here's what I've done for  https://cloud.sagemath.com;
which might be similar to what's done with https://www.wakari.io/
(another site that hosts ipython notebooks hopefully securely).   In
cloud.sagemath the security model is that each user can create many
projects, and in turn each project can also have many collaborators.
Technically, a project is exactly the same thing as an account on a
Linux box -- and project collaborators have *full* access to this
Linux account (just like users on Amazon EC2 have full access to their
VM's).     Right now projects run in one of many VM's, but without
much further isolation; in the future, they will live in LXC
containers in a VM.

When a user opens an IPython notebook in cloud.sagemath, an IPython
notebook server is started in their project as that user (if one isn't
already running for the directory containing the file), and set to
listen only on the *external* network interface of that virtual
machine; in particular, the notebook server does not listen on
localhost.  The VM's are firewalled, so they can't connect to ports
above 1023 between each other.  This way no other user on any other VM
(or localhost) can connect to the IPython notebook server directly, so
its not necessary to use IPython's own login/password support or SSL
support.

Connections to the IPython notebook server happen via the following
pathway (with everything after stunnel going over an encrypted VPN):

           client --> stunnel --> haproxy --> node-proxy --> ipython
notebook server

Here node-proxy is an HTTP proxy server written in node.js, which
consults a database to decide whether the given client is allowed to
connect to the given project.  If so, the proxy is created; if not,
they get an error.  Also, haproxy does load balancing across many
different proxy servers.

The upshot is that you can share ipynb's with a specified list of
other users, and access is limited to only those users. However, each
user has their own credentials.

[1] http://sagemath.blogspot.com/2013/09/ipython-notebooks-in-cloud-with.html

 -- William

-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org


From pelson.pub at gmail.com  Thu Sep 19 03:56:59 2013
From: pelson.pub at gmail.com (Phil Elson)
Date: Thu, 19 Sep 2013 08:56:59 +0100
Subject: [IPython-dev] Security
In-Reply-To: <CACLE5GAVztBghVLvec3=ohD-VZwx-R9P9=Wh4LjFqE3oPU7+Wg@mail.gmail.com>
References: <CAAfFNYHHsThvFgAFv8ePDxbJ8cr9+P511BF9AZAUvV6_NwprGQ@mail.gmail.com>
	<CAHNn8BX4S7rf0oxSN12WB2OU-AUd1Qfoz-mnr9aFQ=nmPd-8rg@mail.gmail.com>
	<CACLE5GAVztBghVLvec3=ohD-VZwx-R9P9=Wh4LjFqE3oPU7+Wg@mail.gmail.com>
Message-ID: <CA+L60sCKWCfMkxqmBGibe0xhU4jnR6gjC7+2NZe6KSUaMkSGbA@mail.gmail.com>

Thanks for sharing how sagemath has done it. I'm also aware of
https://github.com/cni/ipython-hydra which authenticates IPython sessions
via LDAP.

I've been pondering how to go about setting up an enterprise installation
of IPython notebooks for 300+ internal scientific users with each of their
desktops hosting a single instance accessible by just one user (i.e. no
collaboration at this stage) available only on the internal network. For
me, the LDAP approach looks like the way to go, but there are other
security concerns (such as the automatic execution of Python from html at
notebook open time) which also make me a little nervous. My problem is that
for the notebook to be of any value to the users I support, it needs to be
running on their desktops, which by its very nature isn't as throwaway as a
garden walled VM.

Anyone else have any experience of configuring up a similar setup? Did you
harden the notebook against malicious ipynb creators?

Thanks for any insight.



On 19 September 2013 07:30, William Stein <wstein at gmail.com> wrote:

> On Wed, Sep 18, 2013 at 9:12 PM, MinRK <benjaminrk at gmail.com> wrote:
> > On Wed, Sep 18, 2013 at 9:06 PM, Suminda Dharmasena <sirinath at sakrio.com
> >
> > wrote:
> >> What is the level of security and sand boxing built in notebook so that
> >> the Python scripts write cannot evoke any dangerous applications. E.g.
> use
> >> kernel or system functions, write or read to disk, make network
> connections,
> >> etc.
> >
> > There is none at all - a logged in notebook has full access to do
> anything
> > they want as you on the system.
> >
> >> If notebook is to be used on a web facing front end project accessible
> to
> >> everyone this will need some thinking about.
>
> I thought about this problem recently [1].     In case anybody is
> interested, here's what I've done for  https://cloud.sagemath.com;
> which might be similar to what's done with https://www.wakari.io/
> (another site that hosts ipython notebooks hopefully securely).   In
> cloud.sagemath the security model is that each user can create many
> projects, and in turn each project can also have many collaborators.
> Technically, a project is exactly the same thing as an account on a
> Linux box -- and project collaborators have *full* access to this
> Linux account (just like users on Amazon EC2 have full access to their
> VM's).     Right now projects run in one of many VM's, but without
> much further isolation; in the future, they will live in LXC
> containers in a VM.
>
> When a user opens an IPython notebook in cloud.sagemath, an IPython
> notebook server is started in their project as that user (if one isn't
> already running for the directory containing the file), and set to
> listen only on the *external* network interface of that virtual
> machine; in particular, the notebook server does not listen on
> localhost.  The VM's are firewalled, so they can't connect to ports
> above 1023 between each other.  This way no other user on any other VM
> (or localhost) can connect to the IPython notebook server directly, so
> its not necessary to use IPython's own login/password support or SSL
> support.
>
> Connections to the IPython notebook server happen via the following
> pathway (with everything after stunnel going over an encrypted VPN):
>
>            client --> stunnel --> haproxy --> node-proxy --> ipython
> notebook server
>
> Here node-proxy is an HTTP proxy server written in node.js, which
> consults a database to decide whether the given client is allowed to
> connect to the given project.  If so, the proxy is created; if not,
> they get an error.  Also, haproxy does load balancing across many
> different proxy servers.
>
> The upshot is that you can share ipynb's with a specified list of
> other users, and access is limited to only those users. However, each
> user has their own credentials.
>
> [1]
> http://sagemath.blogspot.com/2013/09/ipython-notebooks-in-cloud-with.html
>
>  -- William
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/c145b651/attachment.html>

From bussonniermatthias at gmail.com  Thu Sep 19 04:40:59 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 19 Sep 2013 10:40:59 +0200
Subject: [IPython-dev] Security
In-Reply-To: <CA+L60sCKWCfMkxqmBGibe0xhU4jnR6gjC7+2NZe6KSUaMkSGbA@mail.gmail.com>
References: <CAAfFNYHHsThvFgAFv8ePDxbJ8cr9+P511BF9AZAUvV6_NwprGQ@mail.gmail.com>
	<CAHNn8BX4S7rf0oxSN12WB2OU-AUd1Qfoz-mnr9aFQ=nmPd-8rg@mail.gmail.com>
	<CACLE5GAVztBghVLvec3=ohD-VZwx-R9P9=Wh4LjFqE3oPU7+Wg@mail.gmail.com>
	<CA+L60sCKWCfMkxqmBGibe0xhU4jnR6gjC7+2NZe6KSUaMkSGbA@mail.gmail.com>
Message-ID: <CDCA332E-3134-488C-A5D3-9C026CFF9162@gmail.com>


Le 19 sept. 2013 ? 09:56, Phil Elson a ?crit :

> Thanks for sharing how sagemath has done it. I'm also aware of https://github.com/cni/ipython-hydra which authenticates IPython sessions via LDAP.
> 
> I've been pondering how to go about setting up an enterprise installation of IPython notebooks for 300+ internal scientific users with each of their desktops hosting a single instance accessible by just one user (i.e. no collaboration at this stage) available only on the internal network.


> For me, the LDAP approach looks like the way to go, but there are other security concerns (such as the automatic execution of Python from html at notebook open time) which also make me a little nervous.

Yes we are aware of this one.
Right now, the displayed javascript (in output) is not executed at loaded time, but only if user execute the cell. 
Still you are right that js script can be embedded in script tags in html. 
Right now lots of code **rely** on this, and we need to develop js-plugin architecture *before* sanitizing the html. 
But it will be done.

> My problem is that for the notebook to be of any value to the users I support, it needs to be running on their desktops, which by its very nature isn't as throwaway as a garden walled VM.

Usually I would suggest to setup locally a service close to nbviewer
http://nbviewer.ipython.org/
We will build it as part of IPython soon, it allows to view notebooks without executing them.

You also have the possibility to launch kernels in LXC or any other kind of jail.
Also keep in mind that notebook are meant to be executed, so one you have passed the need to view a static 
version of IPynb, I'm not sure the danger of .ipynb file is much greater than any other script files that scientist usually 
get and run without asking any questions. 
Nothing prevent me to send a "forged" python script that os.remove('/').

For the small story, the bigger security issue we have in my lab is people searching on google 
   webmail <name of our lab>
Fist link is often a classical "Outlook Home page"?where they enter their username and password.
I let you guess the rest.


> Anyone else have any experience of configuring up a similar setup?

> Did you harden the notebook against malicious ipynb creators?
Displayed Js is not executed at load time.
We will sanitize html.
We are working on having a json-validator that will refuse to send the json to the browser if it does not respect 
a json scheme.

> Thanks for any insight.

Even if you jail the kernel, I would be carefull also if the notebook is hosted on a domain where user are authentified.
(like wakari) as malicious code can try to access other resources of this domain.


-- 
Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/aecb54ed/attachment.html>

From eric at depagne.org  Thu Sep 19 04:53:13 2013
From: eric at depagne.org (=?ISO-8859-1?Q?=C9ric?= Depagne)
Date: Thu, 19 Sep 2013 10:53:13 +0200
Subject: [IPython-dev] ipython qtconsole pager problem.
Message-ID: <1893281.EgVyLYPFUq@localhost.localdomain>

Hi all.

I'm bumping into another problem using the qtconsole.
Here's my config: ipython 1.1.0, python 2.7.5

I run the following code with ipython:
%run -p script.py

In a notebook, the profiling is diplayed correctly
In a normal terminal, it is displayed correctly
In a qtconsole, the display goes empty (I do loose the sliders too), and then, nothing 
happens. Using the menu bar, I can open a new tab with the same kernel, indicating 
(I guess) that it's just a display problem of the qtconsole.

Cheers, 
?ric.
-- 
Un clavier azerty en vaut deux
----------------------------------------------------------
?ric Depagne                            eric at depagne.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/637fad5a/attachment.html>

From cyrille.rossant at gmail.com  Thu Sep 19 05:49:36 2013
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Thu, 19 Sep 2013 11:49:36 +0200
Subject: [IPython-dev] pull requests
Message-ID: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>

I have a quick and possibly dumb question: should Pull Requests be
made against master or in a new branch?

Yes according to
https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#pull-requests

No according to
http://ipython.org/ipython-doc/dev/development/gitwash/development_workflow.html#workflow-summary

Could you please clarify?

Thanks!
Cyrille


From bussonniermatthias at gmail.com  Thu Sep 19 06:04:47 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 19 Sep 2013 12:04:47 +0200
Subject: [IPython-dev] pull requests
In-Reply-To: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
Message-ID: <C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>


Le 19 sept. 2013 ? 11:49, Cyrille Rossant a ?crit :

> I have a quick and possibly dumb question: should Pull Requests be
> made against master or in a new branch?
> 
> Yes according to
> https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#pull-requests
> 
> No according to
> http://ipython.org/ipython-doc/dev/development/gitwash/development_workflow.html#workflow-summary

I'm not sure I see where this state that PR should not be done gains master. 
Anyway. 

Usually you will push a named branched on your repo. and you ask to merge this branch into ipython/master
So github will send a mail 

Cyrille want to merge crossant/my-awesome-feature into ipython/master


> 
> Could you please clarify?

Does this help ?

-- 
M

From cyrille.rossant at gmail.com  Thu Sep 19 06:14:43 2013
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Thu, 19 Sep 2013 12:14:43 +0200
Subject: [IPython-dev] pull requests
In-Reply-To: <C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
Message-ID: <CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>

> I'm not sure I see where this state that PR should not be done gains master.
> Anyway.

Ah, I misunderstood then, PR should be made against master on ipython,
but not necessarily on master on one's fork, right?

> Usually you will push a named branched on your repo. and you ask to merge this branch into ipython/master
> So github will send a mail
>
> Cyrille want to merge crossant/my-awesome-feature into ipython/master

So we should always use a new branch in our fork, and never make edits
in master?


From bussonniermatthias at gmail.com  Thu Sep 19 08:06:50 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 19 Sep 2013 14:06:50 +0200
Subject: [IPython-dev] pull requests
In-Reply-To: <CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
Message-ID: <94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>


Le 19 sept. 2013 ? 12:14, Cyrille Rossant a ?crit :

>> I'm not sure I see where this state that PR should not be done gains master.
>> Anyway.
> 
> Ah, I misunderstood then, PR should be made against master on ipython,
> but not necessarily on master on one's fork, right?
Right

> 
>> Usually you will push a named branched on your repo. and you ask to merge this branch into ipython/master
>> So github will send a mail
>> 
>> Cyrille want to merge crossant/my-awesome-feature into ipython/master
> 
> So we should always use a new branch in our fork, and never make edits
> in master?

Technically you can. It is just a pragmatic reasons. 
	- Having people writing on their master increase chance of weird history is one submit many PR and then pull an update version.
	- when you speak of "master" it is not ambiguous.
        - it's easier for a dev to $ git checkout crossant/vizpy-integration as you know what you are expecting and it's easier to search
        - some commit are more descriptive "merge branch fperez/fix-bug-6374 into master" is better than "merge branch ellisonbg/master into master"
	- ? 


Also technically you can cross-push branches, ($ git push github local-name:remote-name) 
so it happen to me to work locally on master and to push on gihub with another name, 
but those are details that we shouldn't expose to newcomers. 

This in one of the reason our doc says (or should say at least) that it is preferable to make the changes 
in named branches which names are as explicit as possible to help future workflow. It is not strictly enforced, but it's a good habit.

-- 
M

From cyrille.rossant at gmail.com  Thu Sep 19 08:21:00 2013
From: cyrille.rossant at gmail.com (Cyrille Rossant)
Date: Thu, 19 Sep 2013 14:21:00 +0200
Subject: [IPython-dev] pull requests
In-Reply-To: <94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
Message-ID: <CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>

2013/9/19 Matthias BUSSONNIER <bussonniermatthias at gmail.com>:
>
> Le 19 sept. 2013 ? 12:14, Cyrille Rossant a ?crit :
>
>>> I'm not sure I see where this state that PR should not be done gains master.
>>> Anyway.
>>
>> Ah, I misunderstood then, PR should be made against master on ipython,
>> but not necessarily on master on one's fork, right?
> Right
>
>>
>>> Usually you will push a named branched on your repo. and you ask to merge this branch into ipython/master
>>> So github will send a mail
>>>
>>> Cyrille want to merge crossant/my-awesome-feature into ipython/master
>>
>> So we should always use a new branch in our fork, and never make edits
>> in master?
>
> Technically you can. It is just a pragmatic reasons.
>         - Having people writing on their master increase chance of weird history is one submit many PR and then pull an update version.
>         - when you speak of "master" it is not ambiguous.
>         - it's easier for a dev to $ git checkout crossant/vizpy-integration as you know what you are expecting and it's easier to search
>         - some commit are more descriptive "merge branch fperez/fix-bug-6374 into master" is better than "merge branch ellisonbg/master into master"
>         - ?
>
>
> Also technically you can cross-push branches, ($ git push github local-name:remote-name)
> so it happen to me to work locally on master and to push on gihub with another name,
> but those are details that we shouldn't expose to newcomers.
>
> This in one of the reason our doc says (or should say at least) that it is preferable to make the changes
> in named branches which names are as explicit as possible to help future workflow. It is not strictly enforced, but it's a good habit.

Great, thanks very much for clarifying.

Cyrille


From johanbeke at hotmail.com  Thu Sep 19 11:43:36 2013
From: johanbeke at hotmail.com (Johan Beke)
Date: Thu, 19 Sep 2013 15:43:36 +0000
Subject: [IPython-dev] nbconvert bug?
Message-ID: <DUB112-W120E344BC1D8F743823B1BCA4210@phx.gbl>

Hello all,

I have a notebook which fails to convert. I've already encountered the problem (see user list) a few weeks ago with another notebook. The message is the same:

johan$ ipython nbconvert Fit_wind.ipynb 
[NbConvertApp] Using existing profile dir: u'/home/johan/.config/ipython/profile_default'
[NbConvertApp] Converting notebook Fit_wind.ipynb to html
[NbConvertApp] Support files will be in Fit_wind_files/
[NbConvertApp] Loaded template html_full.tpl
pandoc: option `--mathjax' requires an argument URL
Try pandoc --help for more information.
Traceback (most recent call last):
  File "/usr/local/bin/ipython", line 6, in <module>
    start_ipython()
  File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", line 118, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 545, in launch_instance
    app.start()
  File "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 358, in start
    return self.subapp.start()
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/nbconvertapp.py", line 267, in start
    self.convert_notebooks()
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/nbconvertapp.py", line 300, in convert_notebooks
    output, resources = exporter.from_filename(notebook_filename, resources=resources)
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/exporter.py", line 289, in from_filename
    return self.from_notebook_node(nbformat.read(f, 'json'), resources=resources,**kw)
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/exporter.py", line 260, in from_notebook_node
    output = self.template.render(nb=nb_copy, resources=resources)
  File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.6-py2.7.egg/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_full.tpl", line 1, in top-level template code
    {%- extends 'html_basic.tpl' -%}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_basic.tpl", line 1, in top-level template code
    {%- extends 'display_priority.tpl' -%}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/display_priority.tpl", line 1, in top-level template code
    {%- extends 'null.tpl' -%}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/null.tpl", line 26, in top-level template code
    {%- block body -%}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_full.tpl", line 62, in block "body"
    {{ super() }}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/null.tpl", line 29, in block "body"
    {%- block any_cell scoped -%}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/null.tpl", line 76, in block "any_cell"
    {%- block headingcell scoped-%}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_basic.tpl", line 64, in block "headingcell"
    {{ ("#" * cell.level + cell.source) | replace('\n', ' ') | strip_math_space | markdown2html | strip_files_prefix | add_anchor }}
  File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/strings.py", line 85, in add_anchor
    h = ElementTree.fromstring(py3compat.cast_bytes_py2(html, encoding='utf-8'))
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1283, in XML
    return parser.close()
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1636, in close
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1488, in _raiseerror
    raise err
ParseError: no element found: line 1, column 0

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev at scipy.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    c.Application.verbose_crash=True

Is this a bug or is there something wrong with my file or system? Notebook at: http://pastebin.com/FZu2j5xY

Kind regards,

Johan
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/07a1a028/attachment.html>

From bussonniermatthias at gmail.com  Thu Sep 19 11:54:10 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 19 Sep 2013 17:54:10 +0200
Subject: [IPython-dev] nbconvert bug?
In-Reply-To: <DUB112-W120E344BC1D8F743823B1BCA4210@phx.gbl>
References: <DUB112-W120E344BC1D8F743823B1BCA4210@phx.gbl>
Message-ID: <007364C7-19AD-4384-89F0-E1B7F08D8594@gmail.com>

Hi there,

Seem to work fine here on master. 

You can grab the result here : 
https://gist.github.com/Carreau/6625498

Are you running dev ? 1.0 ? 1.1 ?

-- 
M


Le 19 sept. 2013 ? 17:43, Johan Beke a ?crit :

> Hello all,
> 
> I have a notebook which fails to convert. I've already encountered the problem (see user list) a few weeks ago with another notebook. The message is the same:
> 
> johan$ ipython nbconvert Fit_wind.ipynb 
> [NbConvertApp] Using existing profile dir: u'/home/johan/.config/ipython/profile_default'
> [NbConvertApp] Converting notebook Fit_wind.ipynb to html
> [NbConvertApp] Support files will be in Fit_wind_files/
> [NbConvertApp] Loaded template html_full.tpl
> pandoc: option `--mathjax' requires an argument URL
> Try pandoc --help for more information.
> Traceback (most recent call last):
>   File "/usr/local/bin/ipython", line 6, in <module>
>     start_ipython()
>   File "/usr/local/lib/python2.7/dist-packages/IPython/__init__.py", line 118, in start_ipython
>     return launch_new_instance(argv=argv, **kwargs)
>   File "/usr/local/lib/python2.7/dist-packages/IPython/config/application.py", line 545, in launch_instance
>     app.start()
>   File "/usr/local/lib/python2.7/dist-packages/IPython/terminal/ipapp.py", line 358, in start
>     return self.subapp.start()
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/nbconvertapp.py", line 267, in start
>     self.convert_notebooks()
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/nbconvertapp.py", line 300, in convert_notebooks
>     output, resources = exporter.from_filename(notebook_filename, resources=resources)
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/exporter.py", line 289, in from_filename
>     return self.from_notebook_node(nbformat.read(f, 'json'), resources=resources,**kw)
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/exporter.py", line 260, in from_notebook_node
>     output = self.template.render(nb=nb_copy, resources=resources)
>   File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.6-py2.7.egg/jinja2/environment.py", line 894, in render
>     return self.environment.handle_exception(exc_info, True)
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_full.tpl", line 1, in top-level template code
>     {%- extends 'html_basic.tpl' -%}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_basic.tpl", line 1, in top-level template code
>     {%- extends 'display_priority.tpl' -%}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/display_priority.tpl", line 1, in top-level template code
>     {%- extends 'null.tpl' -%}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/null.tpl", line 26, in top-level template code
>     {%- block body -%}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_full.tpl", line 62, in block "body"
>     {{ super() }}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/null.tpl", line 29, in block "body"
>     {%- block any_cell scoped -%}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/skeleton/null.tpl", line 76, in block "any_cell"
>     {%- block headingcell scoped-%}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/exporters/../templates/html_basic.tpl", line 64, in block "headingcell"
>     {{ ("#" * cell.level + cell.source) | replace('\n', ' ') | strip_math_space | markdown2html | strip_files_prefix | add_anchor }}
>   File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/strings.py", line 85, in add_anchor
>     h = ElementTree.fromstring(py3compat.cast_bytes_py2(html, encoding='utf-8'))
>   File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1283, in XML
>     return parser.close()
>   File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1636, in close
>     self._raiseerror(v)
>   File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1488, in _raiseerror
>     raise err
> ParseError: no element found: line 1, column 0
> 
> If you suspect this is an IPython bug, please report it at:
>     https://github.com/ipython/ipython/issues
> or send an email to the mailing list at ipython-dev at scipy.org
> 
> You can print a more detailed traceback right now with "%tb", or use "%debug"
> to interactively debug it.
> 
> Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
>     c.Application.verbose_crash=True
> 
> Is this a bug or is there something wrong with my file or system? Notebook at: http://pastebin.com/FZu2j5xY
> 
> Kind regards,
> 
> Johan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/1b7659b4/attachment.html>

From johanbeke at hotmail.com  Thu Sep 19 12:01:50 2013
From: johanbeke at hotmail.com (Johan Beke)
Date: Thu, 19 Sep 2013 16:01:50 +0000
Subject: [IPython-dev] nbconvert bug?
Message-ID: <DUB112-W982C861B10F3236D32F675A4210@phx.gbl>

Version info (running on linux mint): (strange things is that 1.1.0 is mentioned whilst version says 1.0.0)

Python 2.7.2+ (default, Jul 20 2012, 22:15:08) 
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: version
Out[1]: '1.0.0'

Can you point me out to the right release so that I can test this?



Johan 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/c97fd581/attachment.html>

From benjaminrk at gmail.com  Thu Sep 19 12:03:15 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 19 Sep 2013 09:03:15 -0700
Subject: [IPython-dev] pull requests
In-Reply-To: <CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
Message-ID: <CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>

Yes, open a PR against master.  I don't know the provenance of those
gitwash docs, but they say several things that are not true. We should
figure out if we should update them to be accurate, or just remove them
altogether (I would be in favor of the latter).

-MinRK


On Thu, Sep 19, 2013 at 5:21 AM, Cyrille Rossant
<cyrille.rossant at gmail.com>wrote:

> 2013/9/19 Matthias BUSSONNIER <bussonniermatthias at gmail.com>:
> >
> > Le 19 sept. 2013 ? 12:14, Cyrille Rossant a ?crit :
> >
> >>> I'm not sure I see where this state that PR should not be done gains
> master.
> >>> Anyway.
> >>
> >> Ah, I misunderstood then, PR should be made against master on ipython,
> >> but not necessarily on master on one's fork, right?
> > Right
> >
> >>
> >>> Usually you will push a named branched on your repo. and you ask to
> merge this branch into ipython/master
> >>> So github will send a mail
> >>>
> >>> Cyrille want to merge crossant/my-awesome-feature into ipython/master
> >>
> >> So we should always use a new branch in our fork, and never make edits
> >> in master?
> >
> > Technically you can. It is just a pragmatic reasons.
> >         - Having people writing on their master increase chance of weird
> history is one submit many PR and then pull an update version.
> >         - when you speak of "master" it is not ambiguous.
> >         - it's easier for a dev to $ git checkout
> crossant/vizpy-integration as you know what you are expecting and it's
> easier to search
> >         - some commit are more descriptive "merge branch
> fperez/fix-bug-6374 into master" is better than "merge branch
> ellisonbg/master into master"
> >         - ?
> >
> >
> > Also technically you can cross-push branches, ($ git push github
> local-name:remote-name)
> > so it happen to me to work locally on master and to push on gihub with
> another name,
> > but those are details that we shouldn't expose to newcomers.
> >
> > This in one of the reason our doc says (or should say at least) that it
> is preferable to make the changes
> > in named branches which names are as explicit as possible to help future
> workflow. It is not strictly enforced, but it's a good habit.
>
> Great, thanks very much for clarifying.
>
> Cyrille
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/f04704fa/attachment.html>

From stefan at sun.ac.za  Thu Sep 19 12:19:01 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 19 Sep 2013 18:19:01 +0200
Subject: [IPython-dev] pull requests
In-Reply-To: <CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
Message-ID: <CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>

On Thu, Sep 19, 2013 at 6:03 PM, MinRK <benjaminrk at gmail.com> wrote:
> Yes, open a PR against master.  I don't know the provenance of those gitwash
> docs, but they say several things that are not true. We should figure out if
> we should update them to be accurate, or just remove them altogether (I
> would be in favor of the latter).

https://github.com/matthew-brett/gitwash

St?fan


From asmeurer at gmail.com  Thu Sep 19 12:39:09 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Thu, 19 Sep 2013 10:39:09 -0600
Subject: [IPython-dev] pull requests
In-Reply-To: <CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
	<CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
Message-ID: <-2801247592287401791@unknownmsgid>

One thing I don't like about this guide is that it recommends that
people clone from their fork and add the official upstream as a
remote. But in my experience, it's much less confusing to git newbies
to clone from the original upstream and add their fork as a remote.

Aaron Meurer

> On Sep 19, 2013, at 10:19 AM, "St?fan van der Walt" <stefan at sun.ac.za> wrote:
>
>> On Thu, Sep 19, 2013 at 6:03 PM, MinRK <benjaminrk at gmail.com> wrote:
>> Yes, open a PR against master.  I don't know the provenance of those gitwash
>> docs, but they say several things that are not true. We should figure out if
>> we should update them to be accurate, or just remove them altogether (I
>> would be in favor of the latter).
>
> https://github.com/matthew-brett/gitwash
>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Thu Sep 19 12:43:59 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 19 Sep 2013 09:43:59 -0700
Subject: [IPython-dev] pull requests
In-Reply-To: <-2801247592287401791@unknownmsgid>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
	<CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
	<-2801247592287401791@unknownmsgid>
Message-ID: <CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>

I don't agree with that.  It doesn't make sense for a user to have an
origin that they can't push to.

On Thu, Sep 19, 2013 at 9:39 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
> One thing I don't like about this guide is that it recommends that
> people clone from their fork and add the official upstream as a
> remote. But in my experience, it's much less confusing to git newbies
> to clone from the original upstream and add their fork as a remote.
>
> Aaron Meurer
>
>> On Sep 19, 2013, at 10:19 AM, "St?fan van der Walt" <stefan at sun.ac.za> wrote:
>>
>>> On Thu, Sep 19, 2013 at 6:03 PM, MinRK <benjaminrk at gmail.com> wrote:
>>> Yes, open a PR against master.  I don't know the provenance of those gitwash
>>> docs, but they say several things that are not true. We should figure out if
>>> we should update them to be accurate, or just remove them altogether (I
>>> would be in favor of the latter).
>>
>> https://github.com/matthew-brett/gitwash
>>
>> St?fan
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From takowl at gmail.com  Thu Sep 19 12:46:50 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 19 Sep 2013 09:46:50 -0700
Subject: [IPython-dev] nbconvert bug?
In-Reply-To: <DUB112-W982C861B10F3236D32F675A4210@phx.gbl>
References: <DUB112-W982C861B10F3236D32F675A4210@phx.gbl>
Message-ID: <CAOvn4qhZifmCMJ7L3eZKa5OOvBxY4+PgCUkvT1PsqetKRN00Wg@mail.gmail.com>

On 19 September 2013 09:01, Johan Beke <johanbeke at hotmail.com> wrote:

>
> In [1]: version
> Out[1]: '1.0.0'
>

We don't define a 'version' variable in the interactive namespace, so that
must be coming from something in your setup.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/53e52509/attachment.html>

From benjaminrk at gmail.com  Thu Sep 19 12:47:09 2013
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 19 Sep 2013 09:47:09 -0700
Subject: [IPython-dev] pull requests
In-Reply-To: <CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
	<CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
	<-2801247592287401791@unknownmsgid>
	<CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>
Message-ID: <CAHNn8BUze44u7_ZDWZibVs67b9q4XDwUWv2rH7yA_o6KVHq7dQ@mail.gmail.com>

I just don't think a generic git tutorial is appropriate for the IPython
docs.


On Thu, Sep 19, 2013 at 9:43 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> I don't agree with that.  It doesn't make sense for a user to have an
> origin that they can't push to.
>
> On Thu, Sep 19, 2013 at 9:39 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
> > One thing I don't like about this guide is that it recommends that
> > people clone from their fork and add the official upstream as a
> > remote. But in my experience, it's much less confusing to git newbies
> > to clone from the original upstream and add their fork as a remote.
> >
> > Aaron Meurer
> >
> >> On Sep 19, 2013, at 10:19 AM, "St?fan van der Walt" <stefan at sun.ac.za>
> wrote:
> >>
> >>> On Thu, Sep 19, 2013 at 6:03 PM, MinRK <benjaminrk at gmail.com> wrote:
> >>> Yes, open a PR against master.  I don't know the provenance of those
> gitwash
> >>> docs, but they say several things that are not true. We should figure
> out if
> >>> we should update them to be accurate, or just remove them altogether (I
> >>> would be in favor of the latter).
> >>
> >> https://github.com/matthew-brett/gitwash
> >>
> >> St?fan
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/d77ca3c3/attachment.html>

From asmeurer at gmail.com  Thu Sep 19 12:48:21 2013
From: asmeurer at gmail.com (Aaron Meurer)
Date: Thu, 19 Sep 2013 10:48:21 -0600
Subject: [IPython-dev] pull requests
In-Reply-To: <CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
	<CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
	<-2801247592287401791@unknownmsgid>
	<CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>
Message-ID: <-2863356921076779956@unknownmsgid>

You either have to pull from origin and push to a remote or push to
origin and pull from a remote (technically you can set it up so that
you push and pull to different urls, but that's way too complicated
for beginners).

One argument for cloning from the official repo is that you may not
have a fork. You just want to get the source code. You may fork it
later if you want to contribute a patch. To do it your way, in order
to be consistent with all your git repos (which is very important),
you have to fork every repo you want to clone. Furthermore, what if
you clone a git repo that's not on github? To remain consistent, you
have to somehow "fork" it to github before you clone it!

Aaron Meurer

> On Sep 19, 2013, at 10:44 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> I don't agree with that.  It doesn't make sense for a user to have an
> origin that they can't push to.
>
>> On Thu, Sep 19, 2013 at 9:39 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
>> One thing I don't like about this guide is that it recommends that
>> people clone from their fork and add the official upstream as a
>> remote. But in my experience, it's much less confusing to git newbies
>> to clone from the original upstream and add their fork as a remote.
>>
>> Aaron Meurer
>>
>>>> On Sep 19, 2013, at 10:19 AM, "St?fan van der Walt" <stefan at sun.ac.za> wrote:
>>>>
>>>> On Thu, Sep 19, 2013 at 6:03 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>> Yes, open a PR against master.  I don't know the provenance of those gitwash
>>>> docs, but they say several things that are not true. We should figure out if
>>>> we should update them to be accurate, or just remove them altogether (I
>>>> would be in favor of the latter).
>>>
>>> https://github.com/matthew-brett/gitwash
>>>
>>> St?fan
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Thu Sep 19 12:50:14 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 19 Sep 2013 09:50:14 -0700
Subject: [IPython-dev] pull requests
In-Reply-To: <CAHNn8BUze44u7_ZDWZibVs67b9q4XDwUWv2rH7yA_o6KVHq7dQ@mail.gmail.com>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
	<CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
	<-2801247592287401791@unknownmsgid>
	<CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>
	<CAHNn8BUze44u7_ZDWZibVs67b9q4XDwUWv2rH7yA_o6KVHq7dQ@mail.gmail.com>
Message-ID: <CAH4pYpSo55TJ==Uo5T69ohhSBZXDQcdiuy31gFXP7aRHBSkbNg@mail.gmail.com>

These days, probably not.  I think this was written a while back when
people needed more help getting going with git/github.

On Thu, Sep 19, 2013 at 9:47 AM, MinRK <benjaminrk at gmail.com> wrote:
> I just don't think a generic git tutorial is appropriate for the IPython
> docs.
>
>
> On Thu, Sep 19, 2013 at 9:43 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> I don't agree with that.  It doesn't make sense for a user to have an
>> origin that they can't push to.
>>
>> On Thu, Sep 19, 2013 at 9:39 AM, Aaron Meurer <asmeurer at gmail.com> wrote:
>> > One thing I don't like about this guide is that it recommends that
>> > people clone from their fork and add the official upstream as a
>> > remote. But in my experience, it's much less confusing to git newbies
>> > to clone from the original upstream and add their fork as a remote.
>> >
>> > Aaron Meurer
>> >
>> >> On Sep 19, 2013, at 10:19 AM, "St?fan van der Walt" <stefan at sun.ac.za>
>> >> wrote:
>> >>
>> >>> On Thu, Sep 19, 2013 at 6:03 PM, MinRK <benjaminrk at gmail.com> wrote:
>> >>> Yes, open a PR against master.  I don't know the provenance of those
>> >>> gitwash
>> >>> docs, but they say several things that are not true. We should figure
>> >>> out if
>> >>> we should update them to be accurate, or just remove them altogether
>> >>> (I
>> >>> would be in favor of the latter).
>> >>
>> >> https://github.com/matthew-brett/gitwash
>> >>
>> >> St?fan
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From wking at tremily.us  Thu Sep 19 12:52:25 2013
From: wking at tremily.us (W. Trevor King)
Date: Thu, 19 Sep 2013 09:52:25 -0700
Subject: [IPython-dev] pull requests
In-Reply-To: <-2863356921076779956@unknownmsgid>
References: <CA+-1RQRbK9DscKeXo0E3QRTYh4WCEdrbdHD0OzJuRy3-ZBO38Q@mail.gmail.com>
	<C0BB47C8-C6A7-4DE6-8106-ABBB3673F252@gmail.com>
	<CA+-1RQS3e4+B11cBTHaHLLj=LmmaPEPHH+Q1vZWxzfptZ66hRg@mail.gmail.com>
	<94CEB734-41ED-4D89-8822-259E02E7DB36@gmail.com>
	<CA+-1RQS0Wa-tbudEGGrzp6nt1qyuLbk5zq2dmbiiEPNvMOt73A@mail.gmail.com>
	<CAHNn8BX_HMwYZS7jFawkxoUwzwVgA=CNphOhBBamqSkQ9qsACQ@mail.gmail.com>
	<CABDkGQ=Ho0ribwWHEtotE1a+riFr2gayfbC=eDssxLEWG9o1ng@mail.gmail.com>
	<-2801247592287401791@unknownmsgid>
	<CAH4pYpTHO4jNSHtKqsFRk+qmaJ3SBqoN_2NCA5Exjdn1JT9=WQ@mail.gmail.com>
	<-2863356921076779956@unknownmsgid>
Message-ID: <20130919165225.GC7110@odin.tremily.us>

On Thu, Sep 19, 2013 at 10:48:21AM -0600, Aaron Meurer wrote:
> You either have to pull from origin and push to a remote or push to
> origin and pull from a remote (technically you can set it up so that
> you push and pull to different urls, but that's way too complicated
> for beginners).

For what it's worth, Hub makes pull requests against the "origin"
repository [1], so if you use Hub you want origin to point to the
central project repo.

Cheers,
Trevor

[1]: http://hub.github.com/hub.1.html

   git pull-request [-f] [-m MESSAGE|-F FILE|-i ISSUE|ISSUE-URL]
     [-b BASE] [-h HEAD]

     Opens a pull request on GitHub for the project that the "origin"
     remote points to?

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/a7d808d1/attachment.sig>

From takowl at gmail.com  Thu Sep 19 15:31:09 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 19 Sep 2013 12:31:09 -0700
Subject: [IPython-dev] ipython qtconsole pager problem.
In-Reply-To: <1893281.EgVyLYPFUq@localhost.localdomain>
References: <1893281.EgVyLYPFUq@localhost.localdomain>
Message-ID: <CAOvn4qgTXUWnTHGwJfRHy4bCDcwLU=kKLaS+SB-=Hac5+i8XVA@mail.gmail.com>

Hi ?ric,

On 19 September 2013 01:53, ?ric Depagne <eric at depagne.org> wrote:

> I run the following code with ipython:
>
> %run -p script.py
>
>
>
> In a notebook, the profiling is diplayed correctly
>
> In a normal terminal, it is displayed correctly
>
> In a qtconsole, the display goes empty (I do loose the sliders too), and
> then, nothing happens. Using the menu bar, I can open a new tab with the
> same kernel, indicating (I guess) that it's just a display problem of the
> qtconsole.
>

It's displaying the pager, but with no content, for some reason. You can
hit q to return to the console. I've created an issue for this:
https://github.com/ipython/ipython/issues/4235

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/f7b38714/attachment.html>

From eric at depagne.org  Thu Sep 19 15:42:19 2013
From: eric at depagne.org (=?ISO-8859-1?Q?=C9ric?= Depagne)
Date: Thu, 19 Sep 2013 21:42:19 +0200
Subject: [IPython-dev] ipython qtconsole pager problem.
In-Reply-To: <CAOvn4qgTXUWnTHGwJfRHy4bCDcwLU=kKLaS+SB-=Hac5+i8XVA@mail.gmail.com>
References: <1893281.EgVyLYPFUq@localhost.localdomain>
	<CAOvn4qgTXUWnTHGwJfRHy4bCDcwLU=kKLaS+SB-=Hac5+i8XVA@mail.gmail.com>
Message-ID: <3662677.5Of1dTaE3b@localhost.localdomain>

On Thursday 19 September 2013 12:31:09 Thomas Kluyver wrote:
> Hi ?ric,
Hi Thomas, 

Many thanks for the answer and the solution.

?ric.
> It's displaying the pager, but with no content, for some reason. You can
> hit q to return to the console. I've created an issue for this:
> https://github.com/ipython/ipython/issues/4235
> 
> Thanks,
> Thomas
Un clavier azerty en vaut deux
----------------------------------------------------------
?ric Depagne                            eric at depagne.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/0bdf8416/attachment.html>

From sychan at lbl.gov  Thu Sep 19 20:14:55 2013
From: sychan at lbl.gov (Stephen Chan)
Date: Thu, 19 Sep 2013 17:14:55 -0700
Subject: [IPython-dev] How to get notebook static url prefix from within
	python?
Message-ID: <CA+n9YfoP8ZA7kSfsCOMiuSkCdJPkHXxG7BH+dCWQP+_3mf6R1g@mail.gmail.com>

   This seems like it should be easy, but I just don't know the right
path through the API and I haven't been having much look googling for
it.

   In the notebook.html template, Javascript includes are wrapped by
static_url().

   If I have code that generates HTML for display in an output cell,
and I want to include JS files from under the static JS directories.
It looks like it is in settings.static_url_prefix but don't seem to be
able to

   It doesn't show up in the dictionary that get_ipython().config returns.

   Thanks,
   Steve


From takowl at gmail.com  Thu Sep 19 23:14:56 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 19 Sep 2013 20:14:56 -0700
Subject: [IPython-dev] Need for separate rich HTML / poor HTML display
	outputs
Message-ID: <CAOvn4qi5cc9OFXu1R45u-8rkitKHs5AXSNe0cvJLPMKN++OD4g@mail.gmail.com>

As we were discussing today in the dev meeting, I think there is a need for
separate display formats for rich HTML and 'poor' HTML (i.e. what the Qt
console can handle).

I've just come across an example illustrating the need for that: pandas has
some ugly code that attempts to determine which IPython frontend it is
running in (which there's no way to do reliably), so that it can offer
different HTML representations to the notebook and other consumers. It
disables HTML tables altogether for the Qt console.

You can see the _repr_html_ code here:

https://github.com/pydata/pandas/blob/master/pandas/core/frame.py#L665

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130919/6179d686/attachment.html>

From bussonniermatthias at gmail.com  Fri Sep 20 07:44:44 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 20 Sep 2013 13:44:44 +0200
Subject: [IPython-dev] How to get notebook static url prefix from within
	python?
In-Reply-To: <CA+n9YfoP8ZA7kSfsCOMiuSkCdJPkHXxG7BH+dCWQP+_3mf6R1g@mail.gmail.com>
References: <CA+n9YfoP8ZA7kSfsCOMiuSkCdJPkHXxG7BH+dCWQP+_3mf6R1g@mail.gmail.com>
Message-ID: <FECD1C99-8B83-42C1-9C82-0BE0D39805F9@gmail.com>


Le 20 sept. 2013 ? 02:14, Stephen Chan a ?crit :

>   This seems like it should be easy, but I just don't know the right
> path through the API and I haven't been having much look googling for
> it.
> 
>   In the notebook.html template, Javascript includes are wrapped by
> static_url().
> 
>   If I have code that generates HTML for display in an output cell,
> and I want to include JS files from under the static JS directories.
> It looks like it is in settings.static_url_prefix but don't seem to be
> able to
> 
>   It doesn't show up in the dictionary that get_ipython().config returns.

$('body').data('baseProjectUrl')

?

I would suggest using require.js (http://requirejs.org/) 
That is smart enough to know it's own location so that you can load relative url. 
And also will do all the caching for you if it is needed? help with dependencies? etc

-- 
M

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130920/219a5703/attachment.html>

From franz.bergesund at gmail.com  Fri Sep 20 09:50:56 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Fri, 20 Sep 2013 15:50:56 +0200
Subject: [IPython-dev] [parallel] @lview.parallel() and .map
Message-ID: <CAOCdBKJ2c603KY+JXiL+v1sa7zSh0j_FpnvuaVY-kHW1PMkW4g@mail.gmail.com>

Dear all,

I've a question about running a function decorated with @lview.parallel().
[python 3.3, ipython 1.1.0]

My function look like

def do_operation(fname):
>     "read file and do sum operation"
>     col = np.loadtxt(fname, usecols=[1,])
>     return col.size, col.sum()


@lview.parallel(block=True)
> def func(fnames)
>      "do some operation over the file names"
>      oplist = []
>      for fn in fnames:
>           oplist.append(do_operation(fn))
>      return oplist


According to the
documentation<http://ipython.org/ipython-doc/dev/parallel/parallel_multiengine.html#remote-function-decorators>
I
can call the above function as
1. res = func(fnlist)
2. res = func.map(fnlist)
where fnlist is a list of file names (so strings)
fnlist = ['/abspath/file1', '/abspath/file2', '/abspath/file3',
'/abspath/file4']

Way 1. works and I get something like

> res = [ [15,45],
>            [14,42],
>            [12,44],
>            [17,46]]

that then I convert to numpy array

But from the example in the above link, it seems to me that can happen to
get some extra grouping (see the example "echo(range(5))")

Way 2 seems safer when it comes to execution order and result recollection,
BUT when I run it I get a bunch of these errors:

[5:apply]:
> ---------------------------------------------------------------------------
> IsADirectoryError                         Traceback (most recent call
> last)<string> in <module>()
> /home/.local/lib/python3.3/site-packages/ipython-1.1.0-py3.3.egg/IPython/parallel/client/remotefunction.py
> in <lambda>(f, *sequences)
>     239             if self._mapping:
>     240                 if sys.version_info[0] >= 3:
> --> 241                     f = lambda f, *sequences: list(map(f,
> *sequences))
>     242                 else:
>     243                     f = map
> <ipython-input-61-740416948047> in get_sums(flist)
> <ipython-input-1-23acb8d29a27> in do_sums(fname)
> [...]
> ----> 6     col = np.loadtxt(fname, usecols=[1,])
> [...]
> /usr/lib/python3/dist-packages/numpy/lib/npyio.py in loadtxt(fname, dtype,
> comments, delimiter, converters, skiprows, usecols, unpack, ndmin)
>     713                 fh = iter(bz2.BZ2File(fname))
>     714             else:
> --> 715                 fh = iter(open(fname, 'U'))
>     716         else:
>     717             fh = iter(fname)
> IsADirectoryError: [Errno 21] Is a directory: '/'


So it seems that "map" breaks up the single strings in the list of file
names.


I know that strings are iterables, but here "map" is breaking up first the
list, and then the strings. I don't know if this is the intended behaviour,
but for me this is a bug.

thanks for the support and THE ipython

Cheers,

Fra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130920/f2512702/attachment.html>

From zvoros at gmail.com  Fri Sep 20 10:07:22 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Fri, 20 Sep 2013 16:07:22 +0200
Subject: [IPython-dev] nbconvert doesn't parse &
Message-ID: <523C569A.8060408@gmail.com>

Hi all (most probably Matthias),

This is the first time I have had to use nbconvert, and I have noticed 
that the ampersand symbol used in the eqnarray environment to align 
equations is not replaced properly in the latex output. In the latex 
file I still have &amp;=&amp; (which can't be compiled by latex), while 
in the JSON file I have the correct form, &=&. Is this a known issue? If 
so, what is the magic fix?

Cheers,
Zolt?n


From zvoros at gmail.com  Fri Sep 20 11:38:27 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Fri, 20 Sep 2013 17:38:27 +0200
Subject: [IPython-dev] nbconvert: eqnarray environment missing in html output
Message-ID: <523C6BF3.7030402@gmail.com>

Hi all,

Sorry for the spamming, for, in some sense, this message still concerns 
the issue that I raised a couple of hours ago. I converted the notebook 
to html, but it seems to me that the eqnarray environment is complete 
stripped from the output. Why should this happen? Is this an issue in 
pandoc, or in nbconvert? In any case, why is the behaviour different, 
depending on the output format? Any ideas as to what could go wrong?

Side note: quite a few people were complaining here on the mailing list 
that the print option has been removed from the notebook, and the 
official line is that one could generate the pdf file by calling

 > ipython nbconvert --to latex somenotebook.ipynb

and then compiling it with latex. Apart from the above-mentioned issue, 
my experience is that the quality of the pdf is quite low, simply 
because latex doesn't handle png files very well. An alternative is to call

 > ipython nbconvert --to html somenotebook.ipynb

and load the output into the browser, and use the browser's 
print-to-file facilities to create the desired pdf file. While it is 
certainly true that figures can be cut into two at page boundaries, yet, 
the quality is much higher than with the latex approach.

Cheers,
Zolt?n


From sychan at lbl.gov  Fri Sep 20 15:42:47 2013
From: sychan at lbl.gov (Stephen Chan)
Date: Fri, 20 Sep 2013 12:42:47 -0700
Subject: [IPython-dev] How to get notebook static url prefix from within
	python?
In-Reply-To: <FECD1C99-8B83-42C1-9C82-0BE0D39805F9@gmail.com>
References: <CA+n9YfoP8ZA7kSfsCOMiuSkCdJPkHXxG7BH+dCWQP+_3mf6R1g@mail.gmail.com>
	<FECD1C99-8B83-42C1-9C82-0BE0D39805F9@gmail.com>
Message-ID: <CA+n9YfpzaGCTbYqObG=PjXA4LkQPVWDyHYqjAw7v-MWnhLuDAg@mail.gmail.com>

Matthias,
   Thanks for the suggestion. When I try that in a markdown cell and write
the output to the console log, it simply returns "/"

   So, if I were to merely use $('body').data('baseProjectUrl')+'static' is
that guaranteed to provide the same url prefix as static_url in the
templates?

   This also brings up the sticky business of having a script include
dependent on the output of another script - if I'm not already using
requireJS, I would have to implement a dumbed down imitation of it. It
would be better if I could simply get the static_url prefix from within the
python backend.

   Steve


On Fri, Sep 20, 2013 at 4:44 AM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

>
> Le 20 sept. 2013 ? 02:14, Stephen Chan a ?crit :
>
>   This seems like it should be easy, but I just don't know the right
> path through the API and I haven't been having much look googling for
> it.
>
>   In the notebook.html template, Javascript includes are wrapped by
> static_url().
>
>   If I have code that generates HTML for display in an output cell,
> and I want to include JS files from under the static JS directories.
> It looks like it is in settings.static_url_prefix but don't seem to be
> able to
>
>   It doesn't show up in the dictionary that get_ipython().config returns.
>
>
> $('body').data('baseProjectUrl')
>
> ?
>
> I would suggest using require.js (http://requirejs.org/)
> That is smart enough to know it's own location so that you can load
> relative url.
> And also will do all the caching for you if it is needed? help with
> dependencies? etc
>
> --
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130920/98cb2a8e/attachment.html>

From bussonniermatthias at gmail.com  Sun Sep 22 08:01:58 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 22 Sep 2013 14:01:58 +0200
Subject: [IPython-dev] How to get notebook static url prefix from within
	python?
In-Reply-To: <CA+n9YfpzaGCTbYqObG=PjXA4LkQPVWDyHYqjAw7v-MWnhLuDAg@mail.gmail.com>
References: <CA+n9YfoP8ZA7kSfsCOMiuSkCdJPkHXxG7BH+dCWQP+_3mf6R1g@mail.gmail.com>
	<FECD1C99-8B83-42C1-9C82-0BE0D39805F9@gmail.com>
	<CA+n9YfpzaGCTbYqObG=PjXA4LkQPVWDyHYqjAw7v-MWnhLuDAg@mail.gmail.com>
Message-ID: <A28D978A-503A-46CB-9CA4-0D03350B4319@gmail.com>


Le 20 sept. 2013 ? 21:42, Stephen Chan a ?crit :

> Matthias,
>    Thanks for the suggestion. When I try that in a markdown cell and write the output to the console log, it simply returns "/"
> 
>    So, if I were to merely use $('body').data('baseProjectUrl')+'static' is that guaranteed to provide the same url prefix as static_url in the templates?

Hum, I'm not sure it it is guaranteed in the long term, but it will definitively work for now.

>    This also brings up the sticky business of having a script include dependent on the output of another script - if I'm not already using requireJS, I would have to implement a dumbed down imitation of it. It would be better if I could simply get the static_url prefix from within the python backend.

Devil advocate, but js is stored in the document. Can you be certain that your code will always be executed with a non-changing static url ? Also using 'static_url' is already reimplementing part of requireJS...
-- 
M

> 
>    Steve
> 
> 
> On Fri, Sep 20, 2013 at 4:44 AM, Matthias BUSSONNIER <bussonniermatthias at gmail.com> wrote:
> 
> Le 20 sept. 2013 ? 02:14, Stephen Chan a ?crit :
> 
>>   This seems like it should be easy, but I just don't know the right
>> path through the API and I haven't been having much look googling for
>> it.
>> 
>>   In the notebook.html template, Javascript includes are wrapped by
>> static_url().
>> 
>>   If I have code that generates HTML for display in an output cell,
>> and I want to include JS files from under the static JS directories.
>> It looks like it is in settings.static_url_prefix but don't seem to be
>> able to
>> 
>>   It doesn't show up in the dictionary that get_ipython().config returns.
> 
> $('body').data('baseProjectUrl')
> 
> ?
> 
> I would suggest using require.js (http://requirejs.org/) 
> That is smart enough to know it's own location so that you can load relative url. 
> And also will do all the caching for you if it is needed? help with dependencies? etc
> 
> -- 
> M
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From bussonniermatthias at gmail.com  Sun Sep 22 08:12:54 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 22 Sep 2013 14:12:54 +0200
Subject: [IPython-dev] nbconvert doesn't parse &
In-Reply-To: <523C569A.8060408@gmail.com>
References: <523C569A.8060408@gmail.com>
Message-ID: <CD6F5D75-3D1D-4FE9-B125-056D92CDE828@gmail.com>

Hey Zolt?n. 

I opened an issue about it on github. 
https://github.com/ipython/ipython/issues/4251

I'm short on time to investigate theses days .

Cheers, 
--
Matthias

Le 20 sept. 2013 ? 16:07, Zolt?n V?r?s a ?crit :

> Hi all (most probably Matthias),
> 
> This is the first time I have had to use nbconvert, and I have noticed 
> that the ampersand symbol used in the eqnarray environment to align 
> equations is not replaced properly in the latex output. In the latex 
> file I still have &amp;=&amp; (which can't be compiled by latex), while 
> in the JSON file I have the correct form, &=&. Is this a known issue? If 
> so, what is the magic fix?
> 
> Cheers,
> Zolt?n
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From dave.hirschfeld at gmail.com  Sun Sep 22 11:53:20 2013
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Sun, 22 Sep 2013 15:53:20 +0000 (UTC)
Subject: [IPython-dev] IPython MPI Hangs on Windows
Message-ID: <loom.20130922T172602-854@post.gmane.org>

The first problem I encounter is that the ipcluster command doesn't seem to 
work:
```
C:\dev\code>ipcluster start -n 4 --engines=MPIEngineSetLauncher
2013-09-22 16:27:15.398 [IPClusterStart] Using existing profile dir: 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default'
2013-09-22 16:27:15.407 [IPClusterStart] Starting ipcluster with 
[daemon=False]
2013-09-22 16:27:15.411 [IPClusterStart] Creating pid file: 
C:\Users\dhirschfeld\.ipython\profile_default\pid\ipcluster.pid
2013-09-22 16:27:15.414 [IPClusterStart] Starting Controller with 
LocalControllerLauncher
2013-09-22 16:27:16.408 [IPClusterStart] Starting 4 Engines with 
MPIEngineSetLauncher
2013-09-22 16:27:17.213 [IPClusterStart] ERROR |
            Engines shutdown early, they probably failed to connect.

            Check the engine log files for output.

            If your controller and engines are not on the same machine, you 
probably
            have to instruct the controller to listen on an interface other 
than localhost.

            You can set this by adding "--ip='*'" to your 
ControllerLauncher.controller_args.

            Be sure to read our security docs before instructing your 
controller to listen on
            a public interface.

2013-09-22 16:27:17.214 [IPClusterStart] ERROR | IPython cluster: stopping
2013-09-22 16:27:20.214 [IPClusterStart] Removing pid file: 
C:\Users\dhirschfeld\.ipython\profile_default\pid\ipcluster.pid
```

The only information in the log was:
```
2013-09-22 16:29:09.286 [IPControllerApp] Hub listening on 
tcp://127.0.0.1:43273 for registration.
2013-09-22 16:29:09.292 [IPControllerApp] Hub using DB backend: 'DictDB'
2013-09-22 16:29:09.545 [IPControllerApp] hub::created hub
2013-09-22 16:29:09.545 [IPControllerApp] writing connection info to 
C:\Users\dhirschfeld\.ipython\profile_default\security\ipcontroller-
client.json
2013-09-22 16:29:09.549 [IPControllerApp] writing connection info to 
C:\Users\dhirschfeld\.ipython\profile_default\security\ipcontroller-
engine.json
2013-09-22 16:29:09.553 [IPControllerApp] task::using Python leastload Task 
scheduler
2013-09-22 16:29:09.553 [IPControllerApp] Heartmonitor started
2013-09-22 16:29:09.569 [IPControllerApp] Creating pid file: 
C:\Users\dhirschfeld\.ipython\profile_default\pid\ipcontroller.pid
2013-09-22 16:29:10.144 [IPControllerApp] client::client 
'\x00\x8a\x95\xf9u\\\xabEb\x99\xc8\x1c\xd9\xf5Z\xe9?' requested 
u'connection_request'
2013-09-22 16:29:10.144 [IPControllerApp] client::client 
['\x00\x8a\x95\xf9u\\\xabEb\x99\xc8\x1c\xd9\xf5Z\xe9?'] connected
```

So I then attempted to use ipcontroller & ipengine.

In the ipcontroller terminal I get:
```
C:\dev\code>ipcontroller
2013-09-22 16:32:07.198 [IPControllerApp] Using existing profile dir: 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default'
2013-09-22 16:32:07.223 [IPControllerApp] Hub listening on 
tcp://127.0.0.1:56375 for registration.
2013-09-22 16:32:07.229 [IPControllerApp] Hub using DB backend: 'DictDB'
2013-09-22 16:32:07.482 [IPControllerApp] hub::created hub
2013-09-22 16:32:07.483 [IPControllerApp] writing connection info to 
C:\Users\dhirschfeld\.ipython\profile_default\security\ipcontroller-
client.json
2013-09-22 16:32:07.487 [IPControllerApp] writing connection info to 
C:\Users\dhirschfeld\.ipython\profile_default\security\ipcontroller-
engine.json
2013-09-22 16:32:07.493 [IPControllerApp] task::using Python leastload Task 
scheduler
2013-09-22 16:32:07.493 [IPControllerApp] Heartmonitor started
2013-09-22 16:32:07.512 [IPControllerApp] Creating pid file: 
C:\Users\dhirschfeld\.ipython\profile_default\pid\ipcontroller.pid
2013-09-22 16:33:14.381 [IPControllerApp] client::client '62f3f434-4659-
4152-beca-b5d62d48b73e' requested u'registration_request'
2013-09-22 16:33:14.382 [IPControllerApp] client::client '76ab6008-31a9-
4482-ab14-003e479882db' requested u'registration_request'
2013-09-22 16:33:19.493 [IPControllerApp] registration::finished registering 
engine 0:62f3f434-4659-4152-beca-b5d62d48b73e
2013-09-22 16:33:19.496 [IPControllerApp] engine::Engine Connected: 0
2013-09-22 16:33:19.500 [IPControllerApp] registration::finished registering 
engine 1:76ab6008-31a9-4482-ab14-003e479882db
2013-09-22 16:33:19.502 [IPControllerApp] engine::Engine Connected: 1
```

...whilst in the ipengine terminal I get:
```
C:\dev\code>mpiexec -n 2 ipengine --mpi=mpi4py
2013-09-22 16:33:14.270 [IPEngineApp] Using existing profile dir: 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default'
2013-09-22 16:33:14.275 [IPEngineApp] Initializing MPI:
2013-09-22 16:33:14.275 [IPEngineApp] from mpi4py import MPI as mpi
mpi.size = mpi.COMM_WORLD.Get_size()
mpi.rank = mpi.COMM_WORLD.Get_rank()

2013-09-22 16:33:14.335 [IPEngineApp] Using existing profile dir: 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default'
2013-09-22 16:33:14.341 [IPEngineApp] Initializing MPI:
2013-09-22 16:33:14.341 [IPEngineApp] from mpi4py import MPI as mpi
mpi.size = mpi.COMM_WORLD.Get_size()
mpi.rank = mpi.COMM_WORLD.Get_rank()

2013-09-22 16:33:14.358 [IPEngineApp] Loading url_file 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default\\security\\ipcontroller-
engine.json'
2013-09-22 16:33:14.358 [IPEngineApp] Loading url_file 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default\\security\\ipcontroller-
engine.json'
2013-09-22 16:33:14.374 [IPEngineApp] Registering with controller at 
tcp://127.0.0.1:56375
2013-09-22 16:33:14.374 [IPEngineApp] Registering with controller at 
tcp://127.0.0.1:56375
2013-09-22 16:33:14.505 [IPEngineApp] Starting to monitor the heartbeat 
signal from the hub every 3010 ms.
2013-09-22 16:33:14.510 [IPEngineApp] Using existing profile dir: 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default'
2013-09-22 16:33:14.512 [IPEngineApp] Completed registration with id 1
2013-09-22 16:33:14.517 [IPEngineApp] Starting to monitor the heartbeat 
signal from the hub every 3010 ms.
2013-09-22 16:33:14.525 [IPEngineApp] Using existing profile dir: 
u'C:\\Users\\dhirschfeld\\.ipython\\profile_default'
2013-09-22 16:33:14.526 [IPEngineApp] Completed registration with id 0
```


So everything seems to have worked. However whenever I try to connect to it 
from a Client it hangs:
```
In [1]: from IPython.parallel import Client
   ...: rc = Client()
   ...: 

In [2]: rc.ids
Out[2]: [0, 1]

In [3]: rc[:].push({'a': 1})
Out[3]: <AsyncResult: _push>

In [4]: _3.result()  <-------------------- Hang!!!
``` 

In [3]: print sys_info()
{'codename': 'Work in Progress',
 'commit_hash': '62e35db',
 'commit_source': 'repository',
 'default_encoding': 'cp1252',
 'ipython_path': 'c:\\dev\\code\\ipython\\IPython',
 'ipython_version': '2.0.0-dev',
 'os_name': 'nt',
 'platform': 'Windows-7-6.1.7601-SP1',
 'sys_executable': 'C:\\dev\\bin\\Anaconda\\python.exe',
 'sys_platform': 'win32',
 'sys_version': '2.7.5 |Anaconda 1.7.0 (64-bit)| (default, Jul  1 2013, 
12:37:52) [MSC v.1500 64 bit (AMD64)]'}

I get this both with my self-compiled mpi4py linked to msmpi and also with 
Christoph Gohlke's version linked to OpenMPI 1.6.5.

mpi4py itself seems to be working fine and passes all the tests except 2 in 
Christoph's version and 4 in my version.

https://groups.google.com/forum/#!topic/mpi4py/arQc9fVhZAo

Let me know if there's anything I can do to help get to the bottom of the 
problem...

Thanks,
Dave






From benzolius at yahoo.com  Sun Sep 22 12:32:39 2013
From: benzolius at yahoo.com (Benedek Zoltan)
Date: Sun, 22 Sep 2013 09:32:39 -0700 (PDT)
Subject: [IPython-dev] installing IPython Notebook by buildout
Message-ID: <1379867559.22985.YahooMailNeo@web121602.mail.ne1.yahoo.com>

Hi,

I tried to install IPython Notebook by buildout and had the same issue like Hinnerk Haardt:

http://mail.scipy.org/pipermail/ipython-user/2012-December/011837.html


Fortunately I could solve it adding one single line:

os.environ['PYTHONPATH'] = ':'.join(sys.path)


into the buildout start script (bin/ipython), before the 'import IPython', according to the answer from:

http://stackoverflow.com/questions/16621896/ipython-notebook-in-zc-buildout-not-using-eggs-path


Thanks for the great work behind IPython Notebook.
Zoltan Benedek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130922/8ec56980/attachment.html>

From zvoros at gmail.com  Sun Sep 22 14:21:18 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Sun, 22 Sep 2013 20:21:18 +0200
Subject: [IPython-dev] nbconvert doesn't parse &
In-Reply-To: <CD6F5D75-3D1D-4FE9-B125-056D92CDE828@gmail.com>
References: <523C569A.8060408@gmail.com>
	<CD6F5D75-3D1D-4FE9-B125-056D92CDE828@gmail.com>
Message-ID: <523F351E.4090407@gmail.com>

Hi Matthias,

Thanks a lot! So then it is a bug, indeed. If you gave me a hint as to 
where the problem might be, I could try to look into it. I can't promise 
to be able come up with a solution, though:)

Cheers,
Zolt?n

On 22/09/13 14:12, Matthias BUSSONNIER wrote:
> Hey Zolt?n.
>
> I opened an issue about it on github.
> https://github.com/ipython/ipython/issues/4251
>
> I'm short on time to investigate theses days .
>
> Cheers,
> --
> Matthias
>
> Le 20 sept. 2013 ? 16:07, Zolt?n V?r?s a ?crit :
>
>> Hi all (most probably Matthias),
>>
>> This is the first time I have had to use nbconvert, and I have noticed
>> that the ampersand symbol used in the eqnarray environment to align
>> equations is not replaced properly in the latex output. In the latex
>> file I still have &amp;=&amp; (which can't be compiled by latex), while
>> in the JSON file I have the correct form, &=&. Is this a known issue? If
>> so, what is the magic fix?
>>
>> Cheers,
>> Zolt?n
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From gager at ilsb.tuwien.ac.at  Mon Sep 23 02:05:26 2013
From: gager at ilsb.tuwien.ac.at (Jakob Gager)
Date: Mon, 23 Sep 2013 08:05:26 +0200
Subject: [IPython-dev] nbconvert: eqnarray environment missing in html
 output
In-Reply-To: <523C6BF3.7030402@gmail.com>
References: <523C6BF3.7030402@gmail.com>
Message-ID: <523FDA26.60607@ilsb.tuwien.ac.at>

Hi Zolt?n,

pandoc strips raw latex (not placed inside $s) when converting to html, but keeps it when converting to 
latex. Vice versa raw html is stripped when converting to latex but retained when converting to html.
So you are actually facing a pandoc shortcoming (I wouldn't call it a bug since this behavior seems to be 
intended). Has been discussed e.g. https://github.com/ipython/ipython/issues/4234

To overcome the png issue you could use svgs instead, try
%config InlineBackend.figure_format = 'svg'

Jakob

On 09/20/2013 05:38 PM, Zolt?n V?r?s wrote:
> Hi all,
>
> Sorry for the spamming, for, in some sense, this message still concerns
> the issue that I raised a couple of hours ago. I converted the notebook
> to html, but it seems to me that the eqnarray environment is complete
> stripped from the output. Why should this happen? Is this an issue in
> pandoc, or in nbconvert? In any case, why is the behaviour different,
> depending on the output format? Any ideas as to what could go wrong?
>
> Side note: quite a few people were complaining here on the mailing list
> that the print option has been removed from the notebook, and the
> official line is that one could generate the pdf file by calling
>
>   > ipython nbconvert --to latex somenotebook.ipynb
>
> and then compiling it with latex. Apart from the above-mentioned issue,
> my experience is that the quality of the pdf is quite low, simply
> because latex doesn't handle png files very well. An alternative is to call
>
>   > ipython nbconvert --to html somenotebook.ipynb
>
> and load the output into the browser, and use the browser's
> print-to-file facilities to create the desired pdf file. While it is
> certainly true that figures can be cut into two at page boundaries, yet,
> the quality is much higher than with the latex approach.
>
> Cheers,
> Zolt?n
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



From zvoros at gmail.com  Mon Sep 23 03:12:47 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Mon, 23 Sep 2013 09:12:47 +0200
Subject: [IPython-dev] nbconvert: eqnarray environment missing in html
 output
In-Reply-To: <523FDA26.60607@ilsb.tuwien.ac.at>
References: <523C6BF3.7030402@gmail.com> <523FDA26.60607@ilsb.tuwien.ac.at>
Message-ID: <523FE9EF.40709@gmail.com>

Greetings Jakob,

Thanks for the comments! Also, the discussion you linked to indicates a 
workaround for this particular issue.
> To overcome the png issue you could use svgs instead, try
> %config InlineBackend.figure_format = 'svg'
This would solve the problem of the resolution, but some months ago, 
someone on this mailing list pointed out that PNGs are returned in more 
or less the same amount of time, regardless the complexity of the 
figure, while this is not true for SVGs, and he reported wildly 
differing times/sizes, depending on what he actually plotted. But this 
might be the only option for latex.

Cheers,
Zolt?n


On 23/09/13 08:05, Jakob Gager wrote:
> Hi Zolt?n,
>
> pandoc strips raw latex (not placed inside $s) when converting to html, but keeps it when converting to
> latex. Vice versa raw html is stripped when converting to latex but retained when converting to html.
> So you are actually facing a pandoc shortcoming (I wouldn't call it a bug since this behavior seems to be
> intended). Has been discussed e.g. https://github.com/ipython/ipython/issues/4234
>
> To overcome the png issue you could use svgs instead, try
> %config InlineBackend.figure_format = 'svg'
>
> Jakob
>
> On 09/20/2013 05:38 PM, Zolt?n V?r?s wrote:
>> Hi all,
>>
>> Sorry for the spamming, for, in some sense, this message still concerns
>> the issue that I raised a couple of hours ago. I converted the notebook
>> to html, but it seems to me that the eqnarray environment is complete
>> stripped from the output. Why should this happen? Is this an issue in
>> pandoc, or in nbconvert? In any case, why is the behaviour different,
>> depending on the output format? Any ideas as to what could go wrong?
>>
>> Side note: quite a few people were complaining here on the mailing list
>> that the print option has been removed from the notebook, and the
>> official line is that one could generate the pdf file by calling
>>
>>    > ipython nbconvert --to latex somenotebook.ipynb
>>
>> and then compiling it with latex. Apart from the above-mentioned issue,
>> my experience is that the quality of the pdf is quite low, simply
>> because latex doesn't handle png files very well. An alternative is to call
>>
>>    > ipython nbconvert --to html somenotebook.ipynb
>>
>> and load the output into the browser, and use the browser's
>> print-to-file facilities to create the desired pdf file. While it is
>> certainly true that figures can be cut into two at page boundaries, yet,
>> the quality is much higher than with the latex approach.
>>
>> Cheers,
>> Zolt?n
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From dave.hirschfeld at gmail.com  Mon Sep 23 11:28:21 2013
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Mon, 23 Sep 2013 15:28:21 +0000 (UTC)
Subject: [IPython-dev] IPython MPI Hangs on Windows
References: <loom.20130922T172602-854@post.gmane.org>
Message-ID: <loom.20130923T172529-437@post.gmane.org>

Dave Hirschfeld <dave.hirschfeld <at> gmail.com> writes:

> 
> I get this both with my self-compiled mpi4py linked to msmpi and also with 
> Christoph Gohlke's version linked to OpenMPI 1.6.5.
> 
> mpi4py itself seems to be working fine and passes all the tests except 2 
in 
> Christoph's version and 4 in my version.
> 
> https://groups.google.com/forum/#!topic/mpi4py/arQc9fVhZAo
> 
> Let me know if there's anything I can do to help get to the bottom of the 
> problem...
> 
> Thanks,
> Dave
> 

Hmm, my compiled mpi4py works fine on another WinPython x64 distribution. So 
it looks like this may just be an issue with my PC or my Anaconda 
distribution.


-Dave




From dave.hirschfeld at gmail.com  Mon Sep 23 11:44:33 2013
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Mon, 23 Sep 2013 15:44:33 +0000 (UTC)
Subject: [IPython-dev] IPython MPI Hangs on Windows
References: <loom.20130922T172602-854@post.gmane.org>
	<loom.20130923T172529-437@post.gmane.org>
Message-ID: <loom.20130923T174219-257@post.gmane.org>

Dave Hirschfeld <dave.hirschfeld <at> gmail.com> writes:

> 
> 
> Hmm, my compiled mpi4py works fine on another WinPython x64 distribution. So 
> it looks like this may just be an issue with my PC or my Anaconda 
> distribution.
> 
> -Dave
> 

It appears to have been some nasty interaction between Anaconda and my usual 
method of running setupegg.py develop. Installing the Anaconda version of 
IPython has resolved the problem for me.

Sorry for the noise...

-Dave



From tapitman11 at gmail.com  Mon Sep 23 21:40:11 2013
From: tapitman11 at gmail.com (Tim Pitman)
Date: Mon, 23 Sep 2013 18:40:11 -0700
Subject: [IPython-dev] Minimum virtualenv version for IPython development?
Message-ID: <CACnfkZtyvL_5bw_1TTFxfqn6embP+Wb5JS7RNVJ-Tz7iyTVohQ@mail.gmail.com>

Hey all,

When using Python 2.7.5 built from source, and installing the latest
virtualenv, the following command from the IPython github page works fine:

pip install -e ".[notebook]"

However, when using Python 2.7.3 and virtualenv 1.7.1.2 from the Ubuntu
12.04 repos, the pip installed by virtualenv (apparently 1.1) fails to
execute that command with the following message:

--editable=.[notebook] should be formatted with svn+URL, git+URL, hg+URL or
bzr+UR

Seems like that version of pip is missing some newer features? Actually I
haven't been able to find documention for the -e ".[notebook]" syntax on
the pip site. Apparently it installs all of the dependencies for the
"notebook" entry in setup.py?

Basically my questions are:

Is there a minimum version of virtualenv/pip required for developing
IPython?

Is anyone developing off a system python from Ubuntu 12.04?

Thanks,
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130923/cac32566/attachment.html>

From benjaminrk at gmail.com  Mon Sep 23 23:48:53 2013
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 23 Sep 2013 20:48:53 -0700
Subject: [IPython-dev] Minimum virtualenv version for IPython
	development?
In-Reply-To: <CACnfkZtyvL_5bw_1TTFxfqn6embP+Wb5JS7RNVJ-Tz7iyTVohQ@mail.gmail.com>
References: <CACnfkZtyvL_5bw_1TTFxfqn6embP+Wb5JS7RNVJ-Tz7iyTVohQ@mail.gmail.com>
Message-ID: <CAHNn8BWf0RzE6U2Kk37y=DzS149vtqkaz=huL5DExo7aY4JWnA@mail.gmail.com>

If anything, it would be a pip version - virtualenv is not involved in that
command, but each version of virtualenv does embed a version of pip, which
you can always upgrade after the fact. What is your pip version in the env?
 For old pip, you may need to make a more explicit call, such as:

    pip install -e git+file://$PWD#egg=ipython[notebook]

In which case `python setup.py develop` might be simpler (though it won't
bring in dependencies).


On Mon, Sep 23, 2013 at 6:40 PM, Tim Pitman <tapitman11 at gmail.com> wrote:

> Hey all,
>
> When using Python 2.7.5 built from source, and installing the latest
> virtualenv, the following command from the IPython github page works fine:
>
> pip install -e ".[notebook]"
>
> However, when using Python 2.7.3 and virtualenv 1.7.1.2 from the Ubuntu
> 12.04 repos, the pip installed by virtualenv (apparently 1.1) fails to
> execute that command with the following message:
>
> --editable=.[notebook] should be formatted with svn+URL, git+URL, hg+URL
> or bzr+UR
>
> Seems like that version of pip is missing some newer features? Actually I
> haven't been able to find documention for the -e ".[notebook]" syntax on
> the pip site. Apparently it installs all of the dependencies for the
> "notebook" entry in setup.py?
>
> Basically my questions are:
>
> Is there a minimum version of virtualenv/pip required for developing
> IPython?
>
> Is anyone developing off a system python from Ubuntu 12.04?
>
> Thanks,
> Tim
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130923/5bc891b8/attachment.html>

From tapitman11 at gmail.com  Tue Sep 24 02:35:39 2013
From: tapitman11 at gmail.com (Tim Pitman)
Date: Mon, 23 Sep 2013 23:35:39 -0700
Subject: [IPython-dev] Minimum virtualenv version for IPython
	development?
In-Reply-To: <CAHNn8BWf0RzE6U2Kk37y=DzS149vtqkaz=huL5DExo7aY4JWnA@mail.gmail.com>
References: <CACnfkZtyvL_5bw_1TTFxfqn6embP+Wb5JS7RNVJ-Tz7iyTVohQ@mail.gmail.com>
	<CAHNn8BWf0RzE6U2Kk37y=DzS149vtqkaz=huL5DExo7aY4JWnA@mail.gmail.com>
Message-ID: <CACnfkZtBQLqYHja8jN19BTaCxWY3sPAYXGDD-U2MBY0xn_MWig@mail.gmail.com>

pip version with my virtualenv is 1.1


On Mon, Sep 23, 2013 at 8:48 PM, MinRK <benjaminrk at gmail.com> wrote:

> If anything, it would be a pip version - virtualenv is not involved in
> that command, but each version of virtualenv does embed a version of pip,
> which you can always upgrade after the fact. What is your pip version in
> the env?  For old pip, you may need to make a more explicit call, such as:
>
>     pip install -e git+file://$PWD#egg=ipython[notebook]
>
> In which case `python setup.py develop` might be simpler (though it won't
> bring in dependencies).
>
>
> On Mon, Sep 23, 2013 at 6:40 PM, Tim Pitman <tapitman11 at gmail.com> wrote:
>
>> Hey all,
>>
>> When using Python 2.7.5 built from source, and installing the latest
>> virtualenv, the following command from the IPython github page works fine:
>>
>> pip install -e ".[notebook]"
>>
>> However, when using Python 2.7.3 and virtualenv 1.7.1.2 from the Ubuntu
>> 12.04 repos, the pip installed by virtualenv (apparently 1.1) fails to
>> execute that command with the following message:
>>
>> --editable=.[notebook] should be formatted with svn+URL, git+URL, hg+URL
>> or bzr+UR
>>
>> Seems like that version of pip is missing some newer features? Actually I
>> haven't been able to find documention for the -e ".[notebook]" syntax on
>> the pip site. Apparently it installs all of the dependencies for the
>> "notebook" entry in setup.py?
>>
>> Basically my questions are:
>>
>> Is there a minimum version of virtualenv/pip required for developing
>> IPython?
>>
>> Is anyone developing off a system python from Ubuntu 12.04?
>>
>> Thanks,
>> Tim
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130923/8b111a10/attachment.html>

From aron at ahmadia.net  Tue Sep 24 11:07:17 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Tue, 24 Sep 2013 11:07:17 -0400
Subject: [IPython-dev] [Discuss] [teaching] how do you display updates
	to code in lessons?
In-Reply-To: <5241AA64.6050300@software-carpentry.org>
References: <5241AA64.6050300@software-carpentry.org>
Message-ID: <CAPhiW4g3YCFk4c6H0GksPkhd9Fxnft1UJGJdz0rXDi_91vC-mQ@mail.gmail.com>

Perhaps keep the original function in rendered Markdown, then enter/update
code cells?

You should definitely get the students used to the idea of a non-linear
notebook *early* in the lesson.  They'll be hopeless confused as soon as
they start trying to jump around otherwise.

A


On Tue, Sep 24, 2013 at 11:06 AM, Greg Wilson <
gvwilson at software-carpentry.org> wrote:

> Hi,
> When I'm teaching programming, I often want to show learners a simple or
> buggy version of a function, discuss its shortcomings, and then update it
> (often several times in succession).  Doing this is straightforward when I
> teach using Emacs (or some lesser text editor): I just keep the text file
> containing the code open and edit the function(s) in question.  The
> drawback, of course, is that learners complain about the old version
> disappearing, so they have no record of how we got to the final program.
> When I teach with the IPython Notebook, on the other hand, I just add a
> new cell and re-define the function in question.  This leaves footprints,
> but learners then complain about being confused by having several versions
> of the function in the notebook.  It becomes particularly problematic when
> I start doing things like executing cells containing test functions, which
> may come *before* the cell redefining the function, so that:
>
> [22] original definition of some_function()
> [23] tests for some_function()
> [24] discussion
> [25] redefinition of some_function()
>
> becomes:
>
> [22] original definition of some_function()
> [26] tests for some_function() # I just re-executed this!
> [24] discussion
> [25] redefinition of some_function()
>
> How do you manage this when you're teaching?  Do you:
>
> a) never re-define a function
> b) do it and not worry about it
> c) something else entirely (if so, what?)
>
> Thanks,
> Greg
>
> ______________________________**_________________
> Discuss mailing list
> Discuss at lists.software-**carpentry.org<Discuss at lists.software-carpentry.org>
> http://lists.software-**carpentry.org/mailman/**listinfo/discuss_lists.**
> software-carpentry.org<http://lists.software-carpentry.org/mailman/listinfo/discuss_lists.software-carpentry.org>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130924/e2378bf3/attachment.html>

From ellisonbg at gmail.com  Tue Sep 24 12:13:05 2013
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 24 Sep 2013 09:13:05 -0700
Subject: [IPython-dev] [Discuss] [teaching] how do you display updates
 to code in lessons?
In-Reply-To: <CAPhiW4g3YCFk4c6H0GksPkhd9Fxnft1UJGJdz0rXDi_91vC-mQ@mail.gmail.com>
References: <5241AA64.6050300@software-carpentry.org>
	<CAPhiW4g3YCFk4c6H0GksPkhd9Fxnft1UJGJdz0rXDi_91vC-mQ@mail.gmail.com>
Message-ID: <CAH4pYpQqRcm7gLQWECKiACu9bw6aKWKTMtESWiYFfedtk54Ryg@mail.gmail.com>

Hi,

I would probably:

* Define the function twice in the notebook, once wrong and once right.

Or:

* Load one or both versions from a separate file using the %load magic.

On Tue, Sep 24, 2013 at 8:07 AM, Aron Ahmadia <aron at ahmadia.net> wrote:
> Perhaps keep the original function in rendered Markdown, then enter/update
> code cells?
>
> You should definitely get the students used to the idea of a non-linear
> notebook *early* in the lesson.  They'll be hopeless confused as soon as
> they start trying to jump around otherwise.
>
> A
>
>
> On Tue, Sep 24, 2013 at 11:06 AM, Greg Wilson
> <gvwilson at software-carpentry.org> wrote:
>>
>> Hi,
>> When I'm teaching programming, I often want to show learners a simple or
>> buggy version of a function, discuss its shortcomings, and then update it
>> (often several times in succession).  Doing this is straightforward when I
>> teach using Emacs (or some lesser text editor): I just keep the text file
>> containing the code open and edit the function(s) in question.  The
>> drawback, of course, is that learners complain about the old version
>> disappearing, so they have no record of how we got to the final program.
>> When I teach with the IPython Notebook, on the other hand, I just add a
>> new cell and re-define the function in question.  This leaves footprints,
>> but learners then complain about being confused by having several versions
>> of the function in the notebook.  It becomes particularly problematic when I
>> start doing things like executing cells containing test functions, which may
>> come *before* the cell redefining the function, so that:
>>
>> [22] original definition of some_function()
>> [23] tests for some_function()
>> [24] discussion
>> [25] redefinition of some_function()
>>
>> becomes:
>>
>> [22] original definition of some_function()
>> [26] tests for some_function() # I just re-executed this!
>> [24] discussion
>> [25] redefinition of some_function()
>>
>> How do you manage this when you're teaching?  Do you:
>>
>> a) never re-define a function
>> b) do it and not worry about it
>> c) something else entirely (if so, what?)
>>
>> Thanks,
>> Greg
>>
>> _______________________________________________
>> Discuss mailing list
>> Discuss at lists.software-carpentry.org
>>
>> http://lists.software-carpentry.org/mailman/listinfo/discuss_lists.software-carpentry.org
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From stanton at haas.berkeley.edu  Tue Sep 24 12:57:51 2013
From: stanton at haas.berkeley.edu (Richard Stanton)
Date: Tue, 24 Sep 2013 16:57:51 +0000
Subject: [IPython-dev] Cell -> All Output -> Clear doesn't do anything
In-Reply-To: <CDB06E1B.2F08C%stanton@haas.berkeley.edu>
Message-ID: <CE6711C9.1D662%stanton@haas.berkeley.edu>

I just upgraded Ipython to the latest master, and now when I open an
Ipython notebook in a browser window, the output can no longer be removed
using Cell -> All Output -> Clear. Moreover, it looks like when I execute
a cell that plots a graph, the new graphs just get appended to the old
output cell, so the output cell gets longer and longer as I execute the
cell repeatedly.

Thanks.

Richard Stanton




From takowl at gmail.com  Tue Sep 24 13:14:17 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 24 Sep 2013 10:14:17 -0700
Subject: [IPython-dev] Cell -> All Output -> Clear doesn't do anything
In-Reply-To: <CE6711C9.1D662%stanton@haas.berkeley.edu>
References: <CDB06E1B.2F08C%stanton@haas.berkeley.edu>
	<CE6711C9.1D662%stanton@haas.berkeley.edu>
Message-ID: <CAOvn4qir8Hd7CVPU-vRhieoVUkRkHY9FT=VGKPBcikN=DS1qQA@mail.gmail.com>

On 24 September 2013 09:57, Richard Stanton <stanton at haas.berkeley.edu>wrote:

> I just upgraded Ipython to the latest master, and now when I open an
> Ipython notebook in a browser window, the output can no longer be removed
> using Cell -> All Output -> Clear. Moreover, it looks like when I execute
> a cell that plots a graph, the new graphs just get appended to the old
> output cell, so the output cell gets longer and longer as I execute the
> cell repeatedly.
>

Could you have stale Javascript in your browser's cache? Try Ctrl-F5 to
reload it.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130924/43df1273/attachment.html>

From zvoros at gmail.com  Wed Sep 25 11:44:55 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Wed, 25 Sep 2013 17:44:55 +0200
Subject: [IPython-dev] ipython bash interaction?
Message-ID: <524304F7.7060704@gmail.com>

Hi all,

I have a trivial question: how does the ipython console interact with 
bash (or the operating system, for that matter)? I can use system 
commands with e.g.,

In [] : !date
Wed Sep 25 17:42:33 CEST 2013

but this fails:

In [] : !myscript
/bin/sh: 1: myscript: not found

where myscript is a bash script in a directory that is in the path. From 
the system console, I can run myscript without any problems, no matter 
where I am. Is it not the case that everything after an ! is passed to 
the underlying system? Or is there something that has to be configured?

Cheers,
Zolt?n



From dave.hirschfeld at gmail.com  Wed Sep 25 11:47:16 2013
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Wed, 25 Sep 2013 15:47:16 +0000 (UTC)
Subject: [IPython-dev] MPI on Windows HPC
Message-ID: <loom.20130925T173724-864@post.gmane.org>

I've got MPI working on my local machine but it seems to me that if I don't 
start all engines together they don't recognise themselves as being part of 
the same group - i.e.

I start 4 engines in a terminal with `mpiexec -n 4 ipengine.bat --
mpi=mpi4py` then, after they've started I start a further two engines in  
another terminal with `mpiexec -n 2 ipengine.bat --mpi=mpi4py`

I then observe that the Client instance recognises all 6 engines, but MPI 
sees them as belonging to two distinct groups of sizes 4 & 2:

In [21]: rc.ids
Out[21]: [0, 1, 2, 3, 4, 5]

In [22]: view = rc[:]

In [23]: @view.remote(block=True)
    ...: def hello():
    ...:     from mpi4py import MPI
    ...:     comm = MPI.COMM_WORLD
    ...:     return "Process {comm.rank} of {comm.size}.".format(comm=comm)
    ...: 

In [26]: hello()
Out[26]: 
['Process 1 of 4.',
 'Process 3 of 4.',
 'Process 0 of 4.',
 'Process 2 of 4.',
 'Process 1 of 2.',
 'Process 0 of 2.']


My understanding is that I really want these to be recognised as part of the 
same group - is this possible?

My concern is that the Windows HPC scheduler can kill jobs at any time and 
restart them later. That seems to be fine for the Client which will 
recognise the newly started engines but it seems that for MPI this will lead 
me to eventually having each engine in an MPI group all of its own which 
won't be very useful AFAICT.

Unfortunately I have nearly zero MPI experience - am I missing something 
here?

Thanks,
Dave






From aron at ahmadia.net  Wed Sep 25 11:51:46 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Wed, 25 Sep 2013 11:51:46 -0400
Subject: [IPython-dev] MPI on Windows HPC
In-Reply-To: <loom.20130925T173724-864@post.gmane.org>
References: <loom.20130925T173724-864@post.gmane.org>
Message-ID: <CAPhiW4gTBpG6yfqPhaJ-CqY8Qv2FboExJW+ah9M9VJyOLns=oQ@mail.gmail.com>

> My understanding is that I really want these to be recognised as part of
> the
> same group - is this possible?
>

This is an MPI question, not an IPython question.

That said, the answer depends on your MPI implementation.  MPI_COMM_WORLD
(which is the communicator you're using here), is populated at process
launch.  There are ways to form groups of processes after process launch,
see intracommunicators in the MPI-2 documentation, but you should
understand that this is not a very commonly used technique in the HPC
world.

Cheers,
Aron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/8a245f32/attachment.html>

From aron at ahmadia.net  Wed Sep 25 11:51:46 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Wed, 25 Sep 2013 11:51:46 -0400
Subject: [IPython-dev] MPI on Windows HPC
In-Reply-To: <loom.20130925T173724-864@post.gmane.org>
References: <loom.20130925T173724-864@post.gmane.org>
Message-ID: <CAPhiW4gTBpG6yfqPhaJ-CqY8Qv2FboExJW+ah9M9VJyOLns=oQ@mail.gmail.com>

> My understanding is that I really want these to be recognised as part of
> the
> same group - is this possible?
>

This is an MPI question, not an IPython question.

That said, the answer depends on your MPI implementation.  MPI_COMM_WORLD
(which is the communicator you're using here), is populated at process
launch.  There are ways to form groups of processes after process launch,
see intracommunicators in the MPI-2 documentation, but you should
understand that this is not a very commonly used technique in the HPC
world.

Cheers,
Aron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/8a245f32/attachment-0001.html>

From dave.hirschfeld at gmail.com  Wed Sep 25 11:58:12 2013
From: dave.hirschfeld at gmail.com (Dave Hirschfeld)
Date: Wed, 25 Sep 2013 15:58:12 +0000 (UTC)
Subject: [IPython-dev] MPI on Windows HPC
References: <loom.20130925T173724-864@post.gmane.org>
	<CAPhiW4gTBpG6yfqPhaJ-CqY8Qv2FboExJW+ah9M9VJyOLns=oQ@mail.gmail.com>
Message-ID: <loom.20130925T175501-541@post.gmane.org>

Aron Ahmadia <aron <at> ahmadia.net> writes:

> 
> 
> My understanding is that I really want these to be recognised as part of the
> same group - is this possible?
> 
> 
> This is an MPI question, not an IPython question.
> 
> That said, the answer depends on your MPI implementation. ?MPI_COMM_WORLD 
(which is the communicator you're using here), is populated at process launch. 
?There are ways to form groups of processes after process launch, see 
intracommunicators in the MPI-2 documentation, but you should understand that 
this is not a very commonly used technique in the HPC world.?
> ?
> Cheers,
> Aron
> 

Thanks for the quick reply! I wasn't sure if it was pure MPI or to do with the 
way ipython was running under MPI.

Since my processes can be killed and restarted I think I'll have to at least 
investigate this technique...

Thanks,
Dave




From takowl at gmail.com  Wed Sep 25 12:40:43 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 Sep 2013 09:40:43 -0700
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <524304F7.7060704@gmail.com>
References: <524304F7.7060704@gmail.com>
Message-ID: <CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>

Yes, commands are passed to a system shell. Check that $PATH is the same
there as it is in your normal shell.

Thomas


On 25 September 2013 08:44, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> Hi all,
>
> I have a trivial question: how does the ipython console interact with
> bash (or the operating system, for that matter)? I can use system
> commands with e.g.,
>
> In [] : !date
> Wed Sep 25 17:42:33 CEST 2013
>
> but this fails:
>
> In [] : !myscript
> /bin/sh: 1: myscript: not found
>
> where myscript is a bash script in a directory that is in the path. From
> the system console, I can run myscript without any problems, no matter
> where I am. Is it not the case that everything after an ! is passed to
> the underlying system? Or is there something that has to be configured?
>
> Cheers,
> Zolt?n
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/3d0517d1/attachment.html>

From franz.bergesund at gmail.com  Wed Sep 25 12:45:21 2013
From: franz.bergesund at gmail.com (Francesco Montesano)
Date: Wed, 25 Sep 2013 18:45:21 +0200
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <524304F7.7060704@gmail.com>
References: <524304F7.7060704@gmail.com>
Message-ID: <CAOCdBKJ4V-wb6ZekngxEF1mRgZ7708Vwm0nj=YYSvJgBoYHiOg@mail.gmail.com>

do you get an error if you run it with
!/bin/bash myscript
?

fra
Il giorno 25/set/2013 17:45, "Zolt?n V?r?s" <zvoros at gmail.com> ha scritto:

> Hi all,
>
> I have a trivial question: how does the ipython console interact with
> bash (or the operating system, for that matter)? I can use system
> commands with e.g.,
>
> In [] : !date
> Wed Sep 25 17:42:33 CEST 2013
>
> but this fails:
>
> In [] : !myscript
> /bin/sh: 1: myscript: not found
>
> where myscript is a bash script in a directory that is in the path. From
> the system console, I can run myscript without any problems, no matter
> where I am. Is it not the case that everything after an ! is passed to
> the underlying system? Or is there something that has to be configured?
>
> Cheers,
> Zolt?n
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/7db6ff0b/attachment.html>

From stanton at haas.berkeley.edu  Wed Sep 25 13:27:04 2013
From: stanton at haas.berkeley.edu (Richard Stanton)
Date: Wed, 25 Sep 2013 17:27:04 +0000
Subject: [IPython-dev] Cell -> All Output -> Clear doesn't do anything
In-Reply-To: <mailman.3.1380128401.21730.ipython-dev@scipy.org>
Message-ID: <CE686A9F.1D79C%stanton@haas.berkeley.edu>

>Date: Tue, 24 Sep 2013 10:14:17 -0700
>From: Thomas Kluyver <takowl at gmail.com>
>
>On 24 September 2013 09:57, Richard Stanton
><stanton at haas.berkeley.edu>wrote:
>
>> I just upgraded Ipython to the latest master, and now when I open an
>> Ipython notebook in a browser window, the output can no longer be
>>removed
>> using Cell -> All Output -> Clear. Moreover, it looks like when I
>>execute
>> a cell that plots a graph, the new graphs just get appended to the old
>> output cell, so the output cell gets longer and longer as I execute the
>> cell repeatedly.
>>
>
>Could you have stale Javascript in your browser's cache? Try Ctrl-F5 to
>reload it.
>
>Thomas

That seems to have been it. Thanks!



From zvoros at gmail.com  Wed Sep 25 14:39:23 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Wed, 25 Sep 2013 20:39:23 +0200
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
Message-ID: <52432DDB.3010604@gmail.com>


Hi Thomas, and Francesco,

Thanks for the feedback!

On 25/09/13 18:40, Thomas Kluyver wrote:
> Yes, commands are passed to a system shell. Check that $PATH is the 
> same there as it is in your normal shell.
No, it is not. You are right, the PATH that I see from the ipython 
console misses the directories that are set by .bashrc. Is there a way 
of making ipython aware of the bash variables? I thought that everything 
after ! is passed to the OS.

On 25/09/13 18:45, Francesco Montesano wrote:
> do you get an error if you run it with
> !/bin/bash myscript
> ?
If I supply the full path to myscript, then this works. But so does

!/full_path/myscript

Cheers,
Zolt?n

> Thomas
>
>
> On 25 September 2013 08:44, Zolt?n V?r?s <zvoros at gmail.com 
> <mailto:zvoros at gmail.com>> wrote:
>
>     Hi all,
>
>     I have a trivial question: how does the ipython console interact with
>     bash (or the operating system, for that matter)? I can use system
>     commands with e.g.,
>
>     In [] : !date
>     Wed Sep 25 17:42:33 CEST 2013
>
>     but this fails:
>
>     In [] : !myscript
>     /bin/sh: 1: myscript: not found
>
>     where myscript is a bash script in a directory that is in the
>     path. From
>     the system console, I can run myscript without any problems, no matter
>     where I am. Is it not the case that everything after an ! is passed to
>     the underlying system? Or is there something that has to be
>     configured?
>
>     Cheers,
>     Zolt?n
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org <mailto:IPython-dev at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/5755b8ad/attachment.html>

From takowl at gmail.com  Wed Sep 25 15:02:58 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 25 Sep 2013 12:02:58 -0700
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <52432DDB.3010604@gmail.com>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
	<52432DDB.3010604@gmail.com>
Message-ID: <CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>

On 25 September 2013 11:39, Zolt?n V?r?s <zvoros at gmail.com> wrote:

> No, it is not. You are right, the PATH that I see from the ipython console
> misses the directories that are set by .bashrc. Is there a way of making
> ipython aware of the bash variables? I thought that everything after ! is
> passed to the OS.


How are you starting IPython? If you start it from inside bash, it should
inherit the environment variables. If you don't, .bashrc won't be read.
However, PR #4260 might help with this if your $SHELL environment variable
is set to bash.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/808e712a/attachment.html>

From zvoros at gmail.com  Wed Sep 25 15:17:21 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Wed, 25 Sep 2013 21:17:21 +0200
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
	<52432DDB.3010604@gmail.com>
	<CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>
Message-ID: <524336C1.7060606@gmail.com>


On 25/09/13 21:02, Thomas Kluyver wrote:
> On 25 September 2013 11:39, Zolt?n V?r?s <zvoros at gmail.com 
> <mailto:zvoros at gmail.com>> wrote:
>
>     No, it is not. You are right, the PATH that I see from the ipython
>     console misses the directories that are set by .bashrc. Is there a
>     way of making ipython aware of the bash variables? I thought that
>     everything after ! is passed to the OS.
>
>
> How are you starting IPython? If you start it from inside bash, it 
> should inherit the environment variables. If you don't, .bashrc won't 
> be read.
Yeah, then that is the problem. I defined a menu item in the desktop 
manager, so I just click on a button on my task bar.
I can live with having to specify the full path, I was just a bit 
astonished that something didn't work as I expected.
Cheers,
Zolt?n

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130925/4f02fe9a/attachment.html>

From fperez.net at gmail.com  Wed Sep 25 21:11:59 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 Sep 2013 18:11:59 -0700
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <524336C1.7060606@gmail.com>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
	<52432DDB.3010604@gmail.com>
	<CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>
	<524336C1.7060606@gmail.com>
Message-ID: <CAHAreOp+_+xBP1gHdeWFjyLvCYtR19acE4B8Bb0J5GcnLS=P5w@mail.gmail.com>

On Wed, Sep 25, 2013 at 12:17 PM, Zolt?n V?r?s <zvoros at gmail.com> wrote:
> Yeah, then that is the problem. I defined a menu item in the desktop
> manager, so I just click on a button on my task bar.

You'll see the same problem for anything else you start via GUI
methods that needs things defined in your .bashrc.  That's why I
always start emacs and lyx from a terminal and not via the GUI, b/c I
have env. vars defined in my bashrc that I need.

I think it's possible to declare those in another one of the bash
startup files that does get read by the GUI systems, but I've never
bothered to look which one it should be. If you spend the time to make
it work that way, let us know!

Cheers,

f



-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From mark.voorhies at ucsf.edu  Wed Sep 25 21:40:12 2013
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Wed, 25 Sep 2013 18:40:12 -0700
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <CAHAreOp+_+xBP1gHdeWFjyLvCYtR19acE4B8Bb0J5GcnLS=P5w@mail.gmail.com>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
	<52432DDB.3010604@gmail.com>
	<CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>
	<524336C1.7060606@gmail.com>
	<CAHAreOp+_+xBP1gHdeWFjyLvCYtR19acE4B8Bb0J5GcnLS=P5w@mail.gmail.com>
Message-ID: <5243907C.5040808@ucsf.edu>

On 09/25/2013 06:11 PM, Fernando Perez wrote:
> On Wed, Sep 25, 2013 at 12:17 PM, Zolt?n V?r?s <zvoros at gmail.com> wrote:
>> Yeah, then that is the problem. I defined a menu item in the desktop
>> manager, so I just click on a button on my task bar.
>
> You'll see the same problem for anything else you start via GUI
> methods that needs things defined in your .bashrc.  That's why I
> always start emacs and lyx from a terminal and not via the GUI, b/c I
> have env. vars defined in my bashrc that I need.
>
> I think it's possible to declare those in another one of the bash
> startup files that does get read by the GUI systems, but I've never
> bothered to look which one it should be. If you spend the time to make
> it work that way, let us know!
>
> Cheers,
>
> f

A related thread for getting the proper PYTHONSTARTUP when starting IDLE from a graphical session:
http://lists.debian.org/debian-python/2009/11/msg00071.html
I think an equivalent IPython hack would be to modify os.environ in the ipython launcher script.

This Xsession documentation looks quite good and is probably a better starting point:
https://wiki.ubuntu.com/CustomXSession

--Mark



From zvoros at gmail.com  Thu Sep 26 03:18:13 2013
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Thu, 26 Sep 2013 09:18:13 +0200
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <5243907C.5040808@ucsf.edu>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
	<52432DDB.3010604@gmail.com>
	<CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>
	<524336C1.7060606@gmail.com>
	<CAHAreOp+_+xBP1gHdeWFjyLvCYtR19acE4B8Bb0J5GcnLS=P5w@mail.gmail.com>
	<5243907C.5040808@ucsf.edu>
Message-ID: <5243DFB5.9070400@gmail.com>

Hi Fernando,

Thanks for the pointer! Based on your comment, I could figure out what 
the solution is.
> On 09/25/2013 06:11 PM, Fernando Perez wrote:
>> I think it's possible to declare those in another one of the bash
>> startup files that does get read by the GUI systems, but I've never
>> bothered to look which one it should be. If you spend the time to make
>> it work that way, let us know!
Setting the path in .profile (for user), or in /etc/environment 
(system-wide) works for me. I suspect that .profile has to be replaced 
by something else, if one uses KDE or XCFE, but I believe, .profile 
should work for gnome-derivatives.
> A related thread for getting the proper PYTHONSTARTUP when starting IDLE from a graphical session:
> http://lists.debian.org/debian-python/2009/11/msg00071.html
> I think an equivalent IPython hack would be to modify os.environ in the ipython launcher script.
But it is quite obvious that this is not an ipython issue. As Fernando 
pointed out, the launcher doesn't know about the environmental 
variables, because it is not operating at the shell level.

Cheers,
Zolt?n


From alimanfoo at googlemail.com  Thu Sep 26 05:02:24 2013
From: alimanfoo at googlemail.com (Alistair Miles)
Date: Thu, 26 Sep 2013 10:02:24 +0100
Subject: [IPython-dev] ! with bash
Message-ID: <CAMr-JwY3Q-XJuxCwN9Oz-+Q9Wc-bcrkUuwOFxThp6-gVwRg-Ng@mail.gmail.com>

Hi, apologies if this has been answered elsewhere but couldn't find an
answer in the docs, is there any way to configure IPython such that
commands prefixed by "!" are run by /bin/bash instead of default system
shell (usually /bin/sh)? Thanks, Alistair.

-- 
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health <http://cggh.org>
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Web: http://purl.org/net/aliman
Email: alimanfoo at gmail.com
Tel: +44 (0)1865 287721 ***new number***
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/d2b1fa89/attachment.html>

From bussonniermatthias at gmail.com  Thu Sep 26 06:08:25 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 26 Sep 2013 12:08:25 +0200
Subject: [IPython-dev] ! with bash
In-Reply-To: <CAMr-JwY3Q-XJuxCwN9Oz-+Q9Wc-bcrkUuwOFxThp6-gVwRg-Ng@mail.gmail.com>
References: <CAMr-JwY3Q-XJuxCwN9Oz-+Q9Wc-bcrkUuwOFxThp6-gVwRg-Ng@mail.gmail.com>
Message-ID: <3D87E71B-1FD6-4A7E-B1AF-FBAC7DE8DAE5@gmail.com>


Le 26 sept. 2013 ? 11:02, Alistair Miles a ?crit :

> Hi, apologies if this has been answered elsewhere but couldn't find an answer in the docs, is there any way to configure IPython such that commands prefixed by "!" are run by /bin/bash instead of default system shell (usually /bin/sh)? Thanks, Alistair.
> 

`!` use os.system under the hood IIRC (through TerminalInteractiveShell.system_raw)
So I don't think there is any IPython specific way to change that. 
-- 
M




From mark.voorhies at ucsf.edu  Thu Sep 26 10:02:00 2013
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Thu, 26 Sep 2013 07:02:00 -0700
Subject: [IPython-dev] ipython bash interaction?
In-Reply-To: <5243DFB5.9070400@gmail.com>
References: <524304F7.7060704@gmail.com>
	<CAOvn4qj+MFUREX105WYtxDE9Kg6GXPuWsaVUYf3r0-Lp476g2g@mail.gmail.com>
	<52432DDB.3010604@gmail.com>
	<CAOvn4qjus-wXz_U2DNZbeT6sqv1T1Lkj6LcJizzR5A3KsYv3yg@mail.gmail.com>
	<524336C1.7060606@gmail.com>
	<CAHAreOp+_+xBP1gHdeWFjyLvCYtR19acE4B8Bb0J5GcnLS=P5w@mail.gmail.com>
	<5243907C.5040808@ucsf.edu> <5243DFB5.9070400@gmail.com>
Message-ID: <52443E58.9080301@ucsf.edu>

On 09/26/2013 12:18 AM, Zolt?n V?r?s wrote:
> Hi Fernando,
>
> Thanks for the pointer! Based on your comment, I could figure out what
> the solution is.
>> >On 09/25/2013 06:11 PM, Fernando Perez wrote:
>>> >>I think it's possible to declare those in another one of the bash
>>> >>startup files that does get read by the GUI systems, but I've never
>>> >>bothered to look which one it should be. If you spend the time to make
>>> >>it work that way, let us know!
> Setting the path in .profile (for user), or in /etc/environment
> (system-wide) works for me. I suspect that .profile has to be replaced
> by something else, if one uses KDE or XCFE,

Yes, I've run into problems with .profile on KDE.  I think the Xsession link I gave is more general, but I haven't tried it.

> but I believe, .profile
> should work for gnome-derivatives.
>> >A related thread for getting the proper PYTHONSTARTUP when starting IDLE from a graphical session:
>> >http://lists.debian.org/debian-python/2009/11/msg00071.html
>> >I think an equivalent IPython hack would be to modify os.environ in the ipython launcher script.
> But it is quite obvious that this is not an ipython issue. As Fernando
> pointed out, the launcher doesn't know about the environmental
> variables, because it is not operating at the shell level.

The launcher sees the same environment as the kernel, which is the important detail for hacking environment variables.
E.g.:

Given:

#!/usr/bin/python
"""Terminal-based IPython entry point.
"""
from os import environ
from IPython import start_ipython
environ["THIS_IS_A_TEST"] = "Hello, world"
environ["PATH"] += ":/My/Favorite/Path"
start_ipython()

The following works from the console or the notebook:

mvoorhie at virgil:~$ ipython console
Python 2.7.3 (default, Apr 10 2013, 06:20:15)
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: !echo "$THIS_IS_A_TEST"
Hello, world

In [2]: !echo "$PATH"
/home/mvoorhie/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/My/Favorite/Path

--Mark



From takowl at gmail.com  Thu Sep 26 12:32:11 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 26 Sep 2013 09:32:11 -0700
Subject: [IPython-dev] ! with bash
In-Reply-To: <CAMr-JwY3Q-XJuxCwN9Oz-+Q9Wc-bcrkUuwOFxThp6-gVwRg-Ng@mail.gmail.com>
References: <CAMr-JwY3Q-XJuxCwN9Oz-+Q9Wc-bcrkUuwOFxThp6-gVwRg-Ng@mail.gmail.com>
Message-ID: <CAOvn4qit8oOcyqyB-+aqnRSmk4=Lu-QF=sVrhrB405oTn65mtQ@mail.gmail.com>

On 26 September 2013 02:02, Alistair Miles <alimanfoo at googlemail.com> wrote:

> Hi, apologies if this has been answered elsewhere but couldn't find an
> answer in the docs, is there any way to configure IPython such that
> commands prefixed by "!" are run by /bin/bash instead of default system
> shell (usually /bin/sh)? Thanks, Alistair.
>

Is your $SHELL environment variable set? If so, PR #4260 should make this
work automatically:
https://github.com/ipython/ipython/pull/4260

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/089ee4d8/attachment.html>

From fperez.net at gmail.com  Thu Sep 26 21:45:49 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 26 Sep 2013 18:45:49 -0700
Subject: [IPython-dev] Microsoft donates $100,000 to IPython via NumFOCUS
Message-ID: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>

Hi all,

as reported here:

http://ipython.org/microsoft-donation-2013.html

today we're very happy to announce that Microsoft has made a donation of
$100,000 to support the continued development of IPython.  We're extremely
grateful for this contribution.

Making this possible was a combination of our ongoing collaboration with
Microsoft along with the efforts of NumFOCUS, whose 501c3 status makes it
possible to handle these financial resources for our community.

I'd like to thank everyone in the NumFOCUS board for all the
behind-the-scenes work they've put into making NumFOCUS a reality.  But in
particular I want to highlight how Leah and Anthony put a ton of effort
into the logistics necessary for this.

Best,

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/caaf4d8e/attachment.html>

From andresete.chaos at gmail.com  Thu Sep 26 21:55:18 2013
From: andresete.chaos at gmail.com (=?ISO-8859-1?Q?Omar_Andr=E9s_Zapata_Mesa?=)
Date: Thu, 26 Sep 2013 20:55:18 -0500
Subject: [IPython-dev] Microsoft donates $100,
	000 to IPython via NumFOCUS
In-Reply-To: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
References: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
Message-ID: <CAOB9eppYZdaWYbKmRC+BjOeWQkRk9G4j8=49UNDjP_rL+J8umw@mail.gmail.com>

Congratulations is very good news!
My best wishes to the ipython's team.


On Thu, Sep 26, 2013 at 8:45 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi all,
>
> as reported here:
>
> http://ipython.org/microsoft-donation-2013.html
>
> today we're very happy to announce that Microsoft has made a donation of
> $100,000 to support the continued development of IPython.  We're extremely
> grateful for this contribution.
>
> Making this possible was a combination of our ongoing collaboration with
> Microsoft along with the efforts of NumFOCUS, whose 501c3 status makes it
> possible to handle these financial resources for our community.
>
> I'd like to thank everyone in the NumFOCUS board for all the
> behind-the-scenes work they've put into making NumFOCUS a reality.  But in
> particular I want to highlight how Leah and Anthony put a ton of effort
> into the logistics necessary for this.
>
> Best,
>
> f
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Omar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/ac22d533/attachment.html>

From tapitman11 at gmail.com  Thu Sep 26 23:35:56 2013
From: tapitman11 at gmail.com (Tim Pitman)
Date: Thu, 26 Sep 2013 20:35:56 -0700
Subject: [IPython-dev] Microsoft donates $100,
	000 to IPython via NumFOCUS
In-Reply-To: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
References: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
Message-ID: <CACnfkZtbXvHyAoHBe7vXg-6jA5L8n1hvFRgj1Fmz+2ybJHsB-Q@mail.gmail.com>

Congrats!
On Sep 26, 2013 6:46 PM, "Fernando Perez" <fperez.net at gmail.com> wrote:

> Hi all,
>
> as reported here:
>
> http://ipython.org/microsoft-donation-2013.html
>
> today we're very happy to announce that Microsoft has made a donation of
> $100,000 to support the continued development of IPython.  We're extremely
> grateful for this contribution.
>
> Making this possible was a combination of our ongoing collaboration with
> Microsoft along with the efforts of NumFOCUS, whose 501c3 status makes it
> possible to handle these financial resources for our community.
>
> I'd like to thank everyone in the NumFOCUS board for all the
> behind-the-scenes work they've put into making NumFOCUS a reality.  But in
> particular I want to highlight how Leah and Anthony put a ton of effort
> into the logistics necessary for this.
>
> Best,
>
> f
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/81634699/attachment.html>

From scopatz at gmail.com  Fri Sep 27 00:29:30 2013
From: scopatz at gmail.com (Anthony Scopatz)
Date: Thu, 26 Sep 2013 23:29:30 -0500
Subject: [IPython-dev] [numfocus] Microsoft donates $100,
	000 to IPython via NumFOCUS
In-Reply-To: <CAMwpFYm9BWPdD1Rdkq3sm9U9_0ozcgewUxEOLuy5+cAtF4GOkg@mail.gmail.com>
References: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
	<CAMwpFYm9BWPdD1Rdkq3sm9U9_0ozcgewUxEOLuy5+cAtF4GOkg@mail.gmail.com>
Message-ID: <CAPk-6T4oc3PBQH6rQYMf37stHCr9K17Gmfb8Jm-Om2YFngAX8g@mail.gmail.com>

Thanks for the kind words, Fernando,

But frankly, my efforts are but a small thanks compared to the huge boons
that the IPython team has provided me with throughout the years.  This
award is well deserved!

Be Well
Anthony


On Thu, Sep 26, 2013 at 11:04 PM, James Bergstra
<james.bergstra at gmail.com>wrote:

> That's fantastic, congratulations!
>
>
> On Thu, Sep 26, 2013 at 9:45 PM, Fernando Perez <fperez.net at gmail.com>wrote:
>
>> Hi all,
>>
>> as reported here:
>>
>> http://ipython.org/microsoft-donation-2013.html
>>
>> today we're very happy to announce that Microsoft has made a donation of
>> $100,000 to support the continued development of IPython.  We're extremely
>> grateful for this contribution.
>>
>> Making this possible was a combination of our ongoing collaboration with
>> Microsoft along with the efforts of NumFOCUS, whose 501c3 status makes it
>> possible to handle these financial resources for our community.
>>
>> I'd like to thank everyone in the NumFOCUS board for all the
>> behind-the-scenes work they've put into making NumFOCUS a reality.  But in
>> particular I want to highlight how Leah and Anthony put a ton of effort
>> into the logistics necessary for this.
>>
>> Best,
>>
>> f
>>
>>
>> --
>> Fernando Perez (@fperez_org; http://fperez.org)
>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>> fernando.perez-at-berkeley: contact me here for any direct mail
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "NumFOCUS" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to numfocus+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "NumFOCUS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to numfocus+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/15186dc4/attachment.html>

From fperez.net at gmail.com  Fri Sep 27 02:34:59 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 26 Sep 2013 23:34:59 -0700
Subject: [IPython-dev] [numfocus] Microsoft donates $100,
	000 to IPython via NumFOCUS
In-Reply-To: <CAEHX+gm+k=w5RKaAwLLSYCNOxmEyiAdX7OOCx_axtaySP-T9_g@mail.gmail.com>
References: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
	<CAMwpFYm9BWPdD1Rdkq3sm9U9_0ozcgewUxEOLuy5+cAtF4GOkg@mail.gmail.com>
	<CAPk-6T4oc3PBQH6rQYMf37stHCr9K17Gmfb8Jm-Om2YFngAX8g@mail.gmail.com>
	<CAEHX+gm+k=w5RKaAwLLSYCNOxmEyiAdX7OOCx_axtaySP-T9_g@mail.gmail.com>
Message-ID: <CAHAreOo0x7wzZ386rAu2V_LsJC03Hr2HVjgPvRoZ84er0enVww@mail.gmail.com>

On Thu, Sep 26, 2013 at 9:48 PM, Nicolas Pettiaux <nicolas at pettiaux.be>wrote:

> Congratulation to all. This is a very very good news.
>
> I would like to know and understand which are the main reasons that have
> driven such a support by Microsoft, not a company that I would have
> qualified as a free software lover and suppoter ;-)
>

Microsoft is a very large company, and in recent years they've been
building relations with various parts of the open source community. As
indicated in our brief note:

http://ipython.org/microsoft-donation-2013.html

our collaboration with MS dates back several years and has been extremely
productive. Before it had consisted mostly of technical discussions that
led to their Python Tools for Visual Studio (http://pytools.codeplex.com/,
which is BTW fully open source) and some small consulting efforts.  But as
described here by Shahrokh:

http://www.reddit.com/r/Python/comments/1n72bm/microsoft_donates_100000_to_the_ipython_team/ccg1eez

the PTVS team's interactions with MS Research led to them being interested
in stronger support.


> Being recognized at the same time by the Free Software Foundation (ok,
> they rewarded the man, Fernando Perez who created and drives the free
> software project and code) and by Microsoft, is quite an achievement !
>

Thanks, but like everything in both IPython and the larger scientific
Python ecosystem, it's all about a long, sustained team effort.

Cheers,

f
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/dc1668a5/attachment.html>

From fperez.net at gmail.com  Fri Sep 27 02:37:03 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 26 Sep 2013 23:37:03 -0700
Subject: [IPython-dev] [numfocus] Microsoft donates $100,
	000 to IPython via NumFOCUS
In-Reply-To: <CAEHX+gmGXhk5eSk+iEZmcEdjqkCxjyq2qTXtKb0A0wte0DEZgQ@mail.gmail.com>
References: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
	<CAMwpFYm9BWPdD1Rdkq3sm9U9_0ozcgewUxEOLuy5+cAtF4GOkg@mail.gmail.com>
	<CAPk-6T4oc3PBQH6rQYMf37stHCr9K17Gmfb8Jm-Om2YFngAX8g@mail.gmail.com>
	<CAEHX+gm+k=w5RKaAwLLSYCNOxmEyiAdX7OOCx_axtaySP-T9_g@mail.gmail.com>
	<CAEHX+gmGXhk5eSk+iEZmcEdjqkCxjyq2qTXtKb0A0wte0DEZgQ@mail.gmail.com>
Message-ID: <CAHAreOonekAM=ZrCG6wgXNt+GAvnUQYbS8knWMpYHq=DMcK84Q@mail.gmail.com>

On Thu, Sep 26, 2013 at 11:02 PM, Nicolas Pettiaux <nicolas at pettiaux.be>wrote:

> I have another question : is the ipython team free or nearly free to chose
> the projects where the money will be spent or is it only/mainly related to
> the advancement of itpyhton on Microsoft platform (with VisualStudio, Azure
> ...) ?
>

This is a donation with no strings attached, not a grant for any specific
project. Under US non-profit regulations, donations like this can *not*
have strings attached.  At Numfocus we are learning how all this works, but
we are being very scrupulous in our following of these regulations to the
best of our understanding.

Cheers,

f

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130926/9ae1e6a3/attachment.html>

From Marco.Halder at frm2.tum.de  Fri Sep 27 07:22:44 2013
From: Marco.Halder at frm2.tum.de (Marco Halder)
Date: Fri, 27 Sep 2013 13:22:44 +0200
Subject: [IPython-dev] Get the running cell in javascript.
Message-ID: <20130927132244.Horde.Qtm_LvNpBAT_8ZkfnF9Avg1@hermes.frm2.tum.de>

Hello,

I'm experimenting with the javascript of the IPython notebook in  
combination with cell magics.
My Question is how do I get the running cell in javascript?

Using

     var cell = IPython.notebook.get_selected_cell();

gives me the next cell, not the one currently running. A stupid workaround is

     var cell =  
IPython.notebook.get_prev_cell(IPython.notebook.get_selected_cell());

But this is not very robust. If the user changes the selection while  
the cell is executing it breaks. I guess there is a better way. I just  
haven't found it yet.

I guess there is a better way. Any help would be apprecia


From bussonniermatthias at gmail.com  Fri Sep 27 07:39:58 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Fri, 27 Sep 2013 13:39:58 +0200
Subject: [IPython-dev] Get the running cell in javascript.
In-Reply-To: <20130927132244.Horde.Qtm_LvNpBAT_8ZkfnF9Avg1@hermes.frm2.tum.de>
References: <20130927132244.Horde.Qtm_LvNpBAT_8ZkfnF9Avg1@hermes.frm2.tum.de>
Message-ID: <8DE7F697-705F-4B15-9A38-4D9D22B8D588@gmail.com>


Le 27 sept. 2013 ? 13:22, Marco Halder a ?crit :

> Hello,
> 
> I'm experimenting with the javascript of the IPython notebook in  
> combination with cell magics.
> My Question is how do I get the running cell in javascript?
> 
> Using
> 
>     var cell = IPython.notebook.get_selected_cell();
> 
> gives me the next cell, not the one currently running. A stupid workaround is
> 
>     var cell =  
> IPython.notebook.get_prev_cell(IPython.notebook.get_selected_cell());


You cannot get the running cell as many cell can be submitted and you don't know when the processing of some code is started. 

You could track the submission order and pop cells when result are arrived then get the first cell in the stack. 
But then nothing can assure you that what is currently processed in the kernel is not something else from another frontend.

-- 
M

From satra at mit.edu  Fri Sep 27 08:30:33 2013
From: satra at mit.edu (Satrajit Ghosh)
Date: Fri, 27 Sep 2013 08:30:33 -0400
Subject: [IPython-dev] Microsoft donates $100,
	000 to IPython via NumFOCUS
In-Reply-To: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
References: <CAHAreOp=dNG_o_s90dZKsB=NnwBvMnP1+vSv7Y9VUbYPWK8H_g@mail.gmail.com>
Message-ID: <CA+A4wOng0Rad9ySFNVzmTbkPSgSTddpS6PfzKQ++bmSMtNw86w@mail.gmail.com>

hi fernando,

congratulations! this is wonderful news and will support a fantastic effort.

cheers,

satra

On Thu, Sep 26, 2013 at 9:45 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi all,
>
> as reported here:
>
> http://ipython.org/microsoft-donation-2013.html
>
> today we're very happy to announce that Microsoft has made a donation of
> $100,000 to support the continued development of IPython.  We're extremely
> grateful for this contribution.
>
> Making this possible was a combination of our ongoing collaboration with
> Microsoft along with the efforts of NumFOCUS, whose 501c3 status makes it
> possible to handle these financial resources for our community.
>
> I'd like to thank everyone in the NumFOCUS board for all the
> behind-the-scenes work they've put into making NumFOCUS a reality.  But in
> particular I want to highlight how Leah and Anthony put a ton of effort
> into the logistics necessary for this.
>
> Best,
>
> f
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/d63adf85/attachment.html>

From ehmatthes at gmail.com  Fri Sep 27 12:15:49 2013
From: ehmatthes at gmail.com (Eric Matthes)
Date: Fri, 27 Sep 2013 08:15:49 -0800
Subject: [IPython-dev] Appropriate use of notebook viewer?
Message-ID: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>

Hi everyone,

I wrote a post<http://peak5390.wordpress.com/2013/09/22/how-ipython-notebook-and-github-have-changed-the-way-i-teach-python/>this
week about how IPython Notebook has changed the way I teach
programming, and it got some good publicity on
HN<https://news.ycombinator.com/item?id=6437925>.
The response from people was overwhelmingly supportive, well beyond what I
expected.

For people who didn't read the post, I basically created a set of
notebooks<http://nbviewer.ipython.org/urls/raw.github.com/ehmatthes/intro_programming/master/notebooks/intro_programming_index.ipynb>as
a basis for an intro to programming class. I posted them on github, so
that students could see the notebooks on Notebook Viewer. The viewer was
perfect for sharing my teaching notebooks with my students.

After sharing that blog post, I am wondering how well-used these notebooks
might become if the project is completed. How appropriate is Notebook
Viewer for a project like this?

I have considered doing my own conversion to html and setting up a static
site, but for now I really like how pushing changes to github updates the
public-facing project.

Thanks,

Eric Matthes
Sitka, AK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/45bccc45/attachment.html>

From jakevdp at cs.washington.edu  Fri Sep 27 12:40:41 2013
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Fri, 27 Sep 2013 09:40:41 -0700
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
Message-ID: <CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>

Hi Eric,
Interesting post!  I think nbviewer works well, but I've opted to spin my
own solution for a course I'm teaching currently.  The result is an
nbconvert plugin for the Hyde website generator that automatically renders
my notebooks to HTML pages.  If you want to see it in action, you can check
out the website at [1]. I started teaching the course last week, and it's
worked extremely well. For each notebook, I link to the html rendering, the
raw version, and views on both nbviewer and wakari.  The nbconvert plugin
can be found within my Hyde pull request [2], and the source used to build
the website is on github [3].  I hope that's useful!
   Jake

[1] http://www.astro.washington.edu/users/vanderplas/Astr599/schedule
[2] https://github.com/hyde/hyde/pull/233
[3] https://github.com/jakevdp/2013_fall_ASTR599/tree/master/website



On Fri, Sep 27, 2013 at 9:15 AM, Eric Matthes <ehmatthes at gmail.com> wrote:

> Hi everyone,
>
> I wrote a post<http://peak5390.wordpress.com/2013/09/22/how-ipython-notebook-and-github-have-changed-the-way-i-teach-python/>this week about how IPython Notebook has changed the way I teach
> programming, and it got some good publicity on HN<https://news.ycombinator.com/item?id=6437925>.
> The response from people was overwhelmingly supportive, well beyond what I
> expected.
>
> For people who didn't read the post, I basically created a set of
> notebooks<http://nbviewer.ipython.org/urls/raw.github.com/ehmatthes/intro_programming/master/notebooks/intro_programming_index.ipynb>as a basis for an intro to programming class. I posted them on github, so
> that students could see the notebooks on Notebook Viewer. The viewer was
> perfect for sharing my teaching notebooks with my students.
>
> After sharing that blog post, I am wondering how well-used these notebooks
> might become if the project is completed. How appropriate is Notebook
> Viewer for a project like this?
>
> I have considered doing my own conversion to html and setting up a static
> site, but for now I really like how pushing changes to github updates the
> public-facing project.
>
> Thanks,
>
> Eric Matthes
> Sitka, AK
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/b2f13f89/attachment.html>

From stefan at sun.ac.za  Fri Sep 27 12:49:12 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 27 Sep 2013 18:49:12 +0200
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
	<CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>
Message-ID: <CABDkGQkQwLE8Mnk0ACNH14TcTXu6BHvGD4RB2PO-7FsZe+mM4w@mail.gmail.com>

Hi Jake

On Fri, Sep 27, 2013 at 6:40 PM, Jacob Vanderplas
<jakevdp at cs.washington.edu> wrote:
> [1] http://www.astro.washington.edu/users/vanderplas/Astr599/schedule
> [2] https://github.com/hyde/hyde/pull/233
> [3] https://github.com/jakevdp/2013_fall_ASTR599/tree/master/website

This is great, thanks for sharing!  As an aside, I was a bit confused
when I opened up the notebook in a wide-screen browser:

http://www.astro.washington.edu/users/vanderplas/Astr599/notebooks/03_IPython_intro

Double column notebooks--bug or feature? :)

St?fan


From takowl at gmail.com  Fri Sep 27 13:08:51 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 27 Sep 2013 10:08:51 -0700
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
Message-ID: <CAOvn4qjwMAXPF1yE195YS1kU8Si2rNENhQhY2RLhV=v0mJHoKQ@mail.gmail.com>

Hi Eric,

On 27 September 2013 09:15, Eric Matthes <ehmatthes at gmail.com> wrote:

> After sharing that blog post, I am wondering how well-used these notebooks
> might become if the project is completed. How appropriate is Notebook
> Viewer for a project like this?
>
> I have considered doing my own conversion to html and setting up a static
> site, but for now I really like how pushing changes to github updates the
> public-facing project.
>

To date, I don't think we're concerned about load on nbviewer. We're keen
to make it as easy as possible for people to share notebooks, so if it's
working for you, there's no pressure to change.

We are aiming to make it easy for people to do static conversions, as well.
For IPython 2.0, we're aiming to have an option in the notebook interface
to export as HTML. And we're definitely keen to see things like Jake's Hyde
plugin. This isn't just for our benefit - serving static HTML files removes
a potential cause of breakage and slowness. You can also send converted
files by e-mail, without posting your notebooks on the public web.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/da7d7204/attachment.html>

From damianavila at gmail.com  Fri Sep 27 13:11:28 2013
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Fri, 27 Sep 2013 14:11:28 -0300
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
	<CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>
Message-ID: <CAH+mRR19jpTWJ-kjuQ+_=YjGd5x4B-yfWseZdnprteqbj0cj_w@mail.gmail.com>

Adding more info... you can also use Nikola [1], a static site generator
that supports not only blogging with IPython notebooks [2], but also the
generation of more general sites, and soon... larger structural units, such
as books or series (of lectures) [3]

[1] http://getnikola.com/
[2]
http://www.damian.oquanta.info/posts/ipython-plugin-for-nikola-updated.html
[3] http://paginated.ralsina.com.ar/stories/a-study-in-scarlet.html (very
experimental)


2013/9/27 Jacob Vanderplas <jakevdp at cs.washington.edu>

> Hi Eric,
> Interesting post!  I think nbviewer works well, but I've opted to spin my
> own solution for a course I'm teaching currently.  The result is an
> nbconvert plugin for the Hyde website generator that automatically renders
> my notebooks to HTML pages.  If you want to see it in action, you can check
> out the website at [1]. I started teaching the course last week, and it's
> worked extremely well. For each notebook, I link to the html rendering, the
> raw version, and views on both nbviewer and wakari.  The nbconvert plugin
> can be found within my Hyde pull request [2], and the source used to build
> the website is on github [3].  I hope that's useful!
>    Jake
>
> [1] http://www.astro.washington.edu/users/vanderplas/Astr599/schedule
> [2] https://github.com/hyde/hyde/pull/233
> [3] https://github.com/jakevdp/2013_fall_ASTR599/tree/master/website
>
>
>
> On Fri, Sep 27, 2013 at 9:15 AM, Eric Matthes <ehmatthes at gmail.com> wrote:
>
>> Hi everyone,
>>
>> I wrote a post<http://peak5390.wordpress.com/2013/09/22/how-ipython-notebook-and-github-have-changed-the-way-i-teach-python/>this week about how IPython Notebook has changed the way I teach
>> programming, and it got some good publicity on HN<https://news.ycombinator.com/item?id=6437925>.
>> The response from people was overwhelmingly supportive, well beyond what I
>> expected.
>>
>> For people who didn't read the post, I basically created a set of
>> notebooks<http://nbviewer.ipython.org/urls/raw.github.com/ehmatthes/intro_programming/master/notebooks/intro_programming_index.ipynb>as a basis for an intro to programming class. I posted them on github, so
>> that students could see the notebooks on Notebook Viewer. The viewer was
>> perfect for sharing my teaching notebooks with my students.
>>
>> After sharing that blog post, I am wondering how well-used these
>> notebooks might become if the project is completed. How appropriate is
>> Notebook Viewer for a project like this?
>>
>> I have considered doing my own conversion to html and setting up a static
>> site, but for now I really like how pushing changes to github updates the
>> public-facing project.
>>
>> Thanks,
>>
>> Eric Matthes
>> Sitka, AK
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


-- 
Dami?n Avila
Scientific Python Developer
Quantitative Finance Analyst
Statistics, Biostatistics and Econometrics Consultant
Biochemist
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/b139ce1b/attachment.html>

From jakevdp at cs.washington.edu  Fri Sep 27 13:21:10 2013
From: jakevdp at cs.washington.edu (Jacob Vanderplas)
Date: Fri, 27 Sep 2013 10:21:10 -0700
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CABDkGQkQwLE8Mnk0ACNH14TcTXu6BHvGD4RB2PO-7FsZe+mM4w@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
	<CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>
	<CABDkGQkQwLE8Mnk0ACNH14TcTXu6BHvGD4RB2PO-7FsZe+mM4w@mail.gmail.com>
Message-ID: <CACpqBg0GRD02TL+GD1WnGCJHzjs_=HyHZHQMxe_gPiC13YGLdQ@mail.gmail.com>

On Fri, Sep 27, 2013 at 9:49 AM, St?fan van der Walt <stefan at sun.ac.za>wrote:

>
> Double column notebooks--bug or feature? :)
>

Strange: I don't see double columns in my browser, even when I go 2560
pixels wide.  Probably a bug ;)
   Jake


>
> St?fan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/5c97ee8e/attachment.html>

From takowl at gmail.com  Fri Sep 27 13:25:56 2013
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 27 Sep 2013 10:25:56 -0700
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CACpqBg0GRD02TL+GD1WnGCJHzjs_=HyHZHQMxe_gPiC13YGLdQ@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
	<CACpqBg1HnJSV2R2ZjSV1Tnedjc3thODVSjvJ5Pz0ia+gg0XiHQ@mail.gmail.com>
	<CABDkGQkQwLE8Mnk0ACNH14TcTXu6BHvGD4RB2PO-7FsZe+mM4w@mail.gmail.com>
	<CACpqBg0GRD02TL+GD1WnGCJHzjs_=HyHZHQMxe_gPiC13YGLdQ@mail.gmail.com>
Message-ID: <CAOvn4qi-riJVmsuWGUpPuD=H9etYd=JD3iSG9XA5py1X-MMcVw@mail.gmail.com>

On 27 September 2013 10:21, Jacob Vanderplas <jakevdp at cs.washington.edu>wrote:

> Strange: I don't see double columns in my browser, even when I go 2560
> pixels wide.  Probably a bug ;)


I see it too, on Firefox. Here's a screenshot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/8ba6df23/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2013-09-27 10:25:01.png
Type: image/png
Size: 27183 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130927/8ba6df23/attachment.png>

From fperez.net at gmail.com  Fri Sep 27 19:15:54 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 27 Sep 2013 16:15:54 -0700
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
Message-ID: <CAHAreOoEnLfaadi-TgJsDZcYAxONMMmFyP0mQ42HaKVmECYRbA@mail.gmail.com>

Hi Eric,

thanks for sharing this, I was very happy to see your post when it
showed up on HN.  It seems there was a lot of interest and good
discussion there.

The role of nbviewer *today* is to make it as easy as possible to
share notebooks with anyone online, whether colleagues or the wider
world.  But it's not a tool to build a community around a specific
group of notebooks. There's no commenting, indexing, search, etc.

Just like Jake uses very effectively notebooks in his always
interesting blog, if you want a more persistent discussion active
around a group of notebooks you may want to host them yourself.
That's totally OK! All that nbviewer provides is a very convenient,
zero-install service to help people share notebooks with minimal
effort, that's all.

We obviously think about this and we may want to consider making
nbviewer a more powerful service in the long run.  But right now we're
stretched about as thin as possible with the core code and design
efforts, so in keeping with our philosophy of trying to maintain a
sharp focus, we probably won't touch nbviewer too much for now.

Cheers,

f


-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From bussonniermatthias at gmail.com  Sat Sep 28 04:13:42 2013
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 28 Sep 2013 10:13:42 +0200
Subject: [IPython-dev] Appropriate use of notebook viewer?
In-Reply-To: <CAHAreOoEnLfaadi-TgJsDZcYAxONMMmFyP0mQ42HaKVmECYRbA@mail.gmail.com>
References: <CAH-3OQ4eEO8K7z+iXR+YDz_aZXgSc5_+_HxpHf2dGHPa+LHV7A@mail.gmail.com>
	<CAHAreOoEnLfaadi-TgJsDZcYAxONMMmFyP0mQ42HaKVmECYRbA@mail.gmail.com>
Message-ID: <7097CDBF-29C4-43CF-8E92-7AB6B4178F9F@gmail.com>

Hi Eric, 

As Fernando also said, this is a post that I think many
of us read and where happy to see. 

Your use on nbviewer is perfectly reasonable, just be aware that if github goes
down usually nbviewer goes down to (if your notebook are hosted on github) So I
would suggest often having a backup solution.  This might be running nbviewer
locally on your machine. Of course we plan on making this easier to do in the long run.

I also had a prototype of nbviewer able to browse github, but we definitively lack time
to do that. 

In the mean time I may suggest doing a landing page for your projects : like this

http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/

We also have a script to generate markdown index that link to nbviewer

	https://github.com/ipython/ipython/blob/master/tools/mknbindex.py

Then in the repo you have a easy link form reach notebook to the nbviewer version. 

	https://github.com/ipython/ipython/tree/master/examples/notebooks


Lastly if your student are interesting in hacking a project, there are a lot of small improvement
bug fixes that are waiting for them on IPython/nbconvert/nbviewer :-D.

Thanks, and happy to see IPython being used for teaching  and by people like you :-)
-- 
M







From zeitlinie at yahoo.de  Sun Sep 29 11:34:13 2013
From: zeitlinie at yahoo.de (Name Name)
Date: Sun, 29 Sep 2013 16:34:13 +0100 (BST)
Subject: [IPython-dev] Slow ipython --pylab and ipython notebook startup
Message-ID: <1380468853.96386.YahooMailNeo@web133202.mail.ir2.yahoo.com>

This is a cross-post from stackoverflow. I know, this is not absolutelyPC, but hopefully someone on this list can provide help. 
I have switched from ipython 0.10 to 1.1.0. Now I am experiencing very annoying slow-downs of the startup process.

While `ipython` alone is still up in no time, `ipython --pylab` takes a very slow start, i.e. `~8 secs` (on an Intel(R) Core(TM)2 Duo CPU P9500 @ 2.53GHz system) and even more so the new `ipython notebook` which I did not have in ipython 0.10, namely `~12 secs`.

When I do `strace -o tessi.txt -tt ipython --pylab` I can identify at least sections which seem to be responsible for large chunks of these delays (search for HERE!!). Any help on getting rid of those would very welcome.

For the `--pylab` option, strace's output contains a section which eats away `~5 secs`, and which is not present when starting just ipython, namely

??? 10:23:24.331968 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=868, ...}) = 0
??? 10:23:24.332028 socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 9
??? 10:23:24.332074 connect(9, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = 0
??? 10:23:24.332152 sendto(9, "\2\0\0\0\4\0\0\0\35\0\0\0MYHOST.DOM.AIN."..., 41, MSG_NOSIGNAL, NULL, 0) = 41
HERE!! -> 10:23:24.332227 poll([{fd=9, events=POLLIN|POLLERR|POLLHUP}], 1, 5000) = 1 ([{fd=9, revents=POLLIN|POLLHUP}])
HERE!! -> 10:23:29.336301 read(9, "\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 32) = 32
??? 10:23:29.336533 close(9)??????????????? = 0
??? 10:23:29.336839 close(7)??????????????? = 0

where I've replace my actual host name with `MYHOST.DOM.AIN`

In case of `ipython notebook` there seem to be essentially two such regions. The first one seems to be the same as for `ipython --pylab`. The second one takes away another `~4 secs`

??? 10:39:31.823298 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=868, ...}) = 0
??? 10:39:31.823358 socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 9
??? 10:39:31.823405 connect(9, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = 0
??? 10:39:31.823499 sendto(9, "\2\0\0\0\4\0\0\0\35\0\0\0MYHOST.DOM.AIN."..., 41, MSG_NOSIGNAL, NULL, 0) = 41
HERE!! -> 10:39:31.824166 poll([{fd=9, events=POLLIN|POLLERR|POLLHUP}], 1, 5000) = 1 ([{fd=9, revents=POLLIN|POLLHUP}])
HERE!! -> 10:39:36.827298 read(9, "\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 32) = 32
??? 10:39:36.827503 close(9)??????????????? = 0
??? 10:39:36.827828 close(7)??????????????? = 0??? 
??? 
??? 10:39:38.591774 setsockopt(6, SOL_TCP, TCP_NODELAY, [0], 4) = 0
??? 10:39:38.591922 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}], 4, 0) = 0 (Timeout)
??? 10:39:38.592007 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}], 4, 0) = 0 (Timeout)
HERE!! -> 10:39:38.592049 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}], 4, 3600000) = ? ERESTART_RESTARTBLOCK (To be restarted)
HERE!! -> 10:39:42.355658 --- SIGINT (Interrupt) @ 0 (0) ---
??? 10:39:42.355773 write(5, "\0", 1)?????? = 1
??? 10:39:42.355916 rt_sigreturn(0x2)?????? = -1 EINTR (Interrupted system call)
??? 10:39:42.356113 rt_sigaction(SIGINT, {0x7f26dc0fbfc6, [], SA_RESTORER, 0x7f26dbe5a2d0}, {0x7f26dc0fbfc6, [], SA_RESTORER, 0x7f26dbe5a2d0}, 8) = 0
??? 10:39:42.356672 clone(child_stack=0x7f26cfb49ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f26cfb4a9d0, tls=0x7f26cfb4a700, child_tidptr=0x7f26cfb4a9d0) = 11084

Any help on how to get rid of these delays would be most welcome.

Finally, and while I'm at it. When I look at strace's remaing output I see, that on startup, ipython tries to open a lot of 'python related' files in places where I would have naively thought that it should never do so, and where my systems python definitely has not been installed? All these many open calls fail. E.g.:

??? 11:11:26.465594 open("/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/tls/libpython2.7.so.1.0", O_RDONLY) = -1 ENOENT (No such file or directory)
??? 11:11:26.465646 stat("/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/tls", 0x7fff0efb0100) = -1 ENOENT (No such file or directory)
??? 
??? or
???? 
??? 11:11:26.468293 stat("/usr/local/lib/vtk-5.4/tls/x86_64", 0x7fff0efb0100) = -1 ENOENT (No such file or directory)
??? 11:11:26.468347 open("/usr/local/lib/vtk-5.4/tls/libpython2.7.so.1.0", O_RDONLY) = -1 ENOENT (No such file or directory)

The time total for these calls make up for another significant portion of the remaining slow startup, but I cannot point to a specific time slot. Any idea how to reduce those calls?

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130929/0921596d/attachment.html>

From tapitman11 at gmail.com  Sun Sep 29 14:55:32 2013
From: tapitman11 at gmail.com (Tim Pitman)
Date: Sun, 29 Sep 2013 11:55:32 -0700
Subject: [IPython-dev] Slow ipython --pylab and ipython notebook startup
In-Reply-To: <1380468853.96386.YahooMailNeo@web133202.mail.ir2.yahoo.com>
References: <1380468853.96386.YahooMailNeo@web133202.mail.ir2.yahoo.com>
Message-ID: <CACnfkZsPdWtMTKfM3yb+XcDZ+=UZPmgAxien1Pw4ejTKOs3ERw@mail.gmail.com>

Not sure about the first couple, but the /opt/intel/composerxe* stuff is
MKL. /usr/local/lib/vtk* is the vtk plotting library. Not sure if that
helps at all. I've never tried setting IPython up with MKL so I don't know
how to enable/disable it. That might even be Numpys job.


On Sun, Sep 29, 2013 at 8:34 AM, Name Name <zeitlinie at yahoo.de> wrote:

> This is a cross-post from stackoverflow. I know, this is not absolutely PC, but hopefully someone on this list can provide help.
>
> I have switched from ipython 0.10 to 1.1.0. Now I am experiencing very
> annoying slow-downs of the startup process.
>
> While `ipython` alone is still up in no time, `ipython --pylab` takes a
> very slow start, i.e. `~8 secs` (on an Intel(R) Core(TM)2 Duo CPU P9500 @
> 2.53GHz system) and even more so the new `ipython notebook` which I did not
> have in ipython 0.10, namely `~12 secs`.
>
> When I do `strace -o tessi.txt -tt ipython --pylab` I can identify at
> least sections which seem to be responsible for large chunks of these
> delays (search for HERE!!). Any help on getting rid of those would very
> welcome.
>
> For the `--pylab` option, strace's output contains a section which eats
> away `~5 secs`, and which is not present when starting just ipython, namely
>
>     10:23:24.331968 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644,
> st_size=868, ...}) = 0
>     10:23:24.332028 socket(PF_FILE,
> SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 9
>     10:23:24.332074 connect(9, {sa_family=AF_FILE,
> path="/var/run/nscd/socket"}, 110) = 0
>     10:23:24.332152 sendto(9,
> "\2\0\0\0\4\0\0\0\35\0\0\0MYHOST.DOM.AIN."..., 41, MSG_NOSIGNAL, NULL, 0) =
> 41
> HERE!! -> 10:23:24.332227 poll([{fd=9, events=POLLIN|POLLERR|POLLHUP}], 1,
> 5000) = 1 ([{fd=9, revents=POLLIN|POLLHUP}])
> HERE!! -> 10:23:29.336301 read(9,
> "\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 32) = 32
>     10:23:29.336533 close(9)                = 0
>     10:23:29.336839 close(7)                = 0
>
> where I've replace my actual host name with `MYHOST.DOM.AIN`
>
> In case of `ipython notebook` there seem to be essentially two such
> regions. The first one seems to be the same as for `ipython --pylab`. The
> second one takes away another `~4 secs`
>
>     10:39:31.823298 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644,
> st_size=868, ...}) = 0
>     10:39:31.823358 socket(PF_FILE,
> SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 9
>     10:39:31.823405 connect(9, {sa_family=AF_FILE,
> path="/var/run/nscd/socket"}, 110) = 0
>     10:39:31.823499 sendto(9,
> "\2\0\0\0\4\0\0\0\35\0\0\0MYHOST.DOM.AIN."..., 41, MSG_NOSIGNAL, NULL, 0) =
> 41
> HERE!! -> 10:39:31.824166 poll([{fd=9, events=POLLIN|POLLERR|POLLHUP}], 1,
> 5000) = 1 ([{fd=9, revents=POLLIN|POLLHUP}])
> HERE!! -> 10:39:36.827298 read(9,
> "\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 32) = 32
>     10:39:36.827503 close(9)                = 0
>     10:39:36.827828 close(7)                = 0
>
>     10:39:38.591774 setsockopt(6, SOL_TCP, TCP_NODELAY, [0], 4) = 0
>     10:39:38.591922 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN},
> {fd=6, events=POLLIN}, {fd=7, events=POLLIN}], 4, 0) = 0 (Timeout)
>     10:39:38.592007 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN},
> {fd=6, events=POLLIN}, {fd=7, events=POLLIN}], 4, 0) = 0 (Timeout)
> HERE!! -> 10:39:38.592049 poll([{fd=3, events=POLLIN}, {fd=4,
> events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}], 4, 3600000)
> = ? ERESTART_RESTARTBLOCK (To be restarted)
> HERE!! -> 10:39:42.355658 --- SIGINT (Interrupt) @ 0 (0) ---
>     10:39:42.355773 write(5, "\0", 1)       = 1
>     10:39:42.355916 rt_sigreturn(0x2)       = -1 EINTR (Interrupted system
> call)
>     10:39:42.356113 rt_sigaction(SIGINT, {0x7f26dc0fbfc6, [], SA_RESTORER,
> 0x7f26dbe5a2d0}, {0x7f26dc0fbfc6, [], SA_RESTORER, 0x7f26dbe5a2d0}, 8) = 0
>     10:39:42.356672 clone(child_stack=0x7f26cfb49ff0,
> flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
> parent_tidptr=0x7f26cfb4a9d0, tls=0x7f26cfb4a700,
> child_tidptr=0x7f26cfb4a9d0) = 11084
>
> Any help on how to get rid of these delays would be most welcome.
>
> Finally, and while I'm at it. When I look at strace's remaing output I
> see, that on startup, ipython tries to open a lot of 'python related' files
> in places where I would have naively thought that it should never do so,
> and where my systems python definitely has not been installed? All these
> many open calls fail. E.g.:
>
>     11:11:26.465594
> open("/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/tls/libpython2.7.so.1.0",
> O_RDONLY) = -1 ENOENT (No such file or directory)
>     11:11:26.465646
> stat("/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/tls",
> 0x7fff0efb0100) = -1 ENOENT (No such file or directory)
>
>     or
>
>     11:11:26.468293 stat("/usr/local/lib/vtk-5.4/tls/x86_64",
> 0x7fff0efb0100) = -1 ENOENT (No such file or directory)
>     11:11:26.468347 open("/usr/local/lib/vtk-5.4/tls/libpython2.7.so.1.0",
> O_RDONLY) = -1 ENOENT (No such file or directory)
>
> The time total for these calls make up for another significant portion of
> the remaining slow startup, but I cannot point to a specific time slot. Any
> idea how to reduce those calls?
>
> Mark
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130929/8e10da95/attachment.html>

From aron at ahmadia.net  Mon Sep 30 14:26:08 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Mon, 30 Sep 2013 14:26:08 -0400
Subject: [IPython-dev] Fixing silent failures with IPython Notebook / uuid
	on Cygwin64
Message-ID: <CAPhiW4jGSb8K8r1zNKS1iUDOTocLP1Jh5D1RVMqa4QUDiLouqg@mail.gmail.com>

Just a heads up for anybody who is trying to do development in Cygwin64.

First off, you will need to follow the instructions here for getting
ZMQ/PYZMQ:

https://github.com/zeromq/pyzmq/issues/113

If you are trying to run the notebook, you may encounter silent failures
when running IPython Notebook from the command line:

$ ipython notebook

(Simply exits with no messages/errors)

This is coming from a segfault in the Cygwin64 Python uuid module.  I tried
to follow the instructions here:
http://stackoverflow.com/questions/18947163/uuid-python-import-fails-on-cygwin-64bitsand
here:
http://bugs.python.org/file20685/issue11063.patch but this didn't fix the
crash.

My stop-gap solution is to disable the call to _uuid_generate in uuid4
post-patch.  This is reasonably safe, and just uses a different random
number generator instead of the system-provided uuid function.

def uuid4():
    """Generate a random UUID."""

    # When the system provides a version-4 UUID generator, use it.

+    # AA disabling _uuid_generate
+    uuid_generate_random = None
-    uuid_generate_random = _uuid_generate("uuid_generate_random")

The CPython and Cygwin folks appear to be aware of this issue, but since it
took some of my time to debug, I'm hoping this will help others
encountering similar problems in the future.

After switching away from a GitHub release build, I've got IPython Notebook
and the majority of its dependencies building from source using HashDist on
Cygwin64 with a few things coming from the system, such as the broken
Python patched above.

Happy coding.

A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130930/e7cb1548/attachment.html>