From hans_meine at gmx.net  Mon Sep  3 12:25:29 2012
From: hans_meine at gmx.net (Hans Meine)
Date: Mon, 03 Sep 2012 18:25:29 +0200
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
Message-ID: <16242357.L4Hq4o0Kl8@hmeine-pc>

Hi everybody!

Am Dienstag, 14. August 2012, 11:38:14 schrieb Thomas Kluyver:
> I've seen how gettext works from within the code, and I agree that it does
> seem simple. Can I ask about logistics:

One thing I learned from KDE (which BTW has put a lot of thought and effort 
into a good l10n infrastructure, too) is this:

  Short strings should *not* be just wrapped in _(...)!

To be more specific, there are many English words that should be translated 
into different words depending on the *context*.  For example, English 'to' 
translates to different words in German when used in "pick 1 to 4 items", "10 
to 11 o'clock", or "drive to Buffalo".  There are much more difficult problems 
than this, but this one arises most often, e.g. in user interfaces, in which 
labels for dialog fields contain just 'to'.

I brought this up because the consequence is that every developer needs to 
know at least a little bit about the translation system, or someone (e.g. 
Nicolas) has to dive into the code everytime the translators have a problem.

Best regards,
  Hans



From nicolas at pettiaux.be  Mon Sep  3 12:39:11 2012
From: nicolas at pettiaux.be (Nicolas Pettiaux)
Date: Mon, 3 Sep 2012 18:39:11 +0200
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <16242357.L4Hq4o0Kl8@hmeine-pc>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
Message-ID: <CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>

Hello

I have not had the time to answer Thomas (whom I was really happy to
meet at EuroScipy in Brussels), exactely because of the work around
EuroScipy and my main work thereafter.

I expect to be too busy before November 1 to do anything really
constructive, appart from reading and researching about the best
infrastructure to propose and, as Hans writes, also gather
recommandations and good practices from running projects that are
translated. And indeed, such subleties exists ... ans I am ready to
dive into the code for the sake of expansion of the use of ipython by
beginners. The point though is that I master English and French, I
speak rather fluently Dutch but I am not an expert and that is it.

Best regards,

Nicolas


-- 
Nicolas Pettiaux, dr. sc - gsm : +32 496 24 55 01
Lepacte.be - ? promouvoir les libert?s num?riques en Belgique ? - hetpact.be


From takowl at gmail.com  Mon Sep  3 12:54:14 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 3 Sep 2012 17:54:14 +0100
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
Message-ID: <CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>

On 3 September 2012 17:39, Nicolas Pettiaux <nicolas at pettiaux.be> wrote:
> The point though is that I master English and French, I
> speak rather fluently Dutch but I am not an expert and that is it.

I hope that once we've got the infrastructure set up, adding new
languages will be something relatively easy that many people can get
involved with. The results of the user survey indicate that we have
users in a lot of different countries:

http://ipython.org/usersurvey2011.html#countries

In terms of the framework, I would be wary of using anything other
than the standard library gettext module unless there's a clear need
for it. Lots of other software manages with gettext.

Thanks,
Thomas


From nicolas at pettiaux.be  Mon Sep  3 13:08:11 2012
From: nicolas at pettiaux.be (Nicolas Pettiaux)
Date: Mon, 3 Sep 2012 19:08:11 +0200
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
	<CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
Message-ID: <CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>

2012/9/3 Thomas Kluyver <takowl at gmail.com>:
> On 3 September 2012 17:39, Nicolas Pettiaux <nicolas at pettiaux.be> wrote:
>> The point though is that I master English and French, I
>> speak rather fluently Dutch but I am not an expert and that is it.
>
> I hope that once we've got the infrastructure set up, adding new
> languages will be something relatively easy that many people can get
> involved with.

This is exactely what I hope. And there exist many different
infrastructure today,

http://transifex.com
http://weblate.org
http://translatewiki.net
http://pootle.python.org/

that we could use. One thing that I am now trying to do is : compare
them. Some are more appropriate for menus or short sentences, some
other for plain texts and examples, or even books.

>The results of the user survey indicate that we have
> users in a lot of different countries:
>
> http://ipython.org/usersurvey2011.html#countries
>
> In terms of the framework, I would be wary of using anything other
> than the standard library gettext module unless there's a clear need
> for it.

so am I.

> Lots of other software manages with gettext.

indeed. See above for online solutions. There are also desktop ones :
gtranslator, virtaal, omegat, lokalize and probably others.

Thanks,

Nicolas
-- 
Nicolas Pettiaux, dr. sc - gsm : +32 496 24 55 01
Lepacte.be - ? promouvoir les libert?s num?riques en Belgique ? - hetpact.be


From takowl at gmail.com  Mon Sep  3 16:01:17 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 3 Sep 2012 21:01:17 +0100
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
	<CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
	<CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>
Message-ID: <CAOvn4qiHcgWtvvJotw+g6qiOgttH1hYcJwCEqniy+pudpRr0dw@mail.gmail.com>

On 3 September 2012 18:08, Nicolas Pettiaux <nicolas at pettiaux.be> wrote:
> This is exactely what I hope. And there exist many different
> infrastructure today,
>
> http://transifex.com
> http://weblate.org
> http://translatewiki.net
> http://pootle.python.org/

I think that's the relatively easy bit - any of those services, along
with various desktop applications, will let people translate messages
using a standard file format. What I'm less sure of is how we should
integrate the translation files into our development workflow and our
distribution methods. For example:
- If I change a string in development, do I need to remember to update
message catalogues?
- Should translators be working on the development version, or the
stable release?
- Should the IPython package you download include translation files,
or should they be separately downloadable?
- What will distro packagers expect from us?

Thanks,
Thomas


From nicolas at pettiaux.be  Mon Sep  3 16:52:32 2012
From: nicolas at pettiaux.be (Nicolas Pettiaux)
Date: Mon, 3 Sep 2012 22:52:32 +0200
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAOvn4qiHcgWtvvJotw+g6qiOgttH1hYcJwCEqniy+pudpRr0dw@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
	<CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
	<CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>
	<CAOvn4qiHcgWtvvJotw+g6qiOgttH1hYcJwCEqniy+pudpRr0dw@mail.gmail.com>
Message-ID: <CAEHX+gnz-w2LFUTne-AyoSP9LJ=_vL5D02JuWHE+-NiOXc0Evw@mail.gmail.com>

2012/9/3 Thomas Kluyver <takowl at gmail.com>:
> On 3 September 2012 18:08, Nicolas Pettiaux <nicolas at pettiaux.be> wrote:
>> This is exactely what I hope. And there exist many different
>> infrastructure today,
>>
>> http://transifex.com
>> http://weblate.org
>> http://translatewiki.net
>> http://pootle.python.org/
>
> I think that's the relatively easy bit - any of those services, along
> with various desktop applications, will let people translate messages
> using a standard file format.

indeed. But a good infrastructure will help a lot the translators community.

> What I'm less sure of is how we should
> integrate the translation files into our development workflow and our
> distribution methods. For example:

> - If I change a string in development, do I need to remember to update
> message catalogues?

no. Gettext tools will do that. Keep develop using _() for the strings.

> - Should translators be working on the development version, or the
> stable release?

mostly the stable release as far as I know. The gettext tool set helps
to follow the variation, and will only ask to translate the string
that have changed.

> - Should the IPython package you download include translation files,
> or should they be separately downloadable?

I think the package have to include the translations, at least if they
are not big (which will be the case at the beginning).
> - What will distro packagers expect from us?

I think they will expect what they get today and the translated files
(= some more text documents)

Thanks,

Nicolas

-- 
Nicolas Pettiaux, dr. sc - gsm : +32 496 24 55 01
Lepacte.be - ? promouvoir les libert?s num?riques en Belgique ? - hetpact.be


From tomspur at fedoraproject.org  Mon Sep  3 18:06:07 2012
From: tomspur at fedoraproject.org (Thomas Spura)
Date: Tue, 4 Sep 2012 00:06:07 +0200
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAEHX+gnz-w2LFUTne-AyoSP9LJ=_vL5D02JuWHE+-NiOXc0Evw@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
	<CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
	<CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>
	<CAOvn4qiHcgWtvvJotw+g6qiOgttH1hYcJwCEqniy+pudpRr0dw@mail.gmail.com>
	<CAEHX+gnz-w2LFUTne-AyoSP9LJ=_vL5D02JuWHE+-NiOXc0Evw@mail.gmail.com>
Message-ID: <CAE4GLsu8eX9pc2SmkD_HyubZM0o_m9D1gH8x_x7aEr7vjj7Omw@mail.gmail.com>

On Mon, Sep 3, 2012 at 10:52 PM, Nicolas Pettiaux <nicolas at pettiaux.be> wrote:
> 2012/9/3 Thomas Kluyver <takowl at gmail.com>:
>> On 3 September 2012 18:08, Nicolas Pettiaux <nicolas at pettiaux.be> wrote:
>>> This is exactely what I hope. And there exist many different
>>> infrastructure today,
>>>
>>> http://transifex.com
>>> http://weblate.org
>>> http://translatewiki.net
>>> http://pootle.python.org/
>>
>> I think that's the relatively easy bit - any of those services, along
>> with various desktop applications, will let people translate messages
>> using a standard file format.
>
> indeed. But a good infrastructure will help a lot the translators community.

I only know transifex in practice, so my comments below will be biased
a bit towards that direction.

>> What I'm less sure of is how we should
>> integrate the translation files into our development workflow and our
>> distribution methods. For example:
>
>> - If I change a string in development, do I need to remember to update
>> message catalogues?
>
> no. Gettext tools will do that. Keep develop using _() for the strings.

You need to explicitely "tx push" the changed strings to transifex and
give the translators a bit time to translate. This means before the
actual release of a new version, the StringFreeze needs to happen,
where you only move code around, but don't change any _("") strings
anymore.
Just before the release "tx pull" will get the translations back again
and you can include them into the tar ball for downloading (I'm
unsure, if they need to be committed into git or if including into a
tarball is sufficient).

>> - Should translators be working on the development version, or the
>> stable release?
>
> mostly the stable release as far as I know. The gettext tool set helps
> to follow the variation, and will only ask to translate the string
> that have changed.

I'd suggest a string freeze, before each new release, so translators
have a bit time to translate the development version.
(Regularly pushing changed strings to tx would help to always keep a
low number of needed translation strings.)

>> - Should the IPython package you download include translation files,
>> or should they be separately downloadable?
>
> I think the package have to include the translations, at least if they
> are not big (which will be the case at the beginning).

I cannot remember a project, which splits the translations into
another repository off hand...
It might be good to split the translations of the webpage, as that
might become quite big and is normally not needed in a tarball (if you
want to let translate the webpage too...)

>> - What will distro packagers expect from us?
>
> I think they will expect what they get today and the translated files
> (= some more text documents)

They expect setup.py to compile the .po file into a .mo files and
install them into:
/usr/share/locale/$LANG/LC_MESSAGES/ipython.mo (when you use
gettext.install("ipython"))

Some quick googling showed this as an example for compiling the translations:
http://wiki.maemo.org/Internationalize_a_Python_application

Hope that helps.
     Tom


From takowl at gmail.com  Mon Sep  3 18:24:01 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 3 Sep 2012 23:24:01 +0100
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAE4GLsu8eX9pc2SmkD_HyubZM0o_m9D1gH8x_x7aEr7vjj7Omw@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
	<CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
	<CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>
	<CAOvn4qiHcgWtvvJotw+g6qiOgttH1hYcJwCEqniy+pudpRr0dw@mail.gmail.com>
	<CAEHX+gnz-w2LFUTne-AyoSP9LJ=_vL5D02JuWHE+-NiOXc0Evw@mail.gmail.com>
	<CAE4GLsu8eX9pc2SmkD_HyubZM0o_m9D1gH8x_x7aEr7vjj7Omw@mail.gmail.com>
Message-ID: <CAOvn4qgfs2iZTkBH6KBqBcZgLTGvaharj7afnzGwZyX=tQYLcA@mail.gmail.com>

On 3 September 2012 23:06, Thomas Spura <tomspur at fedoraproject.org> wrote:
>>> - Should the IPython package you download include translation files,
>>> or should they be separately downloadable?
>>
>> I think the package have to include the translations, at least if they
>> are not big (which will be the case at the beginning).
>
> I cannot remember a project, which splits the translations into
> another repository off hand...

I'm conscious that we recommend that people install IPython inside
every virtualenv where they're using it, so we want the installation
to be as quick and light as possible. The example from Maemo looks
like it would recompile every .po file to .mo on installation, which I
would prefer to avoid if practical.

Thomas


From nicolas at pettiaux.be  Tue Sep  4 01:00:41 2012
From: nicolas at pettiaux.be (Nicolas Pettiaux)
Date: Tue, 4 Sep 2012 07:00:41 +0200
Subject: [IPython-dev] internationalisation and translation of ipython
In-Reply-To: <CAOvn4qgfs2iZTkBH6KBqBcZgLTGvaharj7afnzGwZyX=tQYLcA@mail.gmail.com>
References: <CAEHX+gmAsaVYT0_nqFrE6xwdztc9+LuyLiUjvH7WcmCKN6=A4g@mail.gmail.com>
	<CAEHX+g=oFkAQ3sEGqBYZ-yo20ZPKMvcJgjp9-OQ5XnP=Wm4PLg@mail.gmail.com>
	<CAOvn4qhKntAJq3FPtmbQjXVjunwx+bRn5pj6C+_0p1B=CURh5Q@mail.gmail.com>
	<16242357.L4Hq4o0Kl8@hmeine-pc>
	<CAEHX+gnY5SrCdR4yFGewDca0qzdn3ZSWLFut_F9qpuz8a6d=4A@mail.gmail.com>
	<CAOvn4qgFmDtLkuaxGqbLAaiNHQPapt_hTpZs98k9BB8QTaqT_Q@mail.gmail.com>
	<CAEHX+g=h2KT8WC-_-r5r=9pPVhxa+xoQgjkYXdqOgH2qBhndww@mail.gmail.com>
	<CAOvn4qiHcgWtvvJotw+g6qiOgttH1hYcJwCEqniy+pudpRr0dw@mail.gmail.com>
	<CAEHX+gnz-w2LFUTne-AyoSP9LJ=_vL5D02JuWHE+-NiOXc0Evw@mail.gmail.com>
	<CAE4GLsu8eX9pc2SmkD_HyubZM0o_m9D1gH8x_x7aEr7vjj7Omw@mail.gmail.com>
	<CAOvn4qgfs2iZTkBH6KBqBcZgLTGvaharj7afnzGwZyX=tQYLcA@mail.gmail.com>
Message-ID: <CAEHX+gnityqg6o0B_R4fXAmV7KtCAMKUGe6ofe74SdHtz-Y=tA@mail.gmail.com>

2012/9/4 Thomas Kluyver <takowl at gmail.com>

> On 3 September 2012 23:06, Thomas Spura <tomspur at fedoraproject.org> wrote:
>
thanks Thomas S for you input.



> > I cannot remember a project, which splits the translations into
> > another repository off hand...
>
> maybe not split the translation, but distribute them separately. When they
are large, it is done. See kde, libreOffice, Gnome.

But I beleive that for ipython (menus, basic help ...) it will not be much
at all.


> I'm conscious that we recommend that people install IPython inside
> every virtualenv where they're using it, so we want the installation
> to be as quick and light as possible.

I think it will be. Everything can be automated as far as I know.

The .po and .mo file can both be distributed.


> The example from Maemo looks
> like it would recompile every .po file to .mo on installation, which I
> would prefer to avoid if practical.
>
> I suppose this can be avoided. Need to investigate.

Have a good day,

NP
-- 
Nicolas Pettiaux, dr. sc - gsm : +32 496 24 55 01
Lepacte.be - ? promouvoir les libert?s num?riques en Belgique ? - hetpact.be
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120904/a38605ab/attachment.html>

From hans_meine at gmx.net  Tue Sep  4 08:52:57 2012
From: hans_meine at gmx.net (Hans Meine)
Date: Tue, 04 Sep 2012 14:52:57 +0200
Subject: [IPython-dev] Bug in our magic_arguments decorator for building
	docstrings (ping R. Kern)?
In-Reply-To: <CAHAreOpV=qcnrFd9M3OGPPwPDKrmRoCFqt4dR5-okNozYnmKGw@mail.gmail.com>
References: <CAHAreOpfo8qDLLm_Sj=x4OcWBxzVv+teMveOXustvobLhRzWMw@mail.gmail.com>
	<jqlrsb$kia$1@dough.gmane.org>
	<CAHAreOpV=qcnrFd9M3OGPPwPDKrmRoCFqt4dR5-okNozYnmKGw@mail.gmail.com>
Message-ID: <25916473.9YhpqHY4JC@hmeine-pc>

Am Dienstag, 5. Juni 2012, 14:00:51 schrieb Fernando Perez:
> On Tue, Jun 5, 2012 at 1:57 PM, Robert Kern <robert.kern at gmail.com> wrote:
> > We probably want to use the RawDescriptionHelpFormatter:
> > 
> > 
http://docs.python.org/library/argparse#argparse.RawDescriptionHelpFormatter
> > 
> > Or a custom implementation that removes the leading indentation but
> > otherwise leaves the line breaks and indents alone.
> 
> Great, thanks for tracking down at least a path to a solution.
> 
> If anyone has the bandwidth to whip out a fix for this, we'd be very
> very grateful.  Right now the docs for any magic with more than one
> line docstring that uses the argparse decorators is completely
> unreadable, so the karma-to-effort ratio of this one is huge.

Has this been fixed, or is there a GH issue for this problem?

Best, Hans


From fperez.net at gmail.com  Tue Sep  4 16:05:31 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 4 Sep 2012 13:05:31 -0700
Subject: [IPython-dev] Bug in our magic_arguments decorator for building
 docstrings (ping R. Kern)?
In-Reply-To: <25916473.9YhpqHY4JC@hmeine-pc>
References: <CAHAreOpfo8qDLLm_Sj=x4OcWBxzVv+teMveOXustvobLhRzWMw@mail.gmail.com>
	<jqlrsb$kia$1@dough.gmane.org>
	<CAHAreOpV=qcnrFd9M3OGPPwPDKrmRoCFqt4dR5-okNozYnmKGw@mail.gmail.com>
	<25916473.9YhpqHY4JC@hmeine-pc>
Message-ID: <CAHAreOonQR+-dQXhdu0yzR0E=Vj_XAShaFjUL0Hp4v2J_+8xNw@mail.gmail.com>

On Tue, Sep 4, 2012 at 5:52 AM, Hans Meine <hans_meine at gmx.net> wrote:

> Has this been fixed, or is there a GH issue for this problem?

Fixed here:

commit 673e53310ce16220ccc235eb0da56df9f654df4f
Merge: 29dc2c6 df97f2d
Author: Fernando Perez <fernando.perez at berkeley.edu>
Date:   Mon Jun 11 21:14:18 2012 -0700

    Merge pull request #1921 from bfroehle/_1890_magic_arguments_docstring

    magic_arguments: dedent but otherwise preserve indentation.

    Previously magic_arguments stripped all whitespace at the
beginning of each line, interferi
ng with formatting syntax which relies on indentation to give context
(e.g., code blocks). Now
the docstring text is passed through dedent to strip the global
indentation before being handed
 off to `RawDescriptionHelpFormatter` which preserves any remaining indentation.

    Thanks to @rkern for suggesting the solution approach.

    Closes gh-1890.


From massimodisasha at gmail.com  Wed Sep  5 08:28:55 2012
From: massimodisasha at gmail.com (Massimo Di Stefano)
Date: Wed, 5 Sep 2012 08:28:55 -0400
Subject: [IPython-dev] R magic extension and default R graphic drivers
In-Reply-To: <249CAC27-8648-40DF-9277-CB2DF8432602@gmail.com>
References: <249CAC27-8648-40DF-9277-CB2DF8432602@gmail.com>
Message-ID: <26A7A009-FDD5-4F81-ADA7-FC0FB82BC0C8@gmail.com>

Hi All,

i'm tring to modify withput lucky the src code in the rmagic extension
in order to use the "type=cairo" as argument to the png() command.

> png(file="myplot.png", bg="transparent", type="cairo'")

please have you any hints ?

thanks,

Massimo.


Il giorno Aug 31, 2012, alle ore 7:45 PM, Massimo Di Stefano ha scritto:

> Hi All,
> 
> 
> 
> i'm trying to run some R code from inside a notebook running on a server that doesn't have X running
> the missing X ... generate an error with the default png driver.
> 
> 
> if i try to run some R code that involve the graphic drivers
> i got an error :
> 
> RRuntimeError: Error in X11(paste("png::", filename, sep = ""), g$width, g$height, pointsize,  : 
>   unable to start device PNG
> 
> 
> 
> full error log is here : 
> 
> http://epifanio.whoi.edu/data/img/rext_log.txt
> 
> 
> 
> i guess, the reason of this error is because the server doesn't allow X fwd 
> and seems doesn't have X running at all.
> 
> i can reproduce this error (or similar) also from the R console itself :
> 
> > png(file="myplot.png", bg="transparent")
> Error in X11(paste("png::", filename, sep = ""), g$width, g$height, pointsize,  : 
>   unable to start device PNG
> In addition: Warning message:
> In png(file = "myplot.png", bg = "transparent") :
>   unable to open connection to X11 display ''
> > 
> 
> it can be avoided if i set the driver to "cairo" :
> 
> > options(bitmapType="cairo") 
> > png(file="myplot.png", bg="transparent")
> > 
> 
> The solution in R was to write a file  ".Rprofile" in my home directory, so now i don't need anymore to use options(bitmapType="cairo")  in the R prompt.
> 
> i was hoping that this option was able to fix also the error inside the IPython notebook .. but instead it persist.
> 
> 
> 
> from the error log i can see that in the file :
> 
> IPython/extensions/rmagic.py  
> 
> line 515
> 
> we have :
> 
> self.r('png("%s/Rplots%%03d.png",%s)' % (tmpd, png_args))
> 
> perhaps it is possible to set the driver to be used directly in rmagic code,
> have you any clue on how can i fix this problem?
> 
> thanks a lot!
> 
> Massimo.
> 
> [1]
> 
> 
> 
>  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120905/a8b780d6/attachment.html>

From shoibalc at gmail.com  Wed Sep  5 09:09:05 2012
From: shoibalc at gmail.com (Shoibal Chakravarty)
Date: Wed, 05 Sep 2012 09:09:05 -0400
Subject: [IPython-dev] ipython html notebook server on dotcloud
Message-ID: <50474EF1.8080104@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120905/a0918627/attachment.html>

From carl.input at gmail.com  Wed Sep  5 10:19:15 2012
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 5 Sep 2012 15:19:15 +0100
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <50474EF1.8080104@gmail.com>
References: <50474EF1.8080104@gmail.com>
Message-ID: <CAP-uhDf-AUPMYZUOfs-PCfwhahtX2cr1ru68YZKK257_m19RrQ@mail.gmail.com>

Hi Shoibal

On 5 September 2012 14:09, Shoibal Chakravarty <shoibalc at gmail.com> wrote:
> I am still concerned about security implications of the notebook server.
> Ideally, I would like  the following:
>
> 1. Let anybody login with a login/passwd or the various authentication
> services.

IPython Notebook web servers can, and normally would, use password
protection, but this is a single password for that server. You can't
do anything more fine-grained at the moment. It's fundamentally a
single user system, but I know the developers are deeply engaged in
adding support for multiple users in the near future, and this will
obviously need to be secure. It's a major todo, but there's a lot of
considerations, so it'll take a lot of work to get it right.

> 2. Give them a secure walled-in temporary directory to work in, with
> restricted shell function access.

There was some discussion on this list about that kind of thing. I
think Julian Taylor had some good ideas along these lines. There's a
lot of interest in these types of uses, but they all depend on
crippling IPython, which is a tricky thing to do well.

> 3. Limit resources they use (memory, CPU load etc) and automatically log
> them out if the violate this.
> 4. Limit sessions to xx minutes.

I think this would have to be done by the PaaS provider, else users
could just hack the loggers. I'm not sure, but I'd imagine that kind
of feature would necessarily be external to IPython.

Sorry I can't be more help. I expect someone from IPython Dev will be
along soon enough, and be a bit more helpful than I've been. All the
best with this though Shoibal. It's a good area for exploration, and
you can almost always get IPython to do what you want with a little
imagination.

Carl


From jason-sage at creativetrax.com  Wed Sep  5 10:26:26 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 05 Sep 2012 09:26:26 -0500
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <50474EF1.8080104@gmail.com>
References: <50474EF1.8080104@gmail.com>
Message-ID: <50476112.7090105@creativetrax.com>

On 9/5/12 8:09 AM, Shoibal Chakravarty wrote:

> 1. Let anybody login with a login/passwd or the various authentication
> services.
> 2. Give them a secure walled-in temporary directory to work in, with
> restricted shell function access.
> 3. Limit resources they use (memory, CPU load etc) and automatically log
> them out if the violate this.
> 4. Limit sessions to xx minutes.
> 5. Set up a parallel database service for authentication and to let
> users save their notebooks etc. (eg, MongoDB uses json as storage)
>
> Which of these would be easily feasible? Especially point 2. Sage seems
> to create (and later destroy) a temporary working directory for each
> cell  and severely limits shell access. One directory per login with
> limited outside access might be better.

It's certainly a difficult problem, since you are basically giving them 
shell access.  I think it boils down to having a very restricted user 
account, quotas, and making sure there aren't, for example, 
world-writable files anywhere.  All inside of some sort of VM that is 
easy to reset to a known good state.

You're right about Sage's approach.  There are several efforts to make 
things more secure and scalable now, including the Sage Cell Server 
which relies on the IPython infrastructure.  Let me know if you want 
more details.

Thanks,

Jason



From shoibalc at gmail.com  Wed Sep  5 12:04:57 2012
From: shoibalc at gmail.com (Shoibal Chakravarty)
Date: Wed, 05 Sep 2012 12:04:57 -0400
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <CAP-uhDf-AUPMYZUOfs-PCfwhahtX2cr1ru68YZKK257_m19RrQ@mail.gmail.com>
References: <50474EF1.8080104@gmail.com>
	<CAP-uhDf-AUPMYZUOfs-PCfwhahtX2cr1ru68YZKK257_m19RrQ@mail.gmail.com>
Message-ID: <50477829.1070304@gmail.com>

Yes, IPython is currently a single user system. Even if it is possible to add 
login/password or authentication layer on top of the IPython Dashboard (before the singer 
server password) without much effort it doesn't solve the security problem as the almost 
unlimited shell access is still there.

dotCloud runs servers on Amazon and gives me a sort of a 'restricted user account' to host 
apps. In this case, the IPython notebook server is the app. The dotcloud platform uses 
supervisord to run ipython, not sure how much more work should go in to monitor individual 
notebooks and terminate those that violate resource usage. Should be possible, at least in 
theory.

I am not sure how restricted shell access in a walled directory would work. In a way, this 
is the problem that these PaaS companies have actually solved to provide the platform.

-Shoibal.

On 09/05/2012 10:19 AM, Carl Smith wrote:
> Hi Shoibal
>
> On 5 September 2012 14:09, Shoibal Chakravarty <shoibalc at gmail.com> wrote:
>> I am still concerned about security implications of the notebook server.
>> Ideally, I would like  the following:
>>
>> 1. Let anybody login with a login/passwd or the various authentication
>> services.
> IPython Notebook web servers can, and normally would, use password
> protection, but this is a single password for that server. You can't
> do anything more fine-grained at the moment. It's fundamentally a
> single user system, but I know the developers are deeply engaged in
> adding support for multiple users in the near future, and this will
> obviously need to be secure. It's a major todo, but there's a lot of
> considerations, so it'll take a lot of work to get it right.
>
>> 2. Give them a secure walled-in temporary directory to work in, with
>> restricted shell function access.
> There was some discussion on this list about that kind of thing. I
> think Julian Taylor had some good ideas along these lines. There's a
> lot of interest in these types of uses, but they all depend on
> crippling IPython, which is a tricky thing to do well.
>
>> 3. Limit resources they use (memory, CPU load etc) and automatically log
>> them out if the violate this.
>> 4. Limit sessions to xx minutes.
> I think this would have to be done by the PaaS provider, else users
> could just hack the loggers. I'm not sure, but I'd imagine that kind
> of feature would necessarily be external to IPython.
>
> Sorry I can't be more help. I expect someone from IPython Dev will be
> along soon enough, and be a bit more helpful than I've been. All the
> best with this though Shoibal. It's a good area for exploration, and
> you can almost always get IPython to do what you want with a little
> imagination.
>
> Carl
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



From ellisonbg at gmail.com  Wed Sep  5 18:58:22 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 5 Sep 2012 15:58:22 -0700
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <50474EF1.8080104@gmail.com>
References: <50474EF1.8080104@gmail.com>
Message-ID: <CAH4pYpR111aVcdgQo2+kNnta5xszOAtg+ij4DOBfFqO+KJYDXw@mail.gmail.com>

Shoibal,

Hi, these are things we are interested in.  There are a number of
different issues you are bringing up:

* Making the notebook server multiuser aware.  This is definitely on
our radar and we are trying to get this work funded through academic
grants.  We are at the early stages of the planning of this.
* Running the notebook server on PaaS platforms.  This is an entirely
separate issue that we will have to revisit in the future - stay tuned
for more on this in the coming months.  One issue you will run into
though is that we rely on zeromq/pyzmq which definitely doesn't run on
most of the Paas vendors I know of.

Cheers,

Brian


On Wed, Sep 5, 2012 at 6:09 AM, Shoibal Chakravarty <shoibalc at gmail.com> wrote:
> Hi,
>
> I just got interested in the possibility of getting ipython html notebook
> server on the various Platform as a Service (PaaS) ideas out there:
> Openshift, dotcloud, heroku, appfog etc. I was interested in seeing if I
> could get people to try out my other python projects  [energy economics
> modeling] without having to take the trouble of downloading and compiling a
> lot of different packages. I would put everything required on one of these
> free PaaS services and have them try it out via the ipython notebook
> interface.
>
> Carl Smith's  notebookcloud (http://github.com/carlsmith) was another
> inspiration. The PaaS approach is a baby version of  the AMI approach of
> notebookcloud. I let the PaaS vendors take care of the building and
> maintaining the AWS infrastructure. It also happens to be completely free
> for small applications.
>
>
> Most of the Paas services do not support websockets and and often expect
> only wsgi compliant python web servers. Only  dotcloud
> (http://www.dotcloud.com/) seems to work for now with tornado and
> websockets. See http://github.com/shoibalc/ipynbserver-dotcloud.
>
>
> I am still concerned about security implications of the notebook server.
> Ideally, I would like  the following:
>
> 1. Let anybody login with a login/passwd or the various authentication
> services.
> 2. Give them a secure walled-in temporary directory to work in, with
> restricted shell function access.
> 3. Limit resources they use (memory, CPU load etc) and automatically log
> them out if the violate this.
> 4. Limit sessions to xx minutes.
> 5. Set up a parallel database service for authentication and to let users
> save their notebooks etc. (eg, MongoDB uses json as storage)
>
> Which of these would be easily feasible? Especially point 2. Sage seems to
> create (and later destroy) a temporary working directory for each cell  and
> severely limits shell access. One directory per login with limited outside
> access might be better.
>
> Please let me know what ideas/suggestions you have. Thanks a lot for
> ipython!
>
> -Shoibal.
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From carl.input at gmail.com  Wed Sep  5 19:04:04 2012
From: carl.input at gmail.com (Carl Smith)
Date: Thu, 6 Sep 2012 00:04:04 +0100
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <CAH4pYpR111aVcdgQo2+kNnta5xszOAtg+ij4DOBfFqO+KJYDXw@mail.gmail.com>
References: <50474EF1.8080104@gmail.com>
	<CAH4pYpR111aVcdgQo2+kNnta5xszOAtg+ij4DOBfFqO+KJYDXw@mail.gmail.com>
Message-ID: <CAP-uhDdO=p9X6JWmeCQof5yZ9Fsna_vioTHrfxXT9kigFdA8hQ@mail.gmail.com>

You may want to speak to the PaaS provider. A lot of what you'd like to do
comes down to how their platform works. IPython is not your typical web app.
On Sep 5, 2012 11:58 PM, "Brian Granger" <ellisonbg at gmail.com> wrote:

> Shoibal,
>
> Hi, these are things we are interested in.  There are a number of
> different issues you are bringing up:
>
> * Making the notebook server multiuser aware.  This is definitely on
> our radar and we are trying to get this work funded through academic
> grants.  We are at the early stages of the planning of this.
> * Running the notebook server on PaaS platforms.  This is an entirely
> separate issue that we will have to revisit in the future - stay tuned
> for more on this in the coming months.  One issue you will run into
> though is that we rely on zeromq/pyzmq which definitely doesn't run on
> most of the Paas vendors I know of.
>
> Cheers,
>
> Brian
>
>
> On Wed, Sep 5, 2012 at 6:09 AM, Shoibal Chakravarty <shoibalc at gmail.com>
> wrote:
> > Hi,
> >
> > I just got interested in the possibility of getting ipython html notebook
> > server on the various Platform as a Service (PaaS) ideas out there:
> > Openshift, dotcloud, heroku, appfog etc. I was interested in seeing if I
> > could get people to try out my other python projects  [energy economics
> > modeling] without having to take the trouble of downloading and
> compiling a
> > lot of different packages. I would put everything required on one of
> these
> > free PaaS services and have them try it out via the ipython notebook
> > interface.
> >
> > Carl Smith's  notebookcloud (http://github.com/carlsmith) was another
> > inspiration. The PaaS approach is a baby version of  the AMI approach of
> > notebookcloud. I let the PaaS vendors take care of the building and
> > maintaining the AWS infrastructure. It also happens to be completely free
> > for small applications.
> >
> >
> > Most of the Paas services do not support websockets and and often expect
> > only wsgi compliant python web servers. Only  dotcloud
> > (http://www.dotcloud.com/) seems to work for now with tornado and
> > websockets. See http://github.com/shoibalc/ipynbserver-dotcloud.
> >
> >
> > I am still concerned about security implications of the notebook server.
> > Ideally, I would like  the following:
> >
> > 1. Let anybody login with a login/passwd or the various authentication
> > services.
> > 2. Give them a secure walled-in temporary directory to work in, with
> > restricted shell function access.
> > 3. Limit resources they use (memory, CPU load etc) and automatically log
> > them out if the violate this.
> > 4. Limit sessions to xx minutes.
> > 5. Set up a parallel database service for authentication and to let users
> > save their notebooks etc. (eg, MongoDB uses json as storage)
> >
> > Which of these would be easily feasible? Especially point 2. Sage seems
> to
> > create (and later destroy) a temporary working directory for each cell
>  and
> > severely limits shell access. One directory per login with limited
> outside
> > access might be better.
> >
> > Please let me know what ideas/suggestions you have. Thanks a lot for
> > ipython!
> >
> > -Shoibal.
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/80cdd66d/attachment.html>

From shoibalc at gmail.com  Wed Sep  5 20:10:10 2012
From: shoibalc at gmail.com (Shoibal Chakravarty)
Date: Wed, 05 Sep 2012 20:10:10 -0400
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <CAH4pYpR111aVcdgQo2+kNnta5xszOAtg+ij4DOBfFqO+KJYDXw@mail.gmail.com>
References: <50474EF1.8080104@gmail.com>
	<CAH4pYpR111aVcdgQo2+kNnta5xszOAtg+ij4DOBfFqO+KJYDXw@mail.gmail.com>
Message-ID: <5047E9E2.70905@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120905/1d519a3b/attachment.html>

From fperez.net at gmail.com  Wed Sep  5 22:32:41 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 5 Sep 2012 19:32:41 -0700
Subject: [IPython-dev] R magic extension and default R graphic drivers
In-Reply-To: <CAHAreOoTkZpVqUOSuhg5EaRDKBzz8yGzAn=05zYYEwY4FLPnsA@mail.gmail.com>
References: <249CAC27-8648-40DF-9277-CB2DF8432602@gmail.com>
	<26A7A009-FDD5-4F81-ADA7-FC0FB82BC0C8@gmail.com>
	<CAHAreOoTkZpVqUOSuhg5EaRDKBzz8yGzAn=05zYYEwY4FLPnsA@mail.gmail.com>
Message-ID: <CAHAreOqvL7xXGP8TMm7+sfSW8F+tr-6_sXsphr=NvCdgDswbOA@mail.gmail.com>

On Wed, Sep 5, 2012 at 7:18 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Unfortunately I'm not an R expert, I don't know.  I've pinged the
> author of that code in case he has some wisdom he can share with us.

Never mind, as I was about to hit send, I double-checked and realized
that Jonathan is indeed subscribed to this list. So there's no need
for a separate ping, hopefully he'll be able to give you some
pointers.

Cheers,

f


From fperez.net at gmail.com  Wed Sep  5 22:18:24 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 5 Sep 2012 19:18:24 -0700
Subject: [IPython-dev] R magic extension and default R graphic drivers
In-Reply-To: <26A7A009-FDD5-4F81-ADA7-FC0FB82BC0C8@gmail.com>
References: <249CAC27-8648-40DF-9277-CB2DF8432602@gmail.com>
	<26A7A009-FDD5-4F81-ADA7-FC0FB82BC0C8@gmail.com>
Message-ID: <CAHAreOoTkZpVqUOSuhg5EaRDKBzz8yGzAn=05zYYEwY4FLPnsA@mail.gmail.com>

On Wed, Sep 5, 2012 at 5:28 AM, Massimo Di Stefano
<massimodisasha at gmail.com> wrote:
> i'm tring to modify withput lucky the src code in the rmagic extension
> in order to use the "type=cairo" as argument to the png() command.
>
> png(file="myplot.png", bg="transparent", type="cairo'")
>
>
> please have you any hints ?

Unfortunately I'm not an R expert, I don't know.  I've pinged the
author of that code in case he has some wisdom he can share with us.

Cheers,

f


From fperez.net at gmail.com  Wed Sep  5 22:46:49 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 5 Sep 2012 19:46:49 -0700
Subject: [IPython-dev] Installation of ipython on cygwin
In-Reply-To: <1346390393.13841.0.camel@bomberx.somewhere-in-the-space.org>
References: <1346390393.13841.0.camel@bomberx.somewhere-in-the-space.org>
Message-ID: <CAHAreOoJH+BRVvX5raeKpxWy3DoZA6aO0zHa-LiLY5+OR=212Q@mail.gmail.com>

On Thu, Aug 30, 2012 at 10:19 PM, Arnaud Gardelein <arnaud at oscopy.org> wrote:
> It seems that ipython got installed fine on my cygwin install. Just in
> case if it could help anybody else here is the approximate procedure that
> was followed:

Thanks for the info!  It's worth noting that those who don't need
matplotlib should be fine with simply installing on top of their base
cygwin python, the libffi requirement should be only for matplotlib.

cheers,

f


From shoibalc at gmail.com  Thu Sep  6 00:07:24 2012
From: shoibalc at gmail.com (Shoibal Chakravarty)
Date: Thu, 06 Sep 2012 00:07:24 -0400
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <50476112.7090105@creativetrax.com>
References: <50474EF1.8080104@gmail.com> <50476112.7090105@creativetrax.com>
Message-ID: <5048217C.1090009@gmail.com>

Hi Jason,

I'd definitely be interested in learning more about sagecell. Is there a design document 
that I could look at?

Thanks,
Shoibal.

On 09/05/2012 10:26 AM, Jason Grout wrote:
> On 9/5/12 8:09 AM, Shoibal Chakravarty wrote:
>
>> 1. Let anybody login with a login/passwd or the various authentication
>> services.
>> 2. Give them a secure walled-in temporary directory to work in, with
>> restricted shell function access.
>> 3. Limit resources they use (memory, CPU load etc) and automatically log
>> them out if the violate this.
>> 4. Limit sessions to xx minutes.
>> 5. Set up a parallel database service for authentication and to let
>> users save their notebooks etc. (eg, MongoDB uses json as storage)
>>
>> Which of these would be easily feasible? Especially point 2. Sage seems
>> to create (and later destroy) a temporary working directory for each
>> cell  and severely limits shell access. One directory per login with
>> limited outside access might be better.
>
> It's certainly a difficult problem, since you are basically giving them shell access.  I 
> think it boils down to having a very restricted user account, quotas, and making sure 
> there aren't, for example, world-writable files anywhere.  All inside of some sort of VM 
> that is easy to reset to a known good state.
>
> You're right about Sage's approach.  There are several efforts to make things more 
> secure and scalable now, including the Sage Cell Server which relies on the IPython 
> infrastructure.  Let me know if you want more details.
>
> Thanks,
>
> Jason
>
>



From jason-sage at creativetrax.com  Thu Sep  6 00:19:51 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 05 Sep 2012 23:19:51 -0500
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <5048217C.1090009@gmail.com>
References: <50474EF1.8080104@gmail.com> <50476112.7090105@creativetrax.com>
	<5048217C.1090009@gmail.com>
Message-ID: <50482467.4010505@creativetrax.com>

On 9/5/12 11:07 PM, Shoibal Chakravarty wrote:
> Hi Jason,
>
> I'd definitely be interested in learning more about sagecell. Is there a design document
> that I could look at?

We're working on one.  The code base is in the contrib/ipython-testing 
directory: 
https://github.com/sagemath/sagecell/tree/master/contrib/ipython-testing

Very briefly: The web_server.py file is the main program that is run. 
The webserver has a trusted multikernel manager that talks to an 
untrusted multikernel manager (typically another account or another 
computer) over zmq.  The untrusted multikernel manager receives a 
request to start up a session and uses the forking kernel manager to 
create a session by forking and hook up the zmq channels to the webserver.

William Stein is also working on a new design and architecture for 
scalable and secure scientific computing like the notebook which looks 
very promising.  That's one of his top priorities over the next 6-8 
months or so.

Thanks,

Jason



From stefan at sun.ac.za  Thu Sep  6 04:57:21 2012
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 6 Sep 2012 01:57:21 -0700
Subject: [IPython-dev] nbviewer not working for all URLs?
Message-ID: <CABDkGQk6tLVt6onnXjNBjJRQLGMwhNP72uO=0+4wx1569SFYGA@mail.gmail.com>

Hi everyone

I recently discovered nbviewer.ipython.org, with which I'm trying to view

https://raw.github.com/jbpoline/nipy-notebooks/master/principles_statistics.ipynb

Unfortunately, that fails.  Is nbviewer part of the main IPython
project or, if not, whom can I contact to resolve the problem?

St?fan


From takowl at gmail.com  Thu Sep  6 06:57:37 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Sep 2012 11:57:37 +0100
Subject: [IPython-dev] nbviewer not working for all URLs?
In-Reply-To: <CABDkGQk6tLVt6onnXjNBjJRQLGMwhNP72uO=0+4wx1569SFYGA@mail.gmail.com>
References: <CABDkGQk6tLVt6onnXjNBjJRQLGMwhNP72uO=0+4wx1569SFYGA@mail.gmail.com>
Message-ID: <CAOvn4qjg5_2yO_kcxJ4mVsVf7r123zVy5SF1RvQewWUph8Otqw@mail.gmail.com>

On 6 September 2012 09:57, St?fan van der Walt <stefan at sun.ac.za> wrote:
> Unfortunately, that fails.  Is nbviewer part of the main IPython
> project or, if not, whom can I contact to resolve the problem?

There's a repository here that you can file an issue against:

https://github.com/ipython/nbviewer

I'm a bit mystified by the failure - I used it just yesterday with a
Github raw view URL without any problem.

Thomas


From shoibalc at gmail.com  Thu Sep  6 16:34:03 2012
From: shoibalc at gmail.com (Shoibal Chakravarty)
Date: Thu, 06 Sep 2012 16:34:03 -0400
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <50482467.4010505@creativetrax.com>
References: <50474EF1.8080104@gmail.com> <50476112.7090105@creativetrax.com>
	<5048217C.1090009@gmail.com> <50482467.4010505@creativetrax.com>
Message-ID: <504908BB.9060905@gmail.com>

Hi Jason,

I decided to try to get sage cell server work without any sage dependencies ( ie,  I 
removed jmol).

Strangely, I get an output in the bottom cell only when ipython returns an error but not 
otherwise. Are stdout and stderr handled differently?  Does this have to do with the 
version of ipython and other packages used in sage vs the most recent ones.

Thanks,
Shoibal.


On 09/06/2012 12:19 AM, Jason Grout wrote:
> On 9/5/12 11:07 PM, Shoibal Chakravarty wrote:
>> Hi Jason,
>>
>> I'd definitely be interested in learning more about sagecell. Is there a design document
>> that I could look at?
> We're working on one.  The code base is in the contrib/ipython-testing
> directory:
> https://github.com/sagemath/sagecell/tree/master/contrib/ipython-testing
>
> Very briefly: The web_server.py file is the main program that is run.
> The webserver has a trusted multikernel manager that talks to an
> untrusted multikernel manager (typically another account or another
> computer) over zmq.  The untrusted multikernel manager receives a
> request to start up a session and uses the forking kernel manager to
> create a session by forking and hook up the zmq channels to the webserver.
>
> William Stein is also working on a new design and architecture for
> scalable and secure scientific computing like the notebook which looks
> very promising.  That's one of his top priorities over the next 6-8
> months or so.
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



From takowl at gmail.com  Thu Sep  6 16:34:37 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Sep 2012 21:34:37 +0100
Subject: [IPython-dev] IPython In-process kernel support
Message-ID: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>

Hi Pierre,

There's a pull request open to allow the IPython Qt console to use an
in-process kernel. I thought you might be interested in using something
like this in Spyder. If you'd like to take a look, we'd welcome any
feedback:

https://github.com/ipython/ipython/pull/2382

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/abb7054a/attachment.html>

From jason-sage at creativetrax.com  Thu Sep  6 16:37:36 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 06 Sep 2012 15:37:36 -0500
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <504908BB.9060905@gmail.com>
References: <50474EF1.8080104@gmail.com> <50476112.7090105@creativetrax.com>
	<5048217C.1090009@gmail.com> <50482467.4010505@creativetrax.com>
	<504908BB.9060905@gmail.com>
Message-ID: <50490990.3020707@creativetrax.com>

On 9/6/12 3:34 PM, Shoibal Chakravarty wrote:
> Hi Jason,
>
> I decided to try to get sage cell server work without any sage dependencies ( ie,  I
> removed jmol).
>
> Strangely, I get an output in the bottom cell only when ipython returns an error but not
> otherwise. Are stdout and stderr handled differently?  Does this have to do with the
> version of ipython and other packages used in sage vs the most recent ones.
>

I have no idea; I'm not sure exactly what you were trying and what 
codebase you were working from.  Can you describe exactly what you tried?

Thanks,

Jason



From jason-sage at creativetrax.com  Thu Sep  6 16:43:15 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 06 Sep 2012 15:43:15 -0500
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <504908BB.9060905@gmail.com>
References: <50474EF1.8080104@gmail.com> <50476112.7090105@creativetrax.com>
	<5048217C.1090009@gmail.com> <50482467.4010505@creativetrax.com>
	<504908BB.9060905@gmail.com>
Message-ID: <50490AE3.4020003@creativetrax.com>

On 9/6/12 3:34 PM, Shoibal Chakravarty wrote:
> Are stdout and stderr handled differently?

Differently than what?  We use the standard IPython stdout and stderr 
messages to return that information.

As for versions, the current cell server requires several packages that 
don't come with Sage right now.  See 
https://github.com/sagemath/sagecell/issues/344 for some instructions 
that I think are still current.

We do assume Sage, but the actual dependencies are pretty small and can 
be deleted.  Make sure you are working with the codebase in 
contrib/ipython-testing/

Thanks,

Jason



From shoibalc at gmail.com  Thu Sep  6 17:01:35 2012
From: shoibalc at gmail.com (Shoibal Chakravarty)
Date: Thu, 06 Sep 2012 17:01:35 -0400
Subject: [IPython-dev] ipython html notebook server on dotcloud
In-Reply-To: <50490990.3020707@creativetrax.com>
References: <50474EF1.8080104@gmail.com> <50476112.7090105@creativetrax.com>
	<5048217C.1090009@gmail.com> <50482467.4010505@creativetrax.com>
	<504908BB.9060905@gmail.com> <50490990.3020707@creativetrax.com>
Message-ID: <50490F2F.2060105@gmail.com>

I got web_server.py in /sagecell/contrib/ipython-testing to run without any exceptions 
using the usual python on my system (2.7.3) and required packages from pypi.


For example:

$ python web_server.py  -p 8999
ZMQ Connection with computer d8da68f4-57e2-42e8-976a-8e255cd6d646 at port 62165 established.


A proper input in the Sage cell doesn't produce an output but anything that triggers an 
exception does.



The dependencies needed in a virtualenv are:

$ pip freeze

SQLAlchemy==0.7.8
argparse==1.2.1
gevent==0.13.8
greenlet==0.4.0
ipython==0.13
nose==1.1.2
paramiko==1.7.7.2
pycrypto==2.6
pyzmq==2.2.0.1
-e 
git+https://github.com/mrjoes/sockjs-tornado.git at 70fba807ad934ca1e0cbaa82c0ab98930442ed96#egg=sockjs_tornado-dev
tornado==2.4
websocket==0.2.1
wsgiref==0.1.2









On 09/06/2012 04:37 PM, Jason Grout wrote:
> On 9/6/12 3:34 PM, Shoibal Chakravarty wrote:
>> Hi Jason,
>>
>> I decided to try to get sage cell server work without any sage dependencies ( ie,  I
>> removed jmol).
>>
>> Strangely, I get an output in the bottom cell only when ipython returns an error but not
>> otherwise. Are stdout and stderr handled differently?  Does this have to do with the
>> version of ipython and other packages used in sage vs the most recent ones.
>>
> I have no idea; I'm not sure exactly what you were trying and what
> codebase you were working from.  Can you describe exactly what you tried?
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



From epatters at enthought.com  Thu Sep  6 17:14:42 2012
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 6 Sep 2012 16:14:42 -0500
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
Message-ID: <CAFAPEOXowp=vTooCfrnS=0+fP8Z8QHjOeznWG+pW78V+xSBhnQ@mail.gmail.com>

Thanks for the shout out, Thomas.

I'm now at the point where some thorough code review would be useful.
Although more testing is definitely required, all the most important
functionality is working. The only known omission is raw_input for the
terminal frontend, but using an in-process kernel with a terminal frontend
seems a bit perverse, to say the least.

Evan

On Thu, Sep 6, 2012 at 3:34 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Hi Pierre,
>
> There's a pull request open to allow the IPython Qt console to use an
> in-process kernel. I thought you might be interested in using something
> like this in Spyder. If you'd like to take a look, we'd welcome any
> feedback:
>
> https://github.com/ipython/ipython/pull/2382
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/9600cf21/attachment.html>

From takowl at gmail.com  Thu Sep  6 17:20:04 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Sep 2012 22:20:04 +0100
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAFAPEOXowp=vTooCfrnS=0+fP8Z8QHjOeznWG+pW78V+xSBhnQ@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
	<CAFAPEOXowp=vTooCfrnS=0+fP8Z8QHjOeznWG+pW78V+xSBhnQ@mail.gmail.com>
Message-ID: <CAOvn4qh5a8eccby6Q=pWZYqjF0EaW-DmNZ5KYN7k=-hSVkaRJQ@mail.gmail.com>

Is there any major obstacle to implementing raw_input in the terminal
frontend, or do you think it's just not worth it?


From epatters at enthought.com  Thu Sep  6 17:39:36 2012
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 6 Sep 2012 16:39:36 -0500
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAOvn4qh5a8eccby6Q=pWZYqjF0EaW-DmNZ5KYN7k=-hSVkaRJQ@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
	<CAFAPEOXowp=vTooCfrnS=0+fP8Z8QHjOeznWG+pW78V+xSBhnQ@mail.gmail.com>
	<CAOvn4qh5a8eccby6Q=pWZYqjF0EaW-DmNZ5KYN7k=-hSVkaRJQ@mail.gmail.com>
Message-ID: <CAFAPEOX-a5pbCVw2+FejJdPPH5VmZUNHsb4q9DrmqAT96NQdrw@mail.gmail.com>

There is no real obstacle, but it isn't very convenient. The trouble is
that the terminal frontend is truly synchronous, unlike a GUI frontend
which is driven by an event loop. Here is how the terminal frontend
executes code:

https://github.com/ipython/ipython/blob/master/IPython/frontend/terminal/console/interactiveshell.py#L94

You can see that 'handle_stdin_request' will never be called. To handle
stdin, something must be special cased: the terminal frontend, the embedded
kernel, or the embedded kernel manager. I'm not sure how to handle this
cleanly. Perhaps the simplest thing is to add a switch to the kernel for
calling 'raw_input' directly (bypassing the kernel manager). Only terminal
frontends would use this switch. Perhaps others have better ideas.

Evan

On Thu, Sep 6, 2012 at 4:20 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Is there any major obstacle to implementing raw_input in the terminal
> frontend, or do you think it's just not worth it?
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/cbdff2d5/attachment.html>

From ccordoba12 at gmail.com  Thu Sep  6 17:42:19 2012
From: ccordoba12 at gmail.com (=?UTF-8?B?Q2FybG9zIEPDs3Jkb2Jh?=)
Date: Thu, 06 Sep 2012 16:42:19 -0500
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
Message-ID: <504918BB.6030105@gmail.com>

Hi,

I'm the Spyder dev who has contributed most of the code to integrate it 
with IPython. I already left some comments on the pull request, with 
simple doubts about how we can use this new code. It's really promising, 
thanks Evan for taking the time to implement it.

About raw_input, I think Robert Kern (in his original gist) caught it 
and transformed to a QInputDialog. It would be nice to have something 
like that before merging.

Cheers,
Carlos

El 06/09/12 15:34, Thomas Kluyver escribi?:
> Hi Pierre,
>
> There's a pull request open to allow the IPython Qt console to use an 
> in-process kernel. I thought you might be interested in using 
> something like this in Spyder. If you'd like to take a look, we'd 
> welcome any feedback:
>
> https://github.com/ipython/ipython/pull/2382
>
> Thanks,
> Thomas
>
>
> This body part will be downloaded on demand.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/e8a3af5a/attachment.html>

From takowl at gmail.com  Thu Sep  6 18:08:29 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 6 Sep 2012 23:08:29 +0100
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAFAPEOX-a5pbCVw2+FejJdPPH5VmZUNHsb4q9DrmqAT96NQdrw@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
	<CAFAPEOXowp=vTooCfrnS=0+fP8Z8QHjOeznWG+pW78V+xSBhnQ@mail.gmail.com>
	<CAOvn4qh5a8eccby6Q=pWZYqjF0EaW-DmNZ5KYN7k=-hSVkaRJQ@mail.gmail.com>
	<CAFAPEOX-a5pbCVw2+FejJdPPH5VmZUNHsb4q9DrmqAT96NQdrw@mail.gmail.com>
Message-ID: <CAOvn4qikdfdo2xZV+cDY7dVmufHiymaic8E09STiJT9g7EK0Aw@mail.gmail.com>

On 6 September 2012 22:39, Evan Patterson <epatters at enthought.com> wrote:
> You can see that 'handle_stdin_request' will never be called. To handle
> stdin, something must be special cased: the terminal frontend, the embedded
> kernel, or the embedded kernel manager. I'm not sure how to handle this
> cleanly. Perhaps the simplest thing is to add a switch to the kernel for
> calling 'raw_input' directly (bypassing the kernel manager). Only terminal
> frontends would use this switch. Perhaps others have better ideas.

OK, that makes sense. My inclination would be to leave it until
someone finds they actually need it - as you say, running in the
terminal with a local kernel seems a bit odd. If it did become
necessary, another possible route would be to run the kernel in its
own thread, so a degree of asynchronous execution was possible.
Although I'm sure any solution with the word 'thread' in has its share
of problems.

> About raw_input, I think Robert Kern (in his original gist) caught it and transformed to a QInputDialog. It would be nice to have something like that before merging.

If I understand Evan correctly, the problem with raw_input is only
when using this in the terminal frontend. In the Qt console, I assume
it behaves like a remote kernel - the prompt appears inline. Evan, can
you confirm that?

Thanks,
Thomas


From epatters at enthought.com  Thu Sep  6 18:12:24 2012
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 6 Sep 2012 17:12:24 -0500
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAOvn4qikdfdo2xZV+cDY7dVmufHiymaic8E09STiJT9g7EK0Aw@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
	<CAFAPEOXowp=vTooCfrnS=0+fP8Z8QHjOeznWG+pW78V+xSBhnQ@mail.gmail.com>
	<CAOvn4qh5a8eccby6Q=pWZYqjF0EaW-DmNZ5KYN7k=-hSVkaRJQ@mail.gmail.com>
	<CAFAPEOX-a5pbCVw2+FejJdPPH5VmZUNHsb4q9DrmqAT96NQdrw@mail.gmail.com>
	<CAOvn4qikdfdo2xZV+cDY7dVmufHiymaic8E09STiJT9g7EK0Aw@mail.gmail.com>
Message-ID: <CAFAPEOX_LERHDjwiXizUQDzL+Ue4EDgAc2p+dF8vaGrLTsL8OA@mail.gmail.com>

On Thu, Sep 6, 2012 at 5:08 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 6 September 2012 22:39, Evan Patterson <epatters at enthought.com> wrote:
> > You can see that 'handle_stdin_request' will never be called. To handle
> > stdin, something must be special cased: the terminal frontend, the
> embedded
> > kernel, or the embedded kernel manager. I'm not sure how to handle this
> > cleanly. Perhaps the simplest thing is to add a switch to the kernel for
> > calling 'raw_input' directly (bypassing the kernel manager). Only
> terminal
> > frontends would use this switch. Perhaps others have better ideas.
>
> OK, that makes sense. My inclination would be to leave it until
> someone finds they actually need it - as you say, running in the
> terminal with a local kernel seems a bit odd. If it did become
> necessary, another possible route would be to run the kernel in its
> own thread, so a degree of asynchronous execution was possible.
> Although I'm sure any solution with the word 'thread' in has its share
> of problems.
>
> > About raw_input, I think Robert Kern (in his original gist) caught it
> and transformed to a QInputDialog. It would be nice to have something like
> that before merging.
>
> If I understand Evan correctly, the problem with raw_input is only
> when using this in the terminal frontend. In the Qt console, I assume
> it behaves like a remote kernel - the prompt appears inline. Evan, can
> you confirm that?
>

Yes, that's exactly how it works.

Evan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/0afa4a04/attachment.html>

From bedwards at cs.unm.edu  Thu Sep  6 18:50:20 2012
From: bedwards at cs.unm.edu (Ben Edwards)
Date: Thu, 6 Sep 2012 16:50:20 -0600
Subject: [IPython-dev] (no subject)
Message-ID: <CAP4Ca8WfTpwKQxvWKAwgq4tdPzNP6t9BAhKddaxTUB4R+FZZCQ@mail.gmail.com>

Sorry if this is posted twice, sent it from the wrong email...

For the last week or so I've been working on Mathematica and MATLAB
extensions for the IPython notebook. I've been shamelessly copying and
modifying code from the octave extension. The Mathematica one is in pretty
good working order, you can check it out at (and test it please!):

https://github.com/bjedwards/IPython-mathematicamagic-extension

I know there doesn't need to be an extension for everything under the sun,
but this seems like it might be useful to include. Unless there is some
reason to not include extensions to non-free software, which is probably
reasonable. Could this possibly be included in IPython, or should it stay
an external extension? If it is to stay external I'll perhaps announce it
on the user list.

Ben Edwards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120906/71acde72/attachment.html>

From takowl at gmail.com  Thu Sep  6 19:03:19 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 7 Sep 2012 00:03:19 +0100
Subject: [IPython-dev] (no subject)
In-Reply-To: <CAP4Ca8WfTpwKQxvWKAwgq4tdPzNP6t9BAhKddaxTUB4R+FZZCQ@mail.gmail.com>
References: <CAP4Ca8WfTpwKQxvWKAwgq4tdPzNP6t9BAhKddaxTUB4R+FZZCQ@mail.gmail.com>
Message-ID: <CAOvn4qg_OT8ziNa9D_5MreNqtsCPF=YuT+dXQ4eFC6ekJMrdbw@mail.gmail.com>

Hi Ben,

On 6 September 2012 23:50, Ben Edwards <bedwards at cs.unm.edu> wrote:
> I know there doesn't need to be an extension for everything under the sun,
> but this seems like it might be useful to include. Unless there is some
> reason to not include extensions to non-free software, which is probably
> reasonable. Could this possibly be included in IPython, or should it stay an
> external extension? If it is to stay external I'll perhaps announce it on
> the user list.

I don't have a major objection to interfacing with proprietary
software, but I think it makes sense for most extensions to be
externally maintained. The idea of an extension system is that you can
make things like this without having to get them included in IPython.
I'd encourage you to add it to the extension index on the wiki, as
well as making an announcement:
http://wiki.ipython.org/Extensions_Index

Also, we can easily include an extension later if we realise it would
be more valuable like that, but dropping a default extension is more
disruptive.

Best wishes,
Thomas


From takowl at gmail.com  Thu Sep  6 19:21:36 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 7 Sep 2012 00:21:36 +0100
Subject: [IPython-dev] Windows build service
Message-ID: <CAOvn4qh+ta=F92H7KVgJMLnN+truKDzupJ-nF2BtXoxmGVt7ew@mail.gmail.com>

Hi all,

At my request, ShiningPanda have activated a Windows build environment
for our project, and even thrown in $5 of free build credits to
experiment with it.

After some fiddling, tests on Python 2.7 are passing:
https://jenkins.shiningpanda.com/ipython/job/ipython-win-py27/

This is currently set up to run at a rather conservative twice per
week. But any of the core devs* can trigger an immediate run, and we
might want to increase the rate near release time.

I also tried to get a Python 3.2 job running, but at present a bug in
distribute (https://bitbucket.org/tarek/distribute/issue/284/get_exe_prefixes-passes-bytes-to
) prevents us installing pywin32 inside the virtualenv. If anyone more
Windows-savvy knows a workaround, your help would be appreciated!
Otherwise, hopefully we'll get a fix in place before long.

Best wishes,
Thomas

* Core devs: if you don't have access to our project on ShiningPanda
and you want it, let me know.


From fperez.net at gmail.com  Fri Sep  7 00:17:54 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 6 Sep 2012 21:17:54 -0700
Subject: [IPython-dev] (no subject)
In-Reply-To: <CAOvn4qg_OT8ziNa9D_5MreNqtsCPF=YuT+dXQ4eFC6ekJMrdbw@mail.gmail.com>
References: <CAP4Ca8WfTpwKQxvWKAwgq4tdPzNP6t9BAhKddaxTUB4R+FZZCQ@mail.gmail.com>
	<CAOvn4qg_OT8ziNa9D_5MreNqtsCPF=YuT+dXQ4eFC6ekJMrdbw@mail.gmail.com>
Message-ID: <CAHAreOrA_kWKGasJuFxHZyLNzCJJ+FrK1cX-rjKX9vxffvAc1w@mail.gmail.com>

On Thu, Sep 6, 2012 at 4:03 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> Also, we can easily include an extension later if we realise it would
> be more valuable like that, but dropping a default extension is more
> disruptive.

Pretty much agreeing with everything Thomas said :)  And thanks for
creating the Mathematica one, that could be very useful to many (and
would have been to me a few years ago).

As long as *your* code is properly licensed, there's no problem in
having IPython extensions that interface with proprietary tools,
especially high-value/interest ones such as Mathematica and Matlab.

I agree that we should probably start with having it independently
maintained at least for now, in particular because putting it in the
core creates a testing problem for us: we can all have machines
configured with cython, R and octave to run a full test suite, but we
don't necessarily have Mathematica or Matlab available for testing. As
long as you maintain it, you can ensure that it remains in working
condition.

But we're very happy to see this contribution, by all means add it to
the extensions index!

Cheers,

f


From fperez.net at gmail.com  Fri Sep  7 04:20:49 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 7 Sep 2012 01:20:49 -0700
Subject: [IPython-dev] Bitey magic extension
In-Reply-To: <CAOvn4qgPjv8O=3PFXktgr3OXvWHraq58NeS2Ch66R5kPoMWLew@mail.gmail.com>
References: <CAHXv-MgonNWU5qKH9kaDWde1GLsJ4a-LRjXLE9B52orXNoC=PQ@mail.gmail.com>
	<CAOvn4qgPjv8O=3PFXktgr3OXvWHraq58NeS2Ch66R5kPoMWLew@mail.gmail.com>
Message-ID: <CAHAreOqD16BWTbHB=XLJJStEufP9OWcZ=6hkVMgmTGz3MeVsag@mail.gmail.com>

On Sat, Aug 25, 2012 at 4:00 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Very neat! We've just had a talk from David Beazley (the author of Bitey),
> so it's well timed.

Indeed!  BTW, this should probably go into the extension index on the wiki...


From fperez.net at gmail.com  Fri Sep  7 04:26:48 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 7 Sep 2012 01:26:48 -0700
Subject: [IPython-dev] Blogging from the notebook
Message-ID: <CAHAreOpqXtYr7sw_5cLWXd+nfSbUgNXcTM3G0rypMci3YX1XmQ@mail.gmail.com>

Hi folks,

I had to answer a question from a colleague today about how to use the
notebook to blog, and I ended up spending a bit more time on it than I
should (the fact that I had to write a grant helped, I'm sure...
Matthew can shoot me now).  But I suspect others will find this
useful:

http://blog.fperez.org/2012/09/blogging-with-ipython-notebook.html

Kudos to everyone who has worked on nbconvert to improve it and make
this possible.  We should be able to streamline this further without
too much difficulty, but even now, once configured the amount of
manual labor involved is really minimal.

Cheers,

f


From fperez.net at gmail.com  Fri Sep  7 04:38:55 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 7 Sep 2012 01:38:55 -0700
Subject: [IPython-dev] John Hunter's memorial service: Oct 1, 2012
Message-ID: <CAHAreOp0Qe5jYNAKXVT1-iAE-z_Z0OJRC6b4pYuSbg7WcMqTcw@mail.gmail.com>

Hi all,

I have just received the following information from John's family
regarding the memorial service:

John's memorial service will be held on Monday, October 1, 2012, at
11.a.m. at Rockefeller Chapel at the University of Chicago.  The exact
address is 5850 S. Woodlawn Ave, Chicago, IL 60615.  The service is
open to the public.  The service will be fully planned and scripted
with no room for people to eulogize, however, we will have a reception
after the service, hosted by Tradelink, where people can talk.

Regards,

f


From ellisonbg at gmail.com  Fri Sep  7 14:20:22 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 7 Sep 2012 11:20:22 -0700
Subject: [IPython-dev] Moving all dev docs and IPEPs to github
In-Reply-To: <CAHAreOo3UzCSUG2f3qNZEAvW2EipZwm7wLvSaJF3FF5Oj=JMjQ@mail.gmail.com>
References: <CAH4pYpRth8wufuQ5Vj1o83QLdHD-EnWV_HXyN1xrGDzaU2pfAA@mail.gmail.com>
	<CAHAreOo3UzCSUG2f3qNZEAvW2EipZwm7wLvSaJF3FF5Oj=JMjQ@mail.gmail.com>
Message-ID: <CAH4pYpSwi2YVY+GNDuuLyb7xs13SoTiU1CroN4P=q42vWEL_VQ@mail.gmail.com>

On Mon, Aug 13, 2012 at 12:04 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Sun, Aug 12, 2012 at 10:02 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> I propose that we transition all of our dev docs (those focused on
>> developing IPython itself) to the github IPython wiki.
>
> I think that's sensible, but only if we also move the existing wiki
> (we've talked about that in the past, with several people in favor,
> and Thomas graciously indicating that despite his time being the
> largest 'sunk cost' here he wouldn't object either).  I don't mind the
> current wiki as much as others, but what I do *not* want is to have
> two wikis again.

In my mind, a user focused wiki and a dev focused wiki could be
maintained separately.  I really want to keep all of our dev related
things on github as it makes our dev process much more discoverable.
But I do agree that having two wikis is a bit silly.

> So if we're going to activate the github wiki, then that should be our
> *only* wiki.  That means we'll need to migrate both the current wiki
> and the current dev docs to it.
>
> I'm sort of +0.5 on the idea, and will happily go along if other devs
> feel it will really make things better for them.

I think that moving everything to github is the best way to have all
our dev related docs in one place rather than 3 or 4.  I am definitely
+1 on the idea.

>> I also propose that we host all IPEPs at this same location for
>> everyone to edit and view in one place (rather than individual gists).
>
> Yes, my putting the first IPEP on a gist was just a quick experiment.
> At the time, I was thinking we could make a separate repo for IPEPS.
> But since the wiki *is* a repo, it would satisfy the criteria I had in
> mind at the time (easier access to all IPEPs in one place, easy
> updates for anyone who proposes one, ...).

Yes, I think we are closing in on a good approach for the IPEPs =
github wiki + ipython-dev

>> I also propose that we start to think about doing some more deliberate
>> planning for the project (like a roadmap) and that we keep that on the
>> wiki.

Yes, I agree we need something like a roadmap in addition to the IPEPs.

> Yes, this is something that we've been hinting and where we
> occasionally drop half-discussions on the list, but having a bit of
> organization is starting to be necessary.  I don't believe in overly
> detailed roadmaps, because the nature of open source work is a highly
> organic, evolutionary process driven by a combination of individual
> interests and the PR review process acting as selective pressure.  So
> I think there's little point in trying to 'dictate' via a roadmap what
> the project needs to be...

Yes, our roadmap needs to be informal, flexible, non-verbose and
dynamic.  I would like something that people feel good about updating
often with small bits of information that is useful to all.  I don't
want something that takes hours to update or is otherwise a burden to
us.  I think that having most of our dev process managed by issues and
PRs works great and I don't want to mess with that.  It is just a
little bit of the bigger picture stuff that we need a bit more
organization on.  And honestly, the IPEPs will help with that as well.

> But where I think there's a lot of value, and that we have a genuine
> need, is in helping us all understand what the 'big ideas' are, what
> the main lines of work for the project are and if anyone in particular
> is already tackling one or not, so that people can coordinate a bit.
> So I think that this roadmap would mostly be in a sense a place to
> suggest directions for work that may need an IPEP, or where one or
> more people may want to coordinate efforts.
>
> With that loose definition of roadmap, I'm all for it and I think it
> will help us all work better...

Great, we just need to make the final decision on the wiki
consolidation and we can start to move content over.

Cheers,

Brian

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Fri Sep  7 14:21:20 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 7 Sep 2012 11:21:20 -0700
Subject: [IPython-dev] John Hunter's memorial service: Oct 1, 2012
In-Reply-To: <CAHAreOp0Qe5jYNAKXVT1-iAE-z_Z0OJRC6b4pYuSbg7WcMqTcw@mail.gmail.com>
References: <CAHAreOp0Qe5jYNAKXVT1-iAE-z_Z0OJRC6b4pYuSbg7WcMqTcw@mail.gmail.com>
Message-ID: <CAHAreOrO-5+-d-5Pjkw1Rt2g_kTZ_96TQjCiwumeZ0ZYGLyVhw@mail.gmail.com>

I just received the official announcement, please note the RSVP requirement
to Miriam at msierig at gmail.com.




John Davidson Hunter, III

1968-2012



 [image: Inline image 1]





Our family invites you to join us to celebrate and remember the

life of John Hunter



Memorial Service

Rockefeller Chapel

5850 South Woodlawn

Chicago, IL 60637



Monday

October 1, 2012

11am



Service will be followed by a reception where family and friends may gather
to share memories of John.



Please RSVP to Miriam at msierig at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120907/adee29e4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jdhj.jpg
Type: image/jpeg
Size: 370050 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120907/adee29e4/attachment.jpg>

From ellisonbg at gmail.com  Fri Sep  7 15:16:41 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 7 Sep 2012 12:16:41 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
Message-ID: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>

Hi,

Recently, we started to use IPEPs (IPython Enhancement Proposals) to
work out the design of significant new developments in IPython.  We
have had two IPEPs so far, and today we setup a wiki page to track
these IPEPs and formalize the process of creating and discussing of
IPEPs:

https://github.com/ipython/ipython/wiki/IPython-Enhancement-Proposals-(IPEPs)

Now that we have a good way of tracking IPEPs, I thought it was a good
opportunity to create a new IPEP myself.  We have started to get posts
on ipython-dev about multiuser capabilities in the notebook.  We have
previously thought and talked a lot about this, but we haven't written
anything down.  We are hoping that this IPEP will fix that and move
the design forward.

If you are interested in multiuser support in the notebook here is
what you should do:

1. Read the draft IPEP:

https://github.com/ipython/ipython/wiki/IPEP-3:-Multiuser-support-in-the-notebook

2. Discuss the IPEP in this email thread.

3. Help us edit the IPEP as the discussion moves along.

Cheers,

Brian

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Fri Sep  7 15:40:50 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 07 Sep 2012 14:40:50 -0500
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
Message-ID: <504A4DC2.8050508@creativetrax.com>

On 9/7/12 2:16 PM, Brian Granger wrote:
> Hi,
>
> 3. Help us edit the IPEP as the discussion moves along.

How do we do that?  For example the last sentence contains a typo 
(either->each).  I'd like to fix it: how?

Thanks,

Jason




From ellisonbg at gmail.com  Fri Sep  7 15:47:28 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 7 Sep 2012 12:47:28 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <504A4DC2.8050508@creativetrax.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
Message-ID: <CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>

OK I just changed a setting and you should be able to edit the wiki.

On Fri, Sep 7, 2012 at 12:40 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> On 9/7/12 2:16 PM, Brian Granger wrote:
>> Hi,
>>
>> 3. Help us edit the IPEP as the discussion moves along.
>
> How do we do that?  For example the last sentence contains a typo
> (either->each).  I'd like to fix it: how?
>
> Thanks,
>
> Jason
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From michael.k.bilow at gmail.com  Fri Sep  7 16:48:37 2012
From: michael.k.bilow at gmail.com (Michael Bilow)
Date: Fri, 7 Sep 2012 13:48:37 -0700
Subject: [IPython-dev] Confusing NameError in Parallelizing with IPython
Message-ID: <CAGMfGm2h=nKCP7T90gMbnC+zbFFaAgiAuRTEsTkwaSk2J8guEA@mail.gmail.com>

Hi!

I'm new to this list and to IPython, and I started using IPython to
parallelize some code I'd written (complete code:
https://github.com/chuyelchulo/ipython-nameerror, requires numpy and PIL).
I've been encountering a confusing NameError that keeps popping up when I
try to run this program: The function to be mapped (parallel_run) is
imported from a module different from __main__, but the interpreter
complains that the functions and modules used by parallel_run are not part
of the namespace.

I asked about the error here (
http://stackoverflow.com/questions/12304847/ipython-parallel-computing-namespace-issues),
and was told to come to this list.

Thanks!

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120907/5c3a3963/attachment.html>

From benjaminrk at gmail.com  Fri Sep  7 17:04:55 2012
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 7 Sep 2012 14:04:55 -0700
Subject: [IPython-dev] Confusing NameError in Parallelizing with IPython
In-Reply-To: <CAGMfGm2h=nKCP7T90gMbnC+zbFFaAgiAuRTEsTkwaSk2J8guEA@mail.gmail.com>
References: <CAGMfGm2h=nKCP7T90gMbnC+zbFFaAgiAuRTEsTkwaSk2J8guEA@mail.gmail.com>
Message-ID: <CAHNn8BX8xZzy4YOoyZ-x5Xyfj-mKJXEDBsG5_04wp0WStPpKEQ@mail.gmail.com>

On Fri, Sep 7, 2012 at 1:48 PM, Michael Bilow <michael.k.bilow at gmail.com>wrote:

> Hi!
>
> I'm new to this list and to IPython, and I started using IPython to
> parallelize some code I'd written (complete code:
> https://github.com/chuyelchulo/ipython-nameerror, requires numpy and
> PIL). I've been encountering a confusing NameError that keeps popping up
> when I try to run this program: The function to be mapped (parallel_run) is
> imported from a module different from __main__, but the interpreter
> complains that the functions and modules used by parallel_run are not part
> of the namespace.
>
> I asked about the error here (
> http://stackoverflow.com/questions/12304847/ipython-parallel-computing-namespace-issues),
> and was told to come to this list.
>

Hi again!

When I do the following:

$> cd /path/to/ipython-nameerror
$> ipcluster start -n 4 --daemon
$> python wrapper_wrapper.py
Time elapsed:  5.85908985138 s

It seems to work as expected.

A few quick questions:

1. what is the cwd when you run ipcluster?  Are your engines local or
remote?  What do you get from:

import os
import pprint
from IPython import parallel

client = parallel.Client()
pprint.pprint(client[:].apply_async(os.getcwdu).get_dict())

or:

%px import wrapper

If I start the engines somewhere else such that `wrapper` is not
importable, I get the NameErrors you are seeing.

-MinRK


> Thanks!
>
> Michael
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120907/ebc0cfca/attachment.html>

From michael.k.bilow at gmail.com  Fri Sep  7 17:18:36 2012
From: michael.k.bilow at gmail.com (Michael Bilow)
Date: Fri, 7 Sep 2012 14:18:36 -0700
Subject: [IPython-dev] Confusing NameError in Parallelizing with IPython
In-Reply-To: <CAHNn8BX8xZzy4YOoyZ-x5Xyfj-mKJXEDBsG5_04wp0WStPpKEQ@mail.gmail.com>
References: <CAGMfGm2h=nKCP7T90gMbnC+zbFFaAgiAuRTEsTkwaSk2J8guEA@mail.gmail.com>
	<CAHNn8BX8xZzy4YOoyZ-x5Xyfj-mKJXEDBsG5_04wp0WStPpKEQ@mail.gmail.com>
Message-ID: <CAGMfGm2GLQw6OfoX8t23wkdaCZDSL9Czw9=T8f0CX8LK-jSCWA@mail.gmail.com>

Wow, you nailed it, thanks.  I was running ipcluster start in a different
directory from the one my code was in.  I still don't understand the
provenance of that particular error, but I can see how it could cause one.

Thanks!

Michael

On Fri, Sep 7, 2012 at 2:04 PM, MinRK <benjaminrk at gmail.com> wrote:

>
>
> On Fri, Sep 7, 2012 at 1:48 PM, Michael Bilow <michael.k.bilow at gmail.com>wrote:
>
>> Hi!
>>
>> I'm new to this list and to IPython, and I started using IPython to
>> parallelize some code I'd written (complete code:
>> https://github.com/chuyelchulo/ipython-nameerror, requires numpy and
>> PIL). I've been encountering a confusing NameError that keeps popping up
>> when I try to run this program: The function to be mapped (parallel_run) is
>> imported from a module different from __main__, but the interpreter
>> complains that the functions and modules used by parallel_run are not part
>> of the namespace.
>>
>> I asked about the error here (
>> http://stackoverflow.com/questions/12304847/ipython-parallel-computing-namespace-issues),
>> and was told to come to this list.
>>
>
> Hi again!
>
> When I do the following:
>
> $> cd /path/to/ipython-nameerror
> $> ipcluster start -n 4 --daemon
> $> python wrapper_wrapper.py
> Time elapsed:  5.85908985138 s
>
> It seems to work as expected.
>
> A few quick questions:
>
> 1. what is the cwd when you run ipcluster?  Are your engines local or
> remote?  What do you get from:
>
> import os
> import pprint
> from IPython import parallel
>
> client = parallel.Client()
> pprint.pprint(client[:].apply_async(os.getcwdu).get_dict())
>
> or:
>
> %px import wrapper
>
> If I start the engines somewhere else such that `wrapper` is not
> importable, I get the NameErrors you are seeing.
>
> -MinRK
>
>
>> Thanks!
>>
>> Michael
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120907/1235d47f/attachment.html>

From jason-sage at creativetrax.com  Fri Sep  7 18:50:38 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 07 Sep 2012 17:50:38 -0500
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
Message-ID: <504A7A3E.8040609@creativetrax.com>

On 9/7/12 2:47 PM, Brian Granger wrote:
> OK I just changed a setting and you should be able to edit the wiki.

Thanks.  It worked.

Regarding: "When a user clicks "live share" they will get a private and 
secure URL they can share with other users. These other users can then 
edit the notebook and run code on the kernel in a Google Docs like manner."

Are you saying that the two of us will be interacting with one running 
kernel?  Or are you saying that we both have different kernels and the 
outputs are somehow pushed back and forth between our representations? 
The former seems more likely, but I was just checking.  In Sage we have 
the latter (IIRC), and sometimes it gets really confusing.

Thanks,

Jason




From fperez.net at gmail.com  Fri Sep  7 19:19:13 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 7 Sep 2012 16:19:13 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <504A7A3E.8040609@creativetrax.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
Message-ID: <CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>

On Fri, Sep 7, 2012 at 3:50 PM, Jason Grout <jason-sage at creativetrax.com> wrote:
> Are you saying that the two of us will be interacting with one running
> kernel?  Or are you saying that we both have different kernels and the
> outputs are somehow pushed back and forth between our representations?
> The former seems more likely, but I was just checking.  In Sage we have
> the latter (IIRC), and sometimes it gets really confusing.

The former.  We envision collaboration similar to Gdocs, running
against the same kernel, but with cell-level granularity for
concurrent typing instead of character-level.  When a user starts
typing in a cell, the others would see it shaded with that user's
name, effectively locking cells, so that they don't trample on each
other.

Creating a framework for synchronizing at the cell level is easier and
less finicky than getting something like google docs that syncs
character-by-character.  And I also don't quite like the feel of
complete 'all hands at the same time' editing gdocs provides, so
hopefully this will be a good experience.  We'll have to see.

Cheers,

f


From benjaminrk at gmail.com  Fri Sep  7 19:55:26 2012
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 7 Sep 2012 16:55:26 -0700
Subject: [IPython-dev] Confusing NameError in Parallelizing with IPython
In-Reply-To: <CAGMfGm2GLQw6OfoX8t23wkdaCZDSL9Czw9=T8f0CX8LK-jSCWA@mail.gmail.com>
References: <CAGMfGm2h=nKCP7T90gMbnC+zbFFaAgiAuRTEsTkwaSk2J8guEA@mail.gmail.com>
	<CAHNn8BX8xZzy4YOoyZ-x5Xyfj-mKJXEDBsG5_04wp0WStPpKEQ@mail.gmail.com>
	<CAGMfGm2GLQw6OfoX8t23wkdaCZDSL9Czw9=T8f0CX8LK-jSCWA@mail.gmail.com>
Message-ID: <CAHNn8BUmCj-Gr2miTuDSOULGECxBzu1CxM8+LjHoTCTfsr1_cQ@mail.gmail.com>

On Fri, Sep 7, 2012 at 2:18 PM, Michael Bilow <michael.k.bilow at gmail.com>wrote:

> Wow, you nailed it, thanks.  I was running ipcluster start in a different
> directory from the one my code was in.  I still don't understand the
> provenance of that particular error, but I can see how it could cause one.


Here's a mor detailed exploration of what is happening:

    import pickle
    from wrapper import parallel_run
    print pickle.dumps(parallel_run)

gives:

    cwrapper
    parallel_run
    p0
    .

In other words, the serialized form of a module function is just the
function name and the module in which it lives.

When the request arrives on the engines, it is deserialized back into its
module namespace like
so<https://github.com/ipython/ipython/blob/master/IPython/utils/pickleutil.py#L98>
:

            try:
                __import__(self.module)
            except ImportError:
                pass
            else:
                g = sys.modules[self.module].__dict__

So what is happening is the import of `wrapper` is failing, but IPython
suppressed that error, and unpacked the function into an empty namespace
instead.  It's possible that this error should *not* be suppressed, because
it certainly would have been more informative to you to see `ImportError:
wrapper` rather than a NameError since it came out in the wrong place.

I don't know what case is actually helped by this suppression, so I am
inclined to just cease catching the
error<https://github.com/ipython/ipython/pull/2395>
.

-MinRK


>
> Thanks!
>
> Michael
>
> On Fri, Sep 7, 2012 at 2:04 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>>
>>
>> On Fri, Sep 7, 2012 at 1:48 PM, Michael Bilow <michael.k.bilow at gmail.com>wrote:
>>
>>> Hi!
>>>
>>> I'm new to this list and to IPython, and I started using IPython to
>>> parallelize some code I'd written (complete code:
>>> https://github.com/chuyelchulo/ipython-nameerror, requires numpy and
>>> PIL). I've been encountering a confusing NameError that keeps popping up
>>> when I try to run this program: The function to be mapped (parallel_run) is
>>> imported from a module different from __main__, but the interpreter
>>> complains that the functions and modules used by parallel_run are not part
>>> of the namespace.
>>>
>>> I asked about the error here (
>>> http://stackoverflow.com/questions/12304847/ipython-parallel-computing-namespace-issues),
>>> and was told to come to this list.
>>>
>>
>> Hi again!
>>
>> When I do the following:
>>
>> $> cd /path/to/ipython-nameerror
>> $> ipcluster start -n 4 --daemon
>> $> python wrapper_wrapper.py
>> Time elapsed:  5.85908985138 s
>>
>> It seems to work as expected.
>>
>> A few quick questions:
>>
>> 1. what is the cwd when you run ipcluster?  Are your engines local or
>> remote?  What do you get from:
>>
>> import os
>> import pprint
>> from IPython import parallel
>>
>> client = parallel.Client()
>> pprint.pprint(client[:].apply_async(os.getcwdu).get_dict())
>>
>> or:
>>
>> %px import wrapper
>>
>> If I start the engines somewhere else such that `wrapper` is not
>> importable, I get the NameErrors you are seeing.
>>
>> -MinRK
>>
>>
>>> Thanks!
>>>
>>> Michael
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120907/db413f9f/attachment.html>

From bussonniermatthias at gmail.com  Sat Sep  8 04:19:44 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 8 Sep 2012 10:19:44 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <504A7A3E.8040609@creativetrax.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
Message-ID: <3E375ECA-EA19-4F71-8A8D-F0C5A39DB512@gmail.com>


Le 8 sept. 2012 ? 00:50, Jason Grout a ?crit :

> On 9/7/12 2:47 PM, Brian Granger wrote:
>> OK I just changed a setting and you should be able to edit the wiki.
> 
> Thanks.  It worked.
> 
> Regarding: "When a user clicks "live share" they will get a private and 
> secure URL they can share with other users. These other users can then 
> edit the notebook and run code on the kernel in a Google Docs like manner."
> 
> Are you saying that the two of us will be interacting with one running 
> kernel?  Or are you saying that we both have different kernels and the 
> outputs are somehow pushed back and forth between our representations? 
> The former seems more likely, but I was just checking.  In Sage we have 
> the latter (IIRC), and sometimes it gets really confusing.

Here is a demo of the former: 
http://elacave.lmdb.eu/~carreau/NotebookLiveCollaboration.mov
(really rough draft)
1 kernel.

If people want to run in different kernel, they clone and merge afterward the ipynb in git. 
see PR #2342 for how we could diff notebooks. 
-- 
Matthias

> 
> Thanks,
> 
> Jason
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From bussonniermatthias at gmail.com  Sat Sep  8 05:02:50 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 8 Sep 2012 11:02:50 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
Message-ID: <8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>



Comment on current IPEP version : 

> 	? Users will have the ability to "live share" notebooks and their kernels. When a user clicks "live share" they will get a private and secure URL they can share with other users. These other users can then edit the notebook and run code on the kernel in a Google Docs like manner.

This is one feature, but I would expect the ability to invite/remove user independently. 
Either by being able to tick checkboxes, or else. But nothing annoy me more than having to look into my email to find the link. 

Ability to have "soft" more grained privileges. Invite a user than can only edit and not run. 

Obviously we are really vulnerable to JS injection. We should definitely try to do something about that, even if we sell the notebook as beeing not totally secured. One thing we can use is html5 sandboxed feature of iframe. This could also solve the "problem" of css in ".rendered_html". 

Implementation wide for live collaborating, we should redirect the message using cell ID, otherwise this is a huge problem to redirect IOPub to all clients. 

> 	? Version control of notebooks will be done using the command line interface to git and other VCSs. These VCS command will be issued through the shell widget.


I think some high level command would be great, like changing 'save' button, to 'commit' with a custom message that can be generated by user.

One of the things I came across while prototyping live collaboration is the real need to have the current state of the notebook on the kernel side.
I think we should define a format of notebook 'patches' that represent common operation on a ipynb file, at least: 
	- appending/moving/deletting worksheet/cells
preferably with reference to the worksheet/cell by UUID. 

I also think that for live collaboration we need both a 'cell is being edited' lock, and also maybe a 'cell is being executed' lock. 

shift+Enter in collaboration is annoying as it makes you select the next cell, releasing your lock and selecting a new cell, potentially locked. 
I would be inclined to change ctrl+enter to execute without selecting all to be able to stay on the same cell.

-- 
Matthias


Le 8 sept. 2012 ? 01:19, Fernando Perez a ?crit :

> On Fri, Sep 7, 2012 at 3:50 PM, Jason Grout <jason-sage at creativetrax.com> wrote:
>> Are you saying that the two of us will be interacting with one running
>> kernel?  Or are you saying that we both have different kernels and the
>> outputs are somehow pushed back and forth between our representations?
>> The former seems more likely, but I was just checking.  In Sage we have
>> the latter (IIRC), and sometimes it gets really confusing.
> 
> The former.  We envision collaboration similar to Gdocs, running
> against the same kernel, but with cell-level granularity for
> concurrent typing instead of character-level.  When a user starts
> typing in a cell, the others would see it shaded with that user's
> name, effectively locking cells, so that they don't trample on each
> other.
> 
> Creating a framework for synchronizing at the cell level is easier and
> less finicky than getting something like google docs that syncs
> character-by-character.  And I also don't quite like the feel of
> complete 'all hands at the same time' editing gdocs provides, so
> hopefully this will be a good experience.  We'll have to see.
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From jason-sage at creativetrax.com  Sat Sep  8 15:16:43 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 14:16:43 -0500
Subject: [IPython-dev] Changing configuration after running
Message-ID: <504B999B.2080406@creativetrax.com>

When I create a new profile, the default ipython_config.py file has 
these lines:

# A list of dotted module names of IPython extensions to load.
c.InteractiveShellApp.extensions = [ ]

[snip...]

# dotted module name of an IPython extension to load.
# c.InteractiveShellApp.extra_extension = ''

What is the difference between .extensions and .extra_extension?

Thanks,

Jason


From jason-sage at creativetrax.com  Sat Sep  8 15:19:21 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 14:19:21 -0500
Subject: [IPython-dev] extra_extension config option
In-Reply-To: <504B999B.2080406@creativetrax.com>
References: <504B999B.2080406@creativetrax.com>
Message-ID: <504B9A39.9020306@creativetrax.com>

Sorry for the previous subject line that didn't make sense; it was left 
over from a message I decided against sending!

Jason



From benjaminrk at gmail.com  Sat Sep  8 15:31:41 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 8 Sep 2012 12:31:41 -0700
Subject: [IPython-dev] Changing configuration after running
In-Reply-To: <504B999B.2080406@creativetrax.com>
References: <504B999B.2080406@creativetrax.com>
Message-ID: <CAHNn8BVt-vsyeQvFx-1F_OMruoHvUqjXObmOg6Sse8FMS_4o9w@mail.gmail.com>

On Sat, Sep 8, 2012 at 12:16 PM, Jason Grout <jason-sage at creativetrax.com>wrote:

> When I create a new profile, the default ipython_config.py file has
> these lines:
>
> # A list of dotted module names of IPython extensions to load.
> c.InteractiveShellApp.extensions = [ ]
>
> [snip...]
>
> # dotted module name of an IPython extension to load.
> # c.InteractiveShellApp.extra_extension = ''
>
> What is the difference between .extensions and .extra_extension?
>

extra_extension just exists to preserve the `--ext` command-line option, so
you can do:

ipython --ext cythonmagic

Where cythonmagic is *added* to the extensions list.

`extra_extension = 'cythonmagic'` is identical to
`extensions.append('cythonmagic')`

This is one case the current config system still doesn't support well:
adding to containers.


>
> Thanks,
>
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120908/5eecf3a7/attachment.html>

From jason-sage at creativetrax.com  Sat Sep  8 15:46:25 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 14:46:25 -0500
Subject: [IPython-dev] changing input_splitter from config
Message-ID: <504BA091.1020301@creativetrax.com>

Is it possible to change the input splitter from within the 
ipython_config.py file?  I can successfully change it from a custom 
extension, but none of the variants I've tried in a config file seem to 
work.  For example, in the ipython_config file:

from IPython.core.inputsplitter import IPythonInputSplitter
class SageIPythonInputSplitter(IPythonInputSplitter):
     def push(self, lines):
         print "push line"
         super(SageaIPythonInputSplitter, self).push(lines)

# Execute the given command string.
c.InteractiveShellApp.input_splitter = SageIPythonInputSplitter()

Thanks,

Jason


From jason-sage at creativetrax.com  Sat Sep  8 15:51:05 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 14:51:05 -0500
Subject: [IPython-dev] Changing configuration after running
In-Reply-To: <CAHNn8BVt-vsyeQvFx-1F_OMruoHvUqjXObmOg6Sse8FMS_4o9w@mail.gmail.com>
References: <504B999B.2080406@creativetrax.com>
	<CAHNn8BVt-vsyeQvFx-1F_OMruoHvUqjXObmOg6Sse8FMS_4o9w@mail.gmail.com>
Message-ID: <504BA1A9.20801@creativetrax.com>

On 9/8/12 2:31 PM, MinRK wrote:
> `extra_extension = 'cythonmagic'` is identical to
> `extensions.append('cythonmagic')`
>
> This is one case the current config system still doesn't support well:
> adding to containers.

Thanks!  That makes sense.

Jason



From fperez.net at gmail.com  Sat Sep  8 16:19:52 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 8 Sep 2012 13:19:52 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
Message-ID: <CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>

On Sat, Sep 8, 2012 at 2:02 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
> Comment on current IPEP version :

> Ability to have "soft" more grained privileges. Invite a user than can only edit and not run.

Definitely.  Given how potentially destructive execution can be, this
is a very important permission level we want to have.  In static
documents (say google docs), the only things that matter are
read/write privileges.  We have read/write/execute, and all three
levels of permission should be available.

> Obviously we are really vulnerable to JS injection. We should definitely try to do something about that, even if we sell the notebook as beeing not totally secured. One thing we can use is html5 sandboxed feature of iframe. This could also solve the "problem" of css in ".rendered_html".
>

Yes, the ipep should have a section on security in the web sense (JS,
cross-scripting possibilities, etc).  I know next to nothing about
that stuff, but at least we should outline what the boundaries of the
problem are, and hopefully we'll get some expertise on the matter over
time.

> One of the things I came across while prototyping live collaboration is the real need to have the current state of the notebook on the kernel side.
> I think we should define a format of notebook 'patches' that represent common operation on a ipynb file, at least:
>         - appending/moving/deletting worksheet/cells
> preferably with reference to the worksheet/cell by UUID.

Yes, this is a tough one: for many reasons a fully stateless server
has many benefits, but it's also a real hindrance for other key use
cases.  Managing long-running processes is very hard with the current
design, and having all save operations send the full notebook back
uplink is simply unsustainable in the long run as it makes the system
unusable with large notebooks over asymmetric or metered links (such
as mobile connections).

We *will* have to come up with a solution where that state is managed
server side somehow, the current design simply fails to solve this key
use case that will become increasingly important as people move more
and more from local notebook use to server-based one.

> I also think that for live collaboration we need both a 'cell is being edited' lock, and also maybe a 'cell is being executed' lock.
>
> shift+Enter in collaboration is annoying as it makes you select the next cell, releasing your lock and selecting a new cell, potentially locked.
> I would be inclined to change ctrl+enter to execute without selecting all to be able to stay on the same cell.

We debated a lot whether to select all or not, initially it didn't.
I'm not opposed to going back to the no-select state on Ctrl-Enter if
we find a good reason for it (and this could be one).  I don't think
the reasons for going with select were super strong.

Cheers,

f


From fperez.net at gmail.com  Sat Sep  8 16:23:54 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 8 Sep 2012 13:23:54 -0700
Subject: [IPython-dev] Moving all dev docs and IPEPs to github
In-Reply-To: <CAH4pYpSwi2YVY+GNDuuLyb7xs13SoTiU1CroN4P=q42vWEL_VQ@mail.gmail.com>
References: <CAH4pYpRth8wufuQ5Vj1o83QLdHD-EnWV_HXyN1xrGDzaU2pfAA@mail.gmail.com>
	<CAHAreOo3UzCSUG2f3qNZEAvW2EipZwm7wLvSaJF3FF5Oj=JMjQ@mail.gmail.com>
	<CAH4pYpSwi2YVY+GNDuuLyb7xs13SoTiU1CroN4P=q42vWEL_VQ@mail.gmail.com>
Message-ID: <CAHAreOrLdNhTyZL+U_8J4KdN4rgcRv1FWhc8bmS4sJfB0A2yrw@mail.gmail.com>

On Fri, Sep 7, 2012 at 11:20 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> In my mind, a user focused wiki and a dev focused wiki could be
> maintained separately.  I really want to keep all of our dev related
> things on github as it makes our dev process much more discoverable.
> But I do agree that having two wikis is a bit silly.

I think a wiki can have a 'dev' area and a 'user' area, but having two
systems is more overhead than makes sense for us.  We have such a
small and constrained team, we really need to minimize all overhead.

> I think that moving everything to github is the best way to have all
> our dev related docs in one place rather than 3 or 4.  I am definitely
> +1 on the idea.

Well, let's go ahead and have a quick vote from everyone who has a
stake in it (aka anyone who has contributed to the wiki or is likely
to)...  I'll just send a separate email on it.  Let's have the vote on
the user list so users who aren't subscribed here also have a chance
to voice their opinion.

> Yes, our roadmap needs to be informal, flexible, non-verbose and
> dynamic.  I would like something that people feel good about updating
> often with small bits of information that is useful to all.  I don't
> want something that takes hours to update or is otherwise a burden to
> us.  I think that having most of our dev process managed by issues and
> PRs works great and I don't want to mess with that.  It is just a
> little bit of the bigger picture stuff that we need a bit more
> organization on.  And honestly, the IPEPs will help with that as well.

Agreed.

Cheers,

f


From carl.input at gmail.com  Sat Sep  8 17:09:54 2012
From: carl.input at gmail.com (Carl Smith)
Date: Sat, 8 Sep 2012 22:09:54 +0100
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
Message-ID: <CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>

I'm not sure, but I don't think you can do cross-scripting in Chrome. Maybe
other browsers will make this concern mute too. I'm not certain, but that's
what I thought. I'm on my phone so I can't do much to look into it right
now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120908/96780a10/attachment.html>

From carl.input at gmail.com  Sat Sep  8 17:19:20 2012
From: carl.input at gmail.com (Carl Smith)
Date: Sat, 8 Sep 2012 22:19:20 +0100
Subject: [IPython-dev] Moving all dev docs and IPEPs to github
In-Reply-To: <CAHAreOrLdNhTyZL+U_8J4KdN4rgcRv1FWhc8bmS4sJfB0A2yrw@mail.gmail.com>
References: <CAH4pYpRth8wufuQ5Vj1o83QLdHD-EnWV_HXyN1xrGDzaU2pfAA@mail.gmail.com>
	<CAHAreOo3UzCSUG2f3qNZEAvW2EipZwm7wLvSaJF3FF5Oj=JMjQ@mail.gmail.com>
	<CAH4pYpSwi2YVY+GNDuuLyb7xs13SoTiU1CroN4P=q42vWEL_VQ@mail.gmail.com>
	<CAHAreOrLdNhTyZL+U_8J4KdN4rgcRv1FWhc8bmS4sJfB0A2yrw@mail.gmail.com>
Message-ID: <CAP-uhDe4ZYew9e9H1RHQJdjAmoNDNPkF6BTTrDC-Dcg0fHiURw@mail.gmail.com>

I'd like to see a user focussed nb based set of docs for new users, but not
a whole seperate wiki. There's currently nothing that I know of that serves
as a self-contained user manual. Perhaps start with A Brief Tour, then one
nb per topic, but as a complete guide for users. I'd be keen to do a bunch
of work on this, but would need some help with it.
On Sep 8, 2012 9:26 PM, "Fernando Perez" <fperez.net at gmail.com> wrote:

> On Fri, Sep 7, 2012 at 11:20 AM, Brian Granger <ellisonbg at gmail.com>
> wrote:
>
> > In my mind, a user focused wiki and a dev focused wiki could be
> > maintained separately.  I really want to keep all of our dev related
> > things on github as it makes our dev process much more discoverable.
> > But I do agree that having two wikis is a bit silly.
>
> I think a wiki can have a 'dev' area and a 'user' area, but having two
> systems is more overhead than makes sense for us.  We have such a
> small and constrained team, we really need to minimize all overhead.
>
> > I think that moving everything to github is the best way to have all
> > our dev related docs in one place rather than 3 or 4.  I am definitely
> > +1 on the idea.
>
> Well, let's go ahead and have a quick vote from everyone who has a
> stake in it (aka anyone who has contributed to the wiki or is likely
> to)...  I'll just send a separate email on it.  Let's have the vote on
> the user list so users who aren't subscribed here also have a chance
> to voice their opinion.
>
> > Yes, our roadmap needs to be informal, flexible, non-verbose and
> > dynamic.  I would like something that people feel good about updating
> > often with small bits of information that is useful to all.  I don't
> > want something that takes hours to update or is otherwise a burden to
> > us.  I think that having most of our dev process managed by issues and
> > PRs works great and I don't want to mess with that.  It is just a
> > little bit of the bigger picture stuff that we need a bit more
> > organization on.  And honestly, the IPEPs will help with that as well.
>
> Agreed.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120908/e6c43d3f/attachment.html>

From fperez.net at gmail.com  Sat Sep  8 17:43:51 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 8 Sep 2012 14:43:51 -0700
Subject: [IPython-dev] Moving all dev docs and IPEPs to github
In-Reply-To: <CAP-uhDe4ZYew9e9H1RHQJdjAmoNDNPkF6BTTrDC-Dcg0fHiURw@mail.gmail.com>
References: <CAH4pYpRth8wufuQ5Vj1o83QLdHD-EnWV_HXyN1xrGDzaU2pfAA@mail.gmail.com>
	<CAHAreOo3UzCSUG2f3qNZEAvW2EipZwm7wLvSaJF3FF5Oj=JMjQ@mail.gmail.com>
	<CAH4pYpSwi2YVY+GNDuuLyb7xs13SoTiU1CroN4P=q42vWEL_VQ@mail.gmail.com>
	<CAHAreOrLdNhTyZL+U_8J4KdN4rgcRv1FWhc8bmS4sJfB0A2yrw@mail.gmail.com>
	<CAP-uhDe4ZYew9e9H1RHQJdjAmoNDNPkF6BTTrDC-Dcg0fHiURw@mail.gmail.com>
Message-ID: <CAHAreOoucu=JD0oPiWu4bDUdUBt+s_UScynQtEd+4RmQRVG4aw@mail.gmail.com>

On Sat, Sep 8, 2012 at 2:19 PM, Carl Smith <carl.input at gmail.com> wrote:
> I'd like to see a user focussed nb based set of docs for new users, but not
> a whole seperate wiki. There's currently nothing that I know of that serves
> as a self-contained user manual. Perhaps start with A Brief Tour, then one
> nb per topic, but as a complete guide for users. I'd be keen to do a bunch
> of work on this, but would need some help with it.

Now that nbconvert is getting into shape, what we need is to
incorporate the conversion of our existing example notebooks into the
documentation, along with restructuring the docs themselves.

That would be a fantastic, self-contained project for someone to help with!

Cheers,

f


From ellisonbg at gmail.com  Sat Sep  8 18:24:15 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 8 Sep 2012 15:24:15 -0700
Subject: [IPython-dev] changing input_splitter from config
In-Reply-To: <504BA091.1020301@creativetrax.com>
References: <504BA091.1020301@creativetrax.com>
Message-ID: <CAH4pYpTvd72U-Q_riTyzdLx6L0HF7zfYdkRAkPPLtvbJniVZrg@mail.gmail.com>

Only class attributes that have config=True set can be configured from
the command line or config files.  In general, these are things that
have values of simple types such as strings, ints, bools, etc.

When you do:

c.InteractiveShellApp.input_splitter = SageIPythonInputSplitter()

You are not setting this attribute on the real InteractiveShellApp (I
think it would actually be InteractiveShell but thats a side point).
Instead you are setting the input_splitter attr on a special config
object that is used when the real InteractiveShell is created.  What
you are wanting to do is fundamentally change how that object is built
so an extension is the only way to go.

I suppose we might be able to create a config=True
input_splitter_class attribute that could be set to
SageIPythonInputSplitter.  Then you could set this in your config
files.  I think that would be a pretty simple PR to put together.  One
thing to be careful of though - the input splitter is used in multiple
locations in the code base you we would need to make sure that all of
those are changed to the same value.

Cheers,

Brian

On Sat, Sep 8, 2012 at 12:46 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> Is it possible to change the input splitter from within the
> ipython_config.py file?  I can successfully change it from a custom
> extension, but none of the variants I've tried in a config file seem to
> work.  For example, in the ipython_config file:
>
> from IPython.core.inputsplitter import IPythonInputSplitter
> class SageIPythonInputSplitter(IPythonInputSplitter):
>      def push(self, lines):
>          print "push line"
>          super(SageaIPythonInputSplitter, self).push(lines)
>
> # Execute the given command string.
> c.InteractiveShellApp.input_splitter = SageIPythonInputSplitter()
>
> Thanks,
>
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Sat Sep  8 19:08:45 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 18:08:45 -0500
Subject: [IPython-dev] changing input_splitter from config
In-Reply-To: <CAH4pYpTvd72U-Q_riTyzdLx6L0HF7zfYdkRAkPPLtvbJniVZrg@mail.gmail.com>
References: <504BA091.1020301@creativetrax.com>
	<CAH4pYpTvd72U-Q_riTyzdLx6L0HF7zfYdkRAkPPLtvbJniVZrg@mail.gmail.com>
Message-ID: <504BCFFD.3000008@creativetrax.com>

On 9/8/12 5:24 PM, Brian Granger wrote:
> Only class attributes that have config=True set can be configured from
> the command line or config files.  In general, these are things that
> have values of simple types such as strings, ints, bools, etc.
>
> When you do:
>
> c.InteractiveShellApp.input_splitter = SageIPythonInputSplitter()
>
> You are not setting this attribute on the real InteractiveShellApp (I
> think it would actually be InteractiveShell but thats a side point).
> Instead you are setting the input_splitter attr on a special config
> object that is used when the real InteractiveShell is created.  What
> you are wanting to do is fundamentally change how that object is built
> so an extension is the only way to go.
>
> I suppose we might be able to create a config=True
> input_splitter_class attribute that could be set to
> SageIPythonInputSplitter.  Then you could set this in your config
> files.  I think that would be a pretty simple PR to put together.  One
> thing to be careful of though - the input splitter is used in multiple
> locations in the code base you we would need to make sure that all of
> those are changed to the same value.
>

Thanks for the explanation!

It's not a problem for me to do it in an extension.  I have to load some 
magics and set up a few other things in an extension anyway, so I can 
set the input splitter there too.  It just looked to me like I should be 
able to configure it from the profile, and then I was puzzled why it 
wasn't working.

I have to say I'm really impressed (again) with the hook system and the 
config system.  I'm trying to shift all of the Sage customizations into 
either a config file or an extension (i.e., a combination of both), and 
so far, it's working out nicely.  (And it's fairly simple thanks to the 
work other Sage devs did already in a first pass at migrating to the 
newest ipython [1]).

Thanks,

Jason

[1] http://trac.sagemath.org/sage_trac/ticket/12719


From carl.input at gmail.com  Sat Sep  8 19:36:02 2012
From: carl.input at gmail.com (Carl Smith)
Date: Sun, 9 Sep 2012 00:36:02 +0100
Subject: [IPython-dev] Moving all dev docs and IPEPs to github
In-Reply-To: <CAHAreOoucu=JD0oPiWu4bDUdUBt+s_UScynQtEd+4RmQRVG4aw@mail.gmail.com>
References: <CAH4pYpRth8wufuQ5Vj1o83QLdHD-EnWV_HXyN1xrGDzaU2pfAA@mail.gmail.com>
	<CAHAreOo3UzCSUG2f3qNZEAvW2EipZwm7wLvSaJF3FF5Oj=JMjQ@mail.gmail.com>
	<CAH4pYpSwi2YVY+GNDuuLyb7xs13SoTiU1CroN4P=q42vWEL_VQ@mail.gmail.com>
	<CAHAreOrLdNhTyZL+U_8J4KdN4rgcRv1FWhc8bmS4sJfB0A2yrw@mail.gmail.com>
	<CAP-uhDe4ZYew9e9H1RHQJdjAmoNDNPkF6BTTrDC-Dcg0fHiURw@mail.gmail.com>
	<CAHAreOoucu=JD0oPiWu4bDUdUBt+s_UScynQtEd+4RmQRVG4aw@mail.gmail.com>
Message-ID: <CAP-uhDdCWwde3D3sW1BGvP+ndE6Ur6EQxYjKLHXNnUKu-oTexg@mail.gmail.com>

I'll put myself forward for this one, no problem. I'll wait and see how
much help I can be with the wiki migration, and start exploring this in the
mean while. I'm real busy tonight, but will be online and free for a few
days as of tomorrow.

Cheers

Carl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/e8ffd921/attachment.html>

From fperez.net at gmail.com  Sat Sep  8 20:17:51 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 8 Sep 2012 17:17:51 -0700
Subject: [IPython-dev] changing input_splitter from config
In-Reply-To: <504BCFFD.3000008@creativetrax.com>
References: <504BA091.1020301@creativetrax.com>
	<CAH4pYpTvd72U-Q_riTyzdLx6L0HF7zfYdkRAkPPLtvbJniVZrg@mail.gmail.com>
	<504BCFFD.3000008@creativetrax.com>
Message-ID: <CAHAreOq3jUxv3FGO2WRPyVG56HUfr1oiT_Y1ghJTstfj-GMFDw@mail.gmail.com>

On Sat, Sep 8, 2012 at 4:08 PM, Jason Grout <jason-sage at creativetrax.com> wrote:
>
> I have to say I'm really impressed (again) with the hook system and the
> config system.  I'm trying to shift all of the Sage customizations into
> either a config file or an extension (i.e., a combination of both), and
> so far, it's working out nicely.  (And it's fairly simple thanks to the
> work other Sage devs did already in a first pass at migrating to the
> newest ipython [1]).

Glad to hear that, and it's important to note that we view that goal
as something that you should be able to achieve; if something prevents
it in IPython, it's a bug.  The Sage command line client is a heavily
customized ipython, but it doesn't fundamentally change the logic of a
REPL, so it should be 100% doable via configuration/extension fixes,
without having to subclass/rewrite any of the main parts of the code.
Keep on pinging us if you find anything that prevents that (and I know
that the docs explaining this could use improvement).

Cheers,

f


From ellisonbg at gmail.com  Sat Sep  8 20:31:15 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 8 Sep 2012 17:31:15 -0700
Subject: [IPython-dev] changing input_splitter from config
In-Reply-To: <504BCFFD.3000008@creativetrax.com>
References: <504BA091.1020301@creativetrax.com>
	<CAH4pYpTvd72U-Q_riTyzdLx6L0HF7zfYdkRAkPPLtvbJniVZrg@mail.gmail.com>
	<504BCFFD.3000008@creativetrax.com>
Message-ID: <CAH4pYpT5w9iQTzNXK13vtUrWxXUMBkC5MLHrpJO18t0_yZicmA@mail.gmail.com>

One other factor:

If you want that bit of configuration to be easily usable by a lot of
people, then putting it into an extension is definitely the way to go.

On Sat, Sep 8, 2012 at 4:08 PM, Jason Grout <jason-sage at creativetrax.com> wrote:
> On 9/8/12 5:24 PM, Brian Granger wrote:
>> Only class attributes that have config=True set can be configured from
>> the command line or config files.  In general, these are things that
>> have values of simple types such as strings, ints, bools, etc.
>>
>> When you do:
>>
>> c.InteractiveShellApp.input_splitter = SageIPythonInputSplitter()
>>
>> You are not setting this attribute on the real InteractiveShellApp (I
>> think it would actually be InteractiveShell but thats a side point).
>> Instead you are setting the input_splitter attr on a special config
>> object that is used when the real InteractiveShell is created.  What
>> you are wanting to do is fundamentally change how that object is built
>> so an extension is the only way to go.
>>
>> I suppose we might be able to create a config=True
>> input_splitter_class attribute that could be set to
>> SageIPythonInputSplitter.  Then you could set this in your config
>> files.  I think that would be a pretty simple PR to put together.  One
>> thing to be careful of though - the input splitter is used in multiple
>> locations in the code base you we would need to make sure that all of
>> those are changed to the same value.
>>
>
> Thanks for the explanation!
>
> It's not a problem for me to do it in an extension.  I have to load some
> magics and set up a few other things in an extension anyway, so I can
> set the input splitter there too.  It just looked to me like I should be
> able to configure it from the profile, and then I was puzzled why it
> wasn't working.
>
> I have to say I'm really impressed (again) with the hook system and the
> config system.  I'm trying to shift all of the Sage customizations into
> either a config file or an extension (i.e., a combination of both), and
> so far, it's working out nicely.  (And it's fairly simple thanks to the
> work other Sage devs did already in a first pass at migrating to the
> newest ipython [1]).
>
> Thanks,
>
> Jason
>
> [1] http://trac.sagemath.org/sage_trac/ticket/12719
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Sat Sep  8 20:45:43 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 19:45:43 -0500
Subject: [IPython-dev] changing input_splitter from config
In-Reply-To: <CAH4pYpT5w9iQTzNXK13vtUrWxXUMBkC5MLHrpJO18t0_yZicmA@mail.gmail.com>
References: <504BA091.1020301@creativetrax.com>
	<CAH4pYpTvd72U-Q_riTyzdLx6L0HF7zfYdkRAkPPLtvbJniVZrg@mail.gmail.com>
	<504BCFFD.3000008@creativetrax.com>
	<CAH4pYpT5w9iQTzNXK13vtUrWxXUMBkC5MLHrpJO18t0_yZicmA@mail.gmail.com>
Message-ID: <504BE6B7.10602@creativetrax.com>

On 9/8/12 7:31 PM, Brian Granger wrote:
> One other factor:
>
> If you want that bit of configuration to be easily usable by a lot of
> people, then putting it into an extension is definitely the way to go.
>

Yes, I was debating how much to put in the config file, and how much to 
do on the fly in an extension.  For example, changing the banner 
definitely goes into the config file.  But the auto imports that Sage 
does, or changing the prompt, and things like that---the in-my-head 
debate is still going about which place is better.

By the way, my ongoing work on this is at 
https://github.com/jasongrout/sage-ipython-profile

Thanks,

Jason



From jason-sage at creativetrax.com  Sat Sep  8 20:58:02 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 08 Sep 2012 19:58:02 -0500
Subject: [IPython-dev] oinspect overrideable?
Message-ID: <504BE99A.9080706@creativetrax.com>

Sage overrides IPython's inspect modules in order to support Sage's 
(admittedly weird) directory structure (a throwback to early Cython 
days, IIRC).  Anyways, in Sage's current pass to use the new IPython, I 
see this comment:

         # Ideally, these would just be methods of the Inspector class
         # that we could override; however, IPython looks them up in
         # the global :class:`IPython.core.oinspect` module namespace.
         # Thus, we have to monkey-patch.
         import sagedoc, sageinspect, IPython.core.oinspect
         IPython.core.oinspect.getdoc = sageinspect.sage_getdoc 
#sagedoc.my_getdoc
         IPython.core.oinspect.getsource = sagedoc.my_getsource
         IPython.core.oinspect.getargspec = sageinspect.sage_getargspec

I searched for each of these functions, and it does seem that they are 
being imported and used directly.  Would it be easy to convert those 
calls to calls on a current inspector object so that these functions 
would be easy to override?

Thanks,

Jason


From bussonniermatthias at gmail.com  Sun Sep  9 05:57:10 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 9 Sep 2012 11:57:10 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
	<CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>
Message-ID: <E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>


Le 8 sept. 2012 ? 23:09, Carl Smith a ?crit :

> I'm not sure, but I don't think you can do cross-scripting in Chrome. Maybe other browsers will make this concern mute too. I'm not certain, but that's what I thought. I'm on my phone so I can't do much to look into it right now.

Well, If I understand XSS it is pretty trivial to do with python notebook as we allow to embed script in .ipynb files. 
They might not be executable but a simple notebook with in a markdown cell :

<a href='onclick=function(){alert('hello')}'> click me </a> should work. 
moreover you can embed iframes, ...Etc.

So could forge a malicious ipynb file and ask you to view it through nbviewer.ipython.org.

assuming you are logged to nbviewer.ipython.org, the scripts in this notebook has all your rights. 

For me, this is XSS.

I won't imagine what people would try to do if you know that JS can send code to execute on the server side ! 
-- 
Matthias


> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From carl.input at gmail.com  Sun Sep  9 08:33:34 2012
From: carl.input at gmail.com (Carl Smith)
Date: Sun, 9 Sep 2012 13:33:34 +0100
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
	<CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>
	<E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>
Message-ID: <CAP-uhDdF9cRM=Wro6K07vg8ZfsyXK45W=s=fnUxzPGOANsLyzA@mail.gmail.com>

I'd totally misunderstood the problem. That's all pretty scary. Thanks for
putting me straight.
On Sep 9, 2012 10:57 AM, "Matthias BUSSONNIER" <bussonniermatthias at gmail.com>
wrote:

>
> Le 8 sept. 2012 ? 23:09, Carl Smith a ?crit :
>
> > I'm not sure, but I don't think you can do cross-scripting in Chrome.
> Maybe other browsers will make this concern mute too. I'm not certain, but
> that's what I thought. I'm on my phone so I can't do much to look into it
> right now.
>
> Well, If I understand XSS it is pretty trivial to do with python notebook
> as we allow to embed script in .ipynb files.
> They might not be executable but a simple notebook with in a markdown cell
> :
>
> <a href='onclick=function(){alert('hello')}'> click me </a> should work.
> moreover you can embed iframes, ...Etc.
>
> So could forge a malicious ipynb file and ask you to view it through
> nbviewer.ipython.org.
>
> assuming you are logged to nbviewer.ipython.org, the scripts in this
> notebook has all your rights.
>
> For me, this is XSS.
>
> I won't imagine what people would try to do if you know that JS can send
> code to execute on the server side !
> --
> Matthias
>
>
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/3d3cef49/attachment.html>

From bussonniermatthias at gmail.com  Sun Sep  9 08:44:10 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 9 Sep 2012 14:44:10 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAP-uhDdF9cRM=Wro6K07vg8ZfsyXK45W=s=fnUxzPGOANsLyzA@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
	<CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>
	<E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>
	<CAP-uhDdF9cRM=Wro6K07vg8ZfsyXK45W=s=fnUxzPGOANsLyzA@mail.gmail.com>
Message-ID: <B62AFC11-8204-4E27-823B-D459002D50DB@gmail.com>


Le 9 sept. 2012 ? 14:33, Carl Smith a ?crit :

> I'd totally misunderstood the problem. That's all pretty scary. Thanks for putting me straight.

Don't worry, I'm discovering it too, actually I discovered it when making a prototype a live collaboration. 
I was like "oh my god, I can do anything I want in my chrome browser from safari... 
F*** this means so else could do that too, without me knowing"

This concern me a little more for nbviewer which is meant for sharing notebook, and for which we would like to add user login, but we would be really vulnerable. 
-- 
Matthias

> 
> On Sep 9, 2012 10:57 AM, "Matthias BUSSONNIER" <bussonniermatthias at gmail.com> wrote:
> 
> Le 8 sept. 2012 ? 23:09, Carl Smith a ?crit :
> 
> > I'm not sure, but I don't think you can do cross-scripting in Chrome. Maybe other browsers will make this concern mute too. I'm not certain, but that's what I thought. I'm on my phone so I can't do much to look into it right now.
> 
> Well, If I understand XSS it is pretty trivial to do with python notebook as we allow to embed script in .ipynb files.
> They might not be executable but a simple notebook with in a markdown cell :
> 
> <a href='onclick=function(){alert('hello')}'> click me </a> should work.
> moreover you can embed iframes, ...Etc.
> 
> So could forge a malicious ipynb file and ask you to view it through nbviewer.ipython.org.
> 
> assuming you are logged to nbviewer.ipython.org, the scripts in this notebook has all your rights.
> 
> For me, this is XSS.
> 
> I won't imagine what people would try to do if you know that JS can send code to execute on the server side !
> --
> Matthias
> 
> 
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From satra at mit.edu  Sun Sep  9 09:35:01 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Sun, 9 Sep 2012 15:35:01 +0200
Subject: [IPython-dev] multiprocessing from within notebook crashes kernel
Message-ID: <CA+A4wOmg85DrpZYnwx_WSZUZ8y9fR9pTdK5-KZJisb0KsYfQmA@mail.gmail.com>

hi all,

we are using a workflow library that internally uses multiprocessing to
distribute jobs locally. when we run this from the notebook, it crashes the
kernel.

any thoughts as to why this might happen.

here is the code from the library:

https://github.com/nipy/nipype/blob/master/nipype/pipeline/plugins/multiproc.py

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/bb7e131e/attachment.html>

From ellisonbg at gmail.com  Sun Sep  9 12:10:42 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 9 Sep 2012 09:10:42 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
Message-ID: <CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>

Matthias,


On Sat, Sep 8, 2012 at 2:02 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
>
> Comment on current IPEP version :
>
>>       ? Users will have the ability to "live share" notebooks and their kernels. When a user clicks "live share" they will get a private and secure URL they can share with other users. These other users can then edit the notebook and run code on the kernel in a Google Docs like manner.
>
> This is one feature, but I would expect the ability to invite/remove user independently.
> Either by being able to tick checkboxes, or else. But nothing annoy me more than having to look into my email to find the link.
>
> Ability to have "soft" more grained privileges. Invite a user than can only edit and not run.

One thing we need to think through is the different "modes" a user can
view a page in.  Minimally we have three boolean flags that determine
the mode: read, write, execute.  Are there others?

One big thing we will have to work out is how to tune the
content/display of each page in a way that reflects these flags, but
that is secure.  Our current approach of setting a variable in JS to
control things is horribly insecure.  Ideas on this?

> Obviously we are really vulnerable to JS injection. We should definitely try to do something about that, even if we sell the notebook as beeing not totally secured. One thing we can use is html5 sandboxed feature of iframe. This could also solve the "problem" of css in ".rendered_html".

I agree we need to think more about security, but that is really off
topic for this IPEP (i.e., the same technical problems exist for the
single user notebook).

> Implementation wide for live collaborating, we should redirect the message using cell ID, otherwise this is a huge problem to redirect IOPub to all clients.

I am not quite following you here.  Don't all clients need to get all
iopub messages?

>>       ? Version control of notebooks will be done using the command line interface to git and other VCSs. These VCS command will be issued through the shell widget.
>
>
> I think some high level command would be great, like changing 'save' button, to 'commit' with a custom message that can be generated by user.

We have talked about this before and we want to avoid creating a git
UI in the notebook.  The other issue is that repo level actions are
not per notebook, but per directory, so if we did add a commit button,
it would need to be on the directory dashboard, not the notebook page.

> One of the things I came across while prototyping live collaboration is the real need to have the current state of the notebook on the kernel side.
> I think we should define a format of notebook 'patches' that represent common operation on a ipynb file, at least:
>         - appending/moving/deletting worksheet/cells
> preferably with reference to the worksheet/cell by UUID.

I am not sure what you mean by having the notebook state on the
kernel.  We have to be very careful to keep the notebook and kernel
state completely separate.  Can you elaborate on why this was needed?
But actually, can you open a separate IPEP for live collaboration - it
is really a big topic in itself.

> I also think that for live collaboration we need both a 'cell is being edited' lock, and also maybe a 'cell is being executed' lock.

Yep, I think at the cell level that having locks like this makes sense
for us.  The problem is doing so in an asynchronous setting (what if
both of us lock the same cell before the server is notified?)

> shift+Enter in collaboration is annoying as it makes you select the next cell, releasing your lock and selecting a new cell, potentially locked.
> I would be inclined to change ctrl+enter to execute without selecting all to be able to stay on the same cell.

Yes, we will have to think carefully about what the cursor does upon
executing a cell in live collaboration.

> --
> Matthias
>
>
> Le 8 sept. 2012 ? 01:19, Fernando Perez a ?crit :
>
>> On Fri, Sep 7, 2012 at 3:50 PM, Jason Grout <jason-sage at creativetrax.com> wrote:
>>> Are you saying that the two of us will be interacting with one running
>>> kernel?  Or are you saying that we both have different kernels and the
>>> outputs are somehow pushed back and forth between our representations?
>>> The former seems more likely, but I was just checking.  In Sage we have
>>> the latter (IIRC), and sometimes it gets really confusing.
>>
>> The former.  We envision collaboration similar to Gdocs, running
>> against the same kernel, but with cell-level granularity for
>> concurrent typing instead of character-level.  When a user starts
>> typing in a cell, the others would see it shaded with that user's
>> name, effectively locking cells, so that they don't trample on each
>> other.
>>
>> Creating a framework for synchronizing at the cell level is easier and
>> less finicky than getting something like google docs that syncs
>> character-by-character.  And I also don't quite like the feel of
>> complete 'all hands at the same time' editing gdocs provides, so
>> hopefully this will be a good experience.  We'll have to see.
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sun Sep  9 12:13:14 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 9 Sep 2012 09:13:14 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
Message-ID: <CAH4pYpRSFieO52WGwo+8O9YseOy+68J8Uq15Qe1ZZ7oe14iNcw@mail.gmail.com>

> Yes, this is a tough one: for many reasons a fully stateless server
> has many benefits, but it's also a real hindrance for other key use
> cases.  Managing long-running processes is very hard with the current
> design, and having all save operations send the full notebook back
> uplink is simply unsustainable in the long run as it makes the system
> unusable with large notebooks over asymmetric or metered links (such
> as mobile connections).
>
> We *will* have to come up with a solution where that state is managed
> server side somehow, the current design simply fails to solve this key
> use case that will become increasingly important as people move more
> and more from local notebook use to server-based one.

I agree this stuff is all important, but let's open an separate IPEP
for live collaboration and notebook saving issues.

-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sun Sep  9 12:19:21 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 9 Sep 2012 09:19:21 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
	<CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>
	<E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>
Message-ID: <CAH4pYpSVyFCQX55ei_A9vSEdwGei+Gyi75hszYDzdFrFo+N9YQ@mail.gmail.com>

On Sun, Sep 9, 2012 at 2:57 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 8 sept. 2012 ? 23:09, Carl Smith a ?crit :
>
>> I'm not sure, but I don't think you can do cross-scripting in Chrome. Maybe other browsers will make this concern mute too. I'm not certain, but that's what I thought. I'm on my phone so I can't do much to look into it right now.
>
> Well, If I understand XSS it is pretty trivial to do with python notebook as we allow to embed script in .ipynb files.
> They might not be executable but a simple notebook with in a markdown cell :
>
> <a href='onclick=function(){alert('hello')}'> click me </a> should work.
> moreover you can embed iframes, ...Etc.
>
> So could forge a malicious ipynb file and ask you to view it through nbviewer.ipython.org.
>
> assuming you are logged to nbviewer.ipython.org, the scripts in this notebook has all your rights.
>
> For me, this is XSS.
>
> I won't imagine what people would try to do if you know that JS can send code to execute on the server side !

I am not sure I am following the discussion here.  The IPython
Notebook is designed to execute arbitrary python and javascript code.
we don't view that as a security vulnerability - it is our central
feature!  Is this what you are talking about or is there some other
subtle aspect you are referring to?

One thing you might be talking about is the fact that <script> tags in
markdown cells can be executed on page load without a user knowing
about them.  The markdown renderer we are using has the ability to
strip <script> tags.  We might want to do that so the only way that
javascript code can be run is by the kernel running code that sends
back dynamic javascript.  I am probably +0.5 on this idea currently.

> --
> Matthias
>
>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Sun Sep  9 12:39:12 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 9 Sep 2012 18:39:12 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
Message-ID: <D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>


Le 9 sept. 2012 ? 18:10, Brian Granger a ?crit :

> Matthias,
> 
> 
> On Sat, Sep 8, 2012 at 2:02 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> 
>> 
>> Comment on current IPEP version :
>> 
>>>      ? Users will have the ability to "live share" notebooks and their kernels. When a user clicks "live share" they will get a private and secure URL they can share with other users. These other users can then edit the notebook and run code on the kernel in a Google Docs like manner.
>> 
>> This is one feature, but I would expect the ability to invite/remove user independently.
>> Either by being able to tick checkboxes, or else. But nothing annoy me more than having to look into my email to find the link.
>> 
>> Ability to have "soft" more grained privileges. Invite a user than can only edit and not run.
> 
> One thing we need to think through is the different "modes" a user can
> view a page in.  Minimally we have three boolean flags that determine
> the mode: read, write, execute.  Are there others?

I see 5 different level of authorization

	? Ability to see a static version of the current saved, or specific version of the notebook (cf nbviewer).
	? Ability to see a live edited version of the notebook, without being able to interact.
	? Ability to edit part of the notebook and push the changes
	? Ability to save the current changes on disk
	? Ability to execute code on the kernel.


> One big thing we will have to work out is how to tune the
> content/display of each page in a way that reflects these flags, but
> that is secure.  Our current approach of setting a variable in JS to
> control things is horribly insecure.  Ideas on this?

We have to have more fine grained decorators in the handler side. 
Js side is just to make things pretty. 

> 
>> Obviously we are really vulnerable to JS injection. We should definitely try to do something about that, even if we sell the notebook as beeing not totally secured. One thing we can use is html5 sandboxed feature of iframe. This could also solve the "problem" of css in ".rendered_html".
> 
> I agree we need to think more about security, but that is really off
> topic for this IPEP (i.e., the same technical problems exist for the
> single user notebook).

If you don't allow collaboration, then yes this is off topic. 
As soon as you add collaboration with username and password, you have a risk of user stealing each other account. 
We should think of it while building multi user account. 

> 
>> Implementation wide for live collaborating, we should redirect the message using cell ID, otherwise this is a huge problem to redirect IOPub to all clients.
> 
> I am not quite following you here.  Don't all clients need to get all
> iopub messages?

Yes they do. 
But they don't now how to handle it. 

if I run Safari and Chrome on "notebook_a"
When I execute "1+1" on Safari, for cell 54, when iopub arrives on Chrome, it has no idea of what to do with it. 
I think we should proxy :

Browser(s) ===>===> [execute+cell_id(+uuid ?)] ===>==> server strip cell_id (, add uuid) ? ==>==> kernel

Browser(s) <===<==<===<===<===<==<===<===<=== server match uuid, append cell_id <==<=== kernel

> 
>>>      ? Version control of notebooks will be done using the command line interface to git and other VCSs. These VCS command will be issued through the shell widget.
>> 
>> 
>> I think some high level command would be great, like changing 'save' button, to 'commit' with a custom message that can be generated by user.
> 
> We have talked about this before and we want to avoid creating a git
> UI in the notebook.  The other issue is that repo level actions are
> not per notebook, but per directory, so if we did add a commit button,
> it would need to be on the directory dashboard, not the notebook page.
> 
>> One of the things I came across while prototyping live collaboration is the real need to have the current state of the notebook on the kernel side.
>> I think we should define a format of notebook 'patches' that represent common operation on a ipynb file, at least:
>>        - appending/moving/deletting worksheet/cells
>> preferably with reference to the worksheet/cell by UUID.
> 
> I am not sure what you mean by having the notebook state on the
> kernel.  We have to be very careful to keep the notebook and kernel
> state completely separate.  Can you elaborate on why this was needed?
> But actually, can you open a separate IPEP for live collaboration - it
> is really a big topic in itself.

I'm loading notebook state_a, starting kernel.... I type, change on cell, 
continue... don't save.... I am on state_b
I invite you, you open your browser, server give you state_a.


>> I also think that for live collaboration we need both a 'cell is being edited' lock, and also maybe a 'cell is being executed' lock.
> 
> Yep, I think at the cell level that having locks like this makes sense
> for us.  The problem is doing so in an asynchronous setting (what if
> both of us lock the same cell before the server is notified?)

No problem, just don't set code_mirror.allow_edit until true once the response : "you were able to put the lock" arrives.
I did it in my prototype with 

request_lock(){

succes... callback:lock_granted()
error... callback:lock_refused()
}

-- 
Matthias



From bussonniermatthias at gmail.com  Sun Sep  9 12:56:40 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 9 Sep 2012 18:56:40 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAH4pYpSVyFCQX55ei_A9vSEdwGei+Gyi75hszYDzdFrFo+N9YQ@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAHAreOqQB9WcYAxLcuzkTDd+coR-gBSFbgqdMNKic8x6BVoWZQ@mail.gmail.com>
	<CAP-uhDf9eF18JE4O_H-iUpLSA4wXv4QO6Gv853=_qhhkeaJi2A@mail.gmail.com>
	<E3210741-7438-410A-A955-2F4A776A0D3C@gmail.com>
	<CAH4pYpSVyFCQX55ei_A9vSEdwGei+Gyi75hszYDzdFrFo+N9YQ@mail.gmail.com>
Message-ID: <D4A6FF37-CD14-48DF-B3B8-D4C79BE38CA5@gmail.com>


Le 9 sept. 2012 ? 18:19, Brian Granger a ?crit :

> On Sun, Sep 9, 2012 at 2:57 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> 
>> Le 8 sept. 2012 ? 23:09, Carl Smith a ?crit :
>> 
>>> I'm not sure, but I don't think you can do cross-scripting in Chrome. Maybe other browsers will make this concern mute too. I'm not certain, but that's what I thought. I'm on my phone so I can't do much to look into it right now.
>> 
>> Well, If I understand XSS it is pretty trivial to do with python notebook as we allow to embed script in .ipynb files.
>> They might not be executable but a simple notebook with in a markdown cell :
>> 
>> <a href='onclick=function(){alert('hello')}'> click me </a> should work.
>> moreover you can embed iframes, ...Etc.
>> 
>> So could forge a malicious ipynb file and ask you to view it through nbviewer.ipython.org.
>> 
>> assuming you are logged to nbviewer.ipython.org, the scripts in this notebook has all your rights.
>> 
>> For me, this is XSS.
>> 
>> I won't imagine what people would try to do if you know that JS can send code to execute on the server side !
> 
> I am not sure I am following the discussion here.  The IPython
> Notebook is designed to execute arbitrary python and javascript code.
> we don't view that as a security vulnerability - it is our central
> feature!  Is this what you are talking about or is there some other
> subtle aspect you are referring to?

I don't agree with : 
> The IPython Notebook is designed to execute arbitrary python and javascript code.

The user should be able to execute any python and javascript he wants. 
I don't want when loading a foreign notebook that some JS execute: "get cookie and send it to eve at hacked.me"
in multi-user environment, you have the user-context, and the notebook context. 
one supersede the other. 
I, from user-context, can interact with every notebook. 
Someone, from inside a notebook, should not be able to access user context. 

let's have a concrete example with nbviewer.
Which for the purpose of the example have log-in and ability for user to grant access to github, rw via OpenAuth, so that I can comment, edit from nbviewer, view their private notebooks... 

I can forge a notebook that on hover of the first cell send a bogus request to nbviewer, that look like user click asking a github to share your entire repos with me. I send it to you, and get access to the secret work you are doing with the DOD.


> One thing you might be talking about is the fact that <script> tags in
> markdown cells can be executed on page load without a user knowing
> about them.  The markdown renderer we are using has the ability to
> strip <script> tags.  We might want to do that so the only way that
> javascript code can be run is by the kernel running code that sends
> back dynamic javascript.  I am probably +0.5 on this idea currently.

Not only on load, but with live multi user on sync.
And it is not only the script tag, but js in href link. 
This also touch the output area that can have scripts in display_html.

Executing script on markdown is usefull as it is asynchrone. I had some good example of that during SciPy and enthough.
I would prefer sandboxing script execution.
-- 
Matthias



From benjaminrk at gmail.com  Sun Sep  9 14:22:55 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 9 Sep 2012 11:22:55 -0700
Subject: [IPython-dev] multiprocessing from within notebook crashes
	kernel
In-Reply-To: <CA+A4wOmg85DrpZYnwx_WSZUZ8y9fR9pTdK5-KZJisb0KsYfQmA@mail.gmail.com>
References: <CA+A4wOmg85DrpZYnwx_WSZUZ8y9fR9pTdK5-KZJisb0KsYfQmA@mail.gmail.com>
Message-ID: <CAHNn8BV+gEZ5OxLF=KQztkm=0Hh2V=tM_Zt2sxokowev_-s+PQ@mail.gmail.com>

What version of pyzmq?  If it's less than 2.2.0.1, try upgrading.
-MinRK

On Sun, Sep 9, 2012 at 6:35 AM, Satrajit Ghosh <satra at mit.edu> wrote:

> hi all,
>
> we are using a workflow library that internally uses multiprocessing to
> distribute jobs locally. when we run this from the notebook, it crashes the
> kernel.
>
> any thoughts as to why this might happen.
>
> here is the code from the library:
>
>
> https://github.com/nipy/nipype/blob/master/nipype/pipeline/plugins/multiproc.py
>
> cheers,
>
> satra
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/0200dbcd/attachment.html>

From benjaminrk at gmail.com  Sun Sep  9 14:22:55 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 9 Sep 2012 11:22:55 -0700
Subject: [IPython-dev] multiprocessing from within notebook crashes
	kernel
In-Reply-To: <CA+A4wOmg85DrpZYnwx_WSZUZ8y9fR9pTdK5-KZJisb0KsYfQmA@mail.gmail.com>
References: <CA+A4wOmg85DrpZYnwx_WSZUZ8y9fR9pTdK5-KZJisb0KsYfQmA@mail.gmail.com>
Message-ID: <CAHNn8BV+gEZ5OxLF=KQztkm=0Hh2V=tM_Zt2sxokowev_-s+PQ@mail.gmail.com>

What version of pyzmq?  If it's less than 2.2.0.1, try upgrading.
-MinRK

On Sun, Sep 9, 2012 at 6:35 AM, Satrajit Ghosh <satra at mit.edu> wrote:

> hi all,
>
> we are using a workflow library that internally uses multiprocessing to
> distribute jobs locally. when we run this from the notebook, it crashes the
> kernel.
>
> any thoughts as to why this might happen.
>
> here is the code from the library:
>
>
> https://github.com/nipy/nipype/blob/master/nipype/pipeline/plugins/multiproc.py
>
> cheers,
>
> satra
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/0200dbcd/attachment-0001.html>

From benjaminrk at gmail.com  Sun Sep  9 14:53:05 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 9 Sep 2012 11:53:05 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
	<D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>
Message-ID: <CAHNn8BW+6yM10Q4AHiab7w697zePLVHR9MDUZCZ4owucz39ScQ@mail.gmail.com>

On Sun, Sep 9, 2012 at 9:39 AM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

>
> Le 9 sept. 2012 ? 18:10, Brian Granger a ?crit :
>
> > Matthias,
> >
> >
> > On Sat, Sep 8, 2012 at 2:02 AM, Matthias BUSSONNIER
> > <bussonniermatthias at gmail.com> wrote:
> >>
> >>
> >> Comment on current IPEP version :
> >>
> >>>      ? Users will have the ability to "live share" notebooks and their
> kernels. When a user clicks "live share" they will get a private and secure
> URL they can share with other users. These other users can then edit the
> notebook and run code on the kernel in a Google Docs like manner.
> >>
> >> This is one feature, but I would expect the ability to invite/remove
> user independently.
> >> Either by being able to tick checkboxes, or else. But nothing annoy me
> more than having to look into my email to find the link.
> >>
> >> Ability to have "soft" more grained privileges. Invite a user than can
> only edit and not run.
> >
> > One thing we need to think through is the different "modes" a user can
> > view a page in.  Minimally we have three boolean flags that determine
> > the mode: read, write, execute.  Are there others?
>
> I see 5 different level of authorization
>
>         ? Ability to see a static version of the current saved, or
> specific version of the notebook (cf nbviewer).
>         ? Ability to see a live edited version of the notebook, without
> being able to interact.
>         ? Ability to edit part of the notebook and push the changes
>         ? Ability to save the current changes on disk
>         ? Ability to execute code on the kernel.
>

I don't think there's any reason to allow editing without saving.  That's
just confusing, so there's at most 4 here, and I'm not convinced that there
are even that many.  I only see:

1. read-only
2. write
3. execute

Now, we can decide whether read-only access gets live-updates or not, but I
cannot imagine a sensible scenario where an owner gives some users the
ability to watch changes, and others the ability to see the same thing only
if they keep hitting refresh.



>
>
> > One big thing we will have to work out is how to tune the
> > content/display of each page in a way that reflects these flags, but
> > that is secure.  Our current approach of setting a variable in JS to
> > control things is horribly insecure.  Ideas on this?
>

> We have to have more fine grained decorators in the handler side.
> Js side is just to make things pretty.
>

The JS code currently has nothing at all to do with security, it is only
there to help the UI be consistent with the permissions enforced on the
server.  If a read-only viewer executed JS to save or execute, they would
fail because they do not have permission.


>
> >
> >> Obviously we are really vulnerable to JS injection. We should
> definitely try to do something about that, even if we sell the notebook as
> beeing not totally secured. One thing we can use is html5 sandboxed feature
> of iframe. This could also solve the "problem" of css in ".rendered_html".
> >
> > I agree we need to think more about security, but that is really off
> > topic for this IPEP (i.e., the same technical problems exist for the
> > single user notebook).
>
> If you don't allow collaboration, then yes this is off topic.
> As soon as you add collaboration with username and password, you have a
> risk of user stealing each other account.
> We should think of it while building multi user account.
>

Collaboration is hugely complex, and I think it warrants an entirely
separate discussion.


>
> >
> >> Implementation wide for live collaborating, we should redirect the
> message using cell ID, otherwise this is a huge problem to redirect IOPub
> to all clients.
> >
> > I am not quite following you here.  Don't all clients need to get all
> > iopub messages?
>
> Yes they do.
> But they don't now how to handle it.
>
> if I run Safari and Chrome on "notebook_a"
> When I execute "1+1" on Safari, for cell 54, when iopub arrives on Chrome,
> it has no idea of what to do with it.
> I think we should proxy :
>
> Browser(s) ===>===> [execute+cell_id(+uuid ?)] ===>==> server strip
> cell_id (, add uuid) ? ==>==> kernel
>
> Browser(s) <===<==<===<===<===<==<===<===<=== server match uuid, append
> cell_id <==<=== kernel
>

I don't think we want to do this.


>
> >
> >>>      ? Version control of notebooks will be done using the command
> line interface to git and other VCSs. These VCS command will be issued
> through the shell widget.
> >>
> >>
> >> I think some high level command would be great, like changing 'save'
> button, to 'commit' with a custom message that can be generated by user.
> >
> > We have talked about this before and we want to avoid creating a git
> > UI in the notebook.  The other issue is that repo level actions are
> > not per notebook, but per directory, so if we did add a commit button,
> > it would need to be on the directory dashboard, not the notebook page.
> >
> >> One of the things I came across while prototyping live collaboration is
> the real need to have the current state of the notebook on the kernel side.
> >> I think we should define a format of notebook 'patches' that represent
> common operation on a ipynb file, at least:
> >>        - appending/moving/deletting worksheet/cells
> >> preferably with reference to the worksheet/cell by UUID.
> >
> > I am not sure what you mean by having the notebook state on the
> > kernel.  We have to be very careful to keep the notebook and kernel
> > state completely separate.  Can you elaborate on why this was needed?
> > But actually, can you open a separate IPEP for live collaboration - it
> > is really a big topic in itself.
>
> I'm loading notebook state_a, starting kernel.... I type, change on cell,
> continue... don't save.... I am on state_b
> I invite you, you open your browser, server give you state_a.
>
>
> >> I also think that for live collaboration we need both a 'cell is being
> edited' lock, and also maybe a 'cell is being executed' lock.
> >
> > Yep, I think at the cell level that having locks like this makes sense
> > for us.  The problem is doing so in an asynchronous setting (what if
> > both of us lock the same cell before the server is notified?)
>
> No problem, just don't set code_mirror.allow_edit until true once the
> response : "you were able to put the lock" arrives.
> I did it in my prototype with
>
> request_lock(){
>
> succes... callback:lock_granted()
> error... callback:lock_refused()
> }
>
> --
> Matthias
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/e608d4fe/attachment.html>

From bussonniermatthias at gmail.com  Sun Sep  9 15:36:58 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 9 Sep 2012 21:36:58 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAHNn8BW+6yM10Q4AHiab7w697zePLVHR9MDUZCZ4owucz39ScQ@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
	<D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>
	<CAHNn8BW+6yM10Q4AHiab7w697zePLVHR9MDUZCZ4owucz39ScQ@mail.gmail.com>
Message-ID: <46E67418-E346-4BBE-82F3-3788F414E1A6@gmail.com>

> 
> >
> > I agree we need to think more about security, but that is really off
> > topic for this IPEP (i.e., the same technical problems exist for the
> > single user notebook).
> 
> If you don't allow collaboration, then yes this is off topic.
> As soon as you add collaboration with username and password, you have a risk of user stealing each other account.
> We should think of it while building multi user account.
> 
> Collaboration is hugely complex, and I think it warrants an entirely separate discussion. 
> 
> >
> >> Implementation wide for live collaborating, we should redirect the message using cell ID, otherwise this is a huge problem to redirect IOPub to all clients.
> >
> > I am not quite following you here.  Don't all clients need to get all
> > iopub messages?
> 
> Yes they do.
> But they don't now how to handle it.
> 
> if I run Safari and Chrome on "notebook_a"
> When I execute "1+1" on Safari, for cell 54, when iopub arrives on Chrome, it has no idea of what to do with it.
> I think we should proxy :
> Browser(s) ===>===> [execute+cell_id(+uuid ?)] ===>==> server strip cell_id (, add uuid) ? ==>==> kernel
> 
> Browser(s) <===<==<===<===<===<==<===<===<=== server match uuid, append cell_id <==<=== kernel
> 
> I don't think we want to do this.

The problem I came across then she doing live collaboration, is others browsers not knowing for which cell iopub is meant.

* Browser send execution request, 
* IOPub and answer came back. 
* current cell/output to json, tagged with cell id. 
* send to the server, that redispatch to other Clients browser. 

I thought this was inefficient, especially depending on how you want syncing to be fine grained, you had to resend at "every" received iopub messages.
-- 
Matthias




From benjaminrk at gmail.com  Sun Sep  9 16:08:26 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 9 Sep 2012 13:08:26 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <46E67418-E346-4BBE-82F3-3788F414E1A6@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
	<D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>
	<CAHNn8BW+6yM10Q4AHiab7w697zePLVHR9MDUZCZ4owucz39ScQ@mail.gmail.com>
	<46E67418-E346-4BBE-82F3-3788F414E1A6@gmail.com>
Message-ID: <CAHNn8BUvHhgsiCKVtbOyBywVTMDJjxAqkhxT-wX_jpi6C_=TdA@mail.gmail.com>

On Sun, Sep 9, 2012 at 12:36 PM, Matthias BUSSONNIER <
bussonniermatthias at gmail.com> wrote:

> >
> > >
> > > I agree we need to think more about security, but that is really off
> > > topic for this IPEP (i.e., the same technical problems exist for the
> > > single user notebook).
> >
> > If you don't allow collaboration, then yes this is off topic.
> > As soon as you add collaboration with username and password, you have a
> risk of user stealing each other account.
> > We should think of it while building multi user account.
> >
> > Collaboration is hugely complex, and I think it warrants an entirely
> separate discussion.
> >
> > >
> > >> Implementation wide for live collaborating, we should redirect the
> message using cell ID, otherwise this is a huge problem to redirect IOPub
> to all clients.
> > >
> > > I am not quite following you here.  Don't all clients need to get all
> > > iopub messages?
> >
> > Yes they do.
> > But they don't now how to handle it.
> >
> > if I run Safari and Chrome on "notebook_a"
> > When I execute "1+1" on Safari, for cell 54, when iopub arrives on
> Chrome, it has no idea of what to do with it.
> > I think we should proxy :
> > Browser(s) ===>===> [execute+cell_id(+uuid ?)] ===>==> server strip
> cell_id (, add uuid) ? ==>==> kernel
> >
> > Browser(s) <===<==<===<===<===<==<===<===<=== server match uuid, append
> cell_id <==<=== kernel
> >
> > I don't think we want to do this.
>
> The problem I came across then she doing live collaboration, is others
> browsers not knowing for which cell iopub is meant.
>
> * Browser send execution request,
> * IOPub and answer came back.
> * current cell/output to json, tagged with cell id.
> * send to the server, that redispatch to other Clients browser.
>
> I thought this was inefficient, especially depending on how you want
> syncing to be fine grained, you had to resend at "every" received iopub
> messages.
>

Absolutely - resending *is* wasteful.  All frontends currently get all
input and output.  This means that to sync, you *do not* need to resend any
content, only IDs/locations, and the frontends can construct the elements
with the IDs, as they already have all of the content.


> --
> Matthias
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120909/16753637/attachment.html>

From ellisonbg at gmail.com  Sun Sep  9 16:13:50 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 9 Sep 2012 13:13:50 -0700
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <46E67418-E346-4BBE-82F3-3788F414E1A6@gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
	<D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>
	<CAHNn8BW+6yM10Q4AHiab7w697zePLVHR9MDUZCZ4owucz39ScQ@mail.gmail.com>
	<46E67418-E346-4BBE-82F3-3788F414E1A6@gmail.com>
Message-ID: <CAH4pYpQCXbowUDQoV6kGaSw5vjap+AVY=wnXmd5UMUCqZkt2Pg@mail.gmail.com>

On Sun, Sep 9, 2012 at 12:36 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>>
>> >
>> > I agree we need to think more about security, but that is really off
>> > topic for this IPEP (i.e., the same technical problems exist for the
>> > single user notebook).
>>
>> If you don't allow collaboration, then yes this is off topic.
>> As soon as you add collaboration with username and password, you have a risk of user stealing each other account.
>> We should think of it while building multi user account.
>>
>> Collaboration is hugely complex, and I think it warrants an entirely separate discussion.
>>
>> >
>> >> Implementation wide for live collaborating, we should redirect the message using cell ID, otherwise this is a huge problem to redirect IOPub to all clients.
>> >
>> > I am not quite following you here.  Don't all clients need to get all
>> > iopub messages?
>>
>> Yes they do.
>> But they don't now how to handle it.
>>
>> if I run Safari and Chrome on "notebook_a"
>> When I execute "1+1" on Safari, for cell 54, when iopub arrives on Chrome, it has no idea of what to do with it.
>> I think we should proxy :
>> Browser(s) ===>===> [execute+cell_id(+uuid ?)] ===>==> server strip cell_id (, add uuid) ? ==>==> kernel
>>
>> Browser(s) <===<==<===<===<===<==<===<===<=== server match uuid, append cell_id <==<=== kernel
>>
>> I don't think we want to do this.
>
> The problem I came across then she doing live collaboration, is others browsers not knowing for which cell iopub is meant.
>
> * Browser send execution request,
> * IOPub and answer came back.
> * current cell/output to json, tagged with cell id.
> * send to the server, that redispatch to other Clients browser.
>
> I thought this was inefficient, especially depending on how you want syncing to be fine grained, you had to resend at "every" received iopub messages.

OK I am following you now and agree that this is not very efficient.

There is also the issue of how to handle long running cells where a
user closes the notebook and then the result returns  over iopub.
Currently any output that is not loaded into an open notebook is lost.
 We do need something on the server side that *always* has the full
state of the notebook document.  My thought for this is that the part
of the server that handles live collaboration and saves, should
subscribe to iopub should put the data into the notebook document and
then broadcast it to clients.  If we move in that direction, the
notebook would no longer monitor iopub directly, but would instead
monitor diffs coming from the server.  But again, these are huge
issues that we need to discuss in a IPEP on the topics.

I think we will have to learn about Operational Transforms (used by
Google Docs and Wave).  ShareJS is an interesting start:

http://sharejs.org/

Cheers,

Brian

> --
> Matthias
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From stephen.pascoe at stfc.ac.uk  Mon Sep 10 05:18:40 2012
From: stephen.pascoe at stfc.ac.uk (stephen.pascoe at stfc.ac.uk)
Date: Mon, 10 Sep 2012 09:18:40 +0000
Subject: [IPython-dev] nbconvert HTML export
Message-ID: <D38782FE-222E-4ECE-8C4A-DA02202BCCDE@stfc.ac.uk>

I'm interested in contributing to nbconvert.  Particularly adding non-embedded images to the HTML exporter (saved into <nb>_files directory) and creating a headerless HTML exporter for injecting into blog posts.  I've browsed the code and have a couple of questions before I begin:

 1. What happened to the "quick-html" format?  It's in known_formats but not supported in main().  If there are plans to add this format what's it's intention.
 2. What is the preferred method of configuring embedded/non-embedded images?  Should I add a new format or a separate switch.  I prefer a switch.

I like to work via tests so I've fixed a few things in the test suite (https://github.com/ipython/nbconvert/pull/33)

Thanks,
Stephen.
-- 
Scanned by iCritical.


From stephen.pascoe at stfc.ac.uk  Mon Sep 10 05:43:25 2012
From: stephen.pascoe at stfc.ac.uk (stephen.pascoe at stfc.ac.uk)
Date: Mon, 10 Sep 2012 09:43:25 +0000
Subject: [IPython-dev] nbconvert HTML export
In-Reply-To: <D38782FE-222E-4ECE-8C4A-DA02202BCCDE@stfc.ac.uk>
References: <D38782FE-222E-4ECE-8C4A-DA02202BCCDE@stfc.ac.uk>
Message-ID: <23B0147C-38A3-4336-ABE4-F2D778526715@stfc.ac.uk>

Apologies, I see Carreau's branch addresses at least some these questions.  I'll take a look at it.

Stephen.


On 10 Sep 2012, at 10:18, <stephen.pascoe at stfc.ac.uk>
 <stephen.pascoe at stfc.ac.uk> wrote:

> I'm interested in contributing to nbconvert.  Particularly adding non-embedded images to the HTML exporter (saved into <nb>_files directory) and creating a headerless HTML exporter for injecting into blog posts.  I've browsed the code and have a couple of questions before I begin:
> 
> 1. What happened to the "quick-html" format?  It's in known_formats but not supported in main().  If there are plans to add this format what's it's intention.
> 2. What is the preferred method of configuring embedded/non-embedded images?  Should I add a new format or a separate switch.  I prefer a switch.
> 
> I like to work via tests so I've fixed a few things in the test suite (https://github.com/ipython/nbconvert/pull/33)
> 
> Thanks,
> Stephen.
> -- 
> Scanned by iCritical.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-- 
Scanned by iCritical.


From bussonniermatthias at gmail.com  Mon Sep 10 05:52:04 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 10 Sep 2012 11:52:04 +0200
Subject: [IPython-dev] nbconvert HTML export
In-Reply-To: <D38782FE-222E-4ECE-8C4A-DA02202BCCDE@stfc.ac.uk>
References: <D38782FE-222E-4ECE-8C4A-DA02202BCCDE@stfc.ac.uk>
Message-ID: <A40B3A2F-545D-40DD-BAEA-0B4CCC6BC3BC@gmail.com>


Le 10 sept. 2012 ? 11:18, <stephen.pascoe at stfc.ac.uk> a ?crit :

> I'm interested in contributing to nbconvert.  Particularly adding non-embedded images to the HTML exporter (saved into <nb>_files directory) and creating a headerless HTML exporter for injecting into blog posts.  I've browsed the code and have a couple of questions before I begin:
> 
> 1. What happened to the "quick-html" format?  It's in known_formats but not supported in main().  If there are plans to add this format what's it's intention.

I saw that too, and I don't know what it was supposed to do. 

> 2. What is the preferred method of configuring embedded/non-embedded images?  Should I add a new format or a separate switch.  I prefer a switch.

I would go with having a switch, ( you saw my PRs) 
But the question i, do we try something more Object oriented or more functional. 
You are welcome to work on it and send us PRs.

> I like to work via tests so I've fixed a few things in the test suite (https://github.com/ipython/nbconvert/pull/33)

We'll look into that when we have more time.

Thanks !
-- 
Matthias

From bussonniermatthias at gmail.com  Mon Sep 10 12:49:34 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 10 Sep 2012 18:49:34 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CAH4pYpQCXbowUDQoV6kGaSw5vjap+AVY=wnXmd5UMUCqZkt2Pg@mail.gmail.com>
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
	<504A4DC2.8050508@creativetrax.com>
	<CAH4pYpSx5UAuTPayUAYmZ5dfiF4jS0ArnZbmBSUC55mUEekbZg@mail.gmail.com>
	<504A7A3E.8040609@creativetrax.com>
	<CAHAreOpxrg6NbedG9oU+rEsHcpPPRDfY252YFu8u+u-a0LqqNQ@mail.gmail.com>
	<8F44A41C-A1BF-40B6-B707-F767DD6091CF@gmail.com>
	<CAH4pYpTkW4vZBgq8hcUixQn6XP7Pc2Np8NyNxBWcHqY=cyhT0g@mail.gmail.com>
	<D2309C60-9C52-4AC5-B45F-154D0EE2BA80@gmail.com>
	<CAHNn8BW+6yM10Q4AHiab7w697zePLVHR9MDUZCZ4owucz39ScQ@mail.gmail.com>
	<46E67418-E346-4BBE-82F3-3788F414E1A6@gmail.com>
	<CAH4pYpQCXbowUDQoV6kGaSw5vjap+AVY=wnXmd5UMUCqZkt2Pg@mail.gmail.com>
Message-ID: <FA586040-83C7-40AB-96DB-12576BC83F61@gmail.com>


Le 9 sept. 2012 ? 22:13, Brian Granger a ?crit :

>>> I don't think we want to do this.
>> 
>> The problem I came across then she doing live collaboration, is others browsers not knowing for which cell iopub is meant.
>> 
>> * Browser send execution request,
>> * IOPub and answer came back.
>> * current cell/output to json, tagged with cell id.
>> * send to the server, that redispatch to other Clients browser.
>> 
>> I thought this was inefficient, especially depending on how you want syncing to be fine grained, you had to resend at "every" received iopub messages.
> 
> OK I am following you now and agree that this is not very efficient.
> 
> There is also the issue of how to handle long running cells where a
> user closes the notebook and then the result returns  over iopub.
> Currently any output that is not loaded into an open notebook is lost.
> We do need something on the server side that *always* has the full
> state of the notebook document.  My thought for this is that the part
> of the server that handles live collaboration and saves, should
> subscribe to iopub should put the data into the notebook document and
> then broadcast it to clients.  If we move in that direction, the
> notebook would no longer monitor iopub directly, but would instead
> monitor diffs coming from the server.  But again, these are huge
> issues that we need to discuss in a IPEP on the topics.

I think you still will have to send some patches from browser to server as some js extension might do stuff to output. 

> 
> I think we will have to learn about Operational Transforms (used by
> Google Docs and Wave).  ShareJS is an interesting start:
> 
> http://sharejs.org/

This is cool !

But this will be quite difficult to integrate IMHO, 
we might really need a more MVC model for the notebook.
-- 
Matthias



From takowl at gmail.com  Tue Sep 11 16:53:42 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 11 Sep 2012 21:53:42 +0100
Subject: [IPython-dev] API documentation
Message-ID: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>

Fernando mentioned a couple of weeks ago the possibility of doing away
with our completely auto-generated API docs. I think having a degree
of manual control would be better for people who refer to the
documentation - as it can be clearer and better laid out - and for us,
in that it makes us think about what functions, methods and classes
comprise the public API.

I've made a start on that in this pull request:

https://github.com/ipython/ipython/pull/2383

It will be a fairly big job, so I wanted to discuss it before I spend
more time on it. I've used the auto-generated docs as a starting
point, and edited the rst. I've cut out a few helper classes and
functions, but if a class is included, all its methods are included
for now (except those marked _private).

Do you think it's worth replacing the auto-generated docs? If so, do
you think this is the right approach?

Thanks,
Thomas


From rhh at crosscompute.com  Tue Sep 11 16:50:50 2012
From: rhh at crosscompute.com (Roy Hyunjin Han)
Date: Tue, 11 Sep 2012 20:50:50 +0000 (UTC)
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
References: <CAH4pYpRxNwgZkU-K5+Tao2WH4o+snZmT5PBasWKYWdjLXQ_aaQ@mail.gmail.com>
Message-ID: <loom.20120911T223241-81@post.gmane.org>

> https://github.com/ipython/ipython/wiki/IPython-Enhancement-Proposals-(IPEPs)

Hi Brian,

I reviewed the current draft.  The following security requirements that you 
included are important:

- Map notebook users to shell users
- Run notebook kernels inside a shell user's process

Although live collaboration is fun to have, I want to make sure that the above 
security requirements are implemented and tested before live collaboration, 
because it is more straightforward from a security perspective.

I can devote resources to implementing the above two features.

RHH



P.S.

Here is our use case for a large organization:
- A single IPython notebook server
- Multiple users with different access privileges to datasets



From rhh at crosscompute.com  Tue Sep 11 16:55:33 2012
From: rhh at crosscompute.com (Roy Hyunjin Han)
Date: Tue, 11 Sep 2012 16:55:33 -0400
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
Message-ID: <CABScDVu+ysEjmWJB0oJFdvnf==1avT9VDwkAZYEbcp1NC8FHZA@mail.gmail.com>

> https://github.com/ipython/ipython/wiki/IPython-Enhancement-Proposals-(IPEPs)
> http://mail.scipy.org/pipermail/ipython-dev/2012-September/010189.html


Hi Brian,

I reviewed the draft.  The following security requirements that you
included are important:

- Map notebook users to shell users
- Run notebook kernels inside a shell user's process

Although live collaboration is fun to have, I want to make sure that
the above security requirements are implemented and tested before live
collaboration, because they is more straightforward from a security
perspective.

I can devote resources to implementing the above two features.

RHH



P.S.

Here is our use case for a large organization:
- A single IPython notebook server
- Multiple users with different access privileges to datasets


From jason-sage at creativetrax.com  Tue Sep 11 17:24:03 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 11 Sep 2012 16:24:03 -0500
Subject: [IPython-dev] Sage customizations
Message-ID: <504FABF3.6030503@creativetrax.com>

Fernando recently posted that we should be able to do the Sage 
customizations to ipython via just config changes and extensions.  As 
I'm getting deeper into what we do, I see the following obstacles. 
Should we be able to do these changes from config changes and extensions?

* interactiveshell system_raw function: Sage messes with the 
LD_LIBRARY_PATH so that programs included in Sage work correctly. 
However, if we're calling out to a system program installed outside of 
Sage, we need to reset the LD_LIBRARY_PATH.  So right now we override 
system_raw to set the correct LD_LIBRARY_PATH if we don't recognize the 
command as a sage-provided command.  What if we could have pre- and 
post- system_raw hooks, where we could run these commands?  I can 
imagine that these would also be useful if you wanted to always set a 
current directory for raw system commands, etc.  I suppose the same 
issue applies to the system_piped command.

* SageTerminalApp inherits from TerminalIPythonApp and overrides the 
crash handler to print out custom text for reporting crashes to Sage. 
Is there a way to get the app, given the shell?  If there was, I could 
override this as part of an extension, but I'm not sure if that's the 
most elegant way to do things.  Having a SageTerminalApp also lets us 
set the name to be "Sage", and just seems more natural in some ways.

Here are several sticky points that seem like they should be easier (or 
at least less controversial) to solve

* transformations are stateless (they only depend on a single line, and 
shouldn't store state) --- what is the rationale behind this?  For 
example, we'd like to do a dedenting transformation (useful if we are 
pasting lots of text, after we turn off autoindenting, or even if we are 
pasting a lot of single commands with autoindenting).  Of course, to do 
a dedenting transformation, we need to know the indentation of the first 
line, so it is not stateless.  Looking at the IPython code, it would be 
trivial to pass the line number in with the line string to transformations.

* oinspect -- we need to monkey-patch ipython to replace these functions:

IPython.core.oinspect.getdoc = sageinspect.sage_getdoc 
IPython.core.oinspect.getsource = sagedoc.my_getsource
IPython.core.oinspect.getargspec = sageinspect.sage_getargspec

because these functions are explicitly imported and used in various 
places in IPython.  Rather, could IPython use, say, something tied to an 
inspector object attached to the shell or something?  Then we could just 
set those attributes.

Comments are welcome!  I'm willing to help with pull requests for any of 
the items above that you guys are okay with changing.

Thanks,

Jason



From bussonniermatthias at gmail.com  Tue Sep 11 17:29:08 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Tue, 11 Sep 2012 23:29:08 +0200
Subject: [IPython-dev] IPEP 3: Multiuser support in the notebook
In-Reply-To: <CABScDVu+ysEjmWJB0oJFdvnf==1avT9VDwkAZYEbcp1NC8FHZA@mail.gmail.com>
References: <CABScDVu+ysEjmWJB0oJFdvnf==1avT9VDwkAZYEbcp1NC8FHZA@mail.gmail.com>
Message-ID: <D80FF3A4-C59A-46BE-B5B9-9DD5EAFEBA39@gmail.com>


Le 11 sept. 2012 ? 22:55, Roy Hyunjin Han a ?crit :

>> https://github.com/ipython/ipython/wiki/IPython-Enhancement-Proposals-(IPEPs)
>> http://mail.scipy.org/pipermail/ipython-dev/2012-September/010189.html
> 
> 
> Hi Brian,
> 
> I reviewed the draft.  The following security requirements that you
> included are important:
> 
> - Map notebook users to shell users
> - Run notebook kernels inside a shell user's process
> 
Hi Roy, 

Thanks for what you did. 

> Although live collaboration is fun to have, I want to make sure that
> the above security requirements are implemented and tested before live
> collaboration, because they is more straightforward from a security
> perspective.

Yes, I have to split everything Live Collaboration related to it's own IPEP, 
Don't have much time though. 

> I can devote resources to implementing the above two features.

We'll be happy to have help to get multi-user working, 
We are also dangerously approaching the 50+ opens PRs, 
So the risk of having merge conflict become more and 
more grater, so be carefull. 

-- 
Matthias



From carl.input at gmail.com  Tue Sep 11 17:42:15 2012
From: carl.input at gmail.com (Carl Smith)
Date: Tue, 11 Sep 2012 22:42:15 +0100
Subject: [IPython-dev] API documentation
In-Reply-To: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>
References: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>
Message-ID: <CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>

I mentioned elsewhere that I'd be happy to do some work on the docs
wherever I can be useful. I'm still totally up for this, but something
really exciting came up this weekend. In short, I now need to raise
?3,000 in thirty days or miss out on a chance to start a small firm
providing remote, time-share access to humanoid robots!

Raising the money isn't going to be easy for me to do, so I've got my
work cut out.

There seems to be some issues you guys need to iron out before too
much can be done on documentation anyway.

Sorry to step up, then step back again, but I am still keen to
contribute in this area.

Carl


From takowl at gmail.com  Tue Sep 11 17:59:21 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 11 Sep 2012 22:59:21 +0100
Subject: [IPython-dev] Sage customizations
In-Reply-To: <504FABF3.6030503@creativetrax.com>
References: <504FABF3.6030503@creativetrax.com>
Message-ID: <CAOvn4qjA8paLRSX9NhFeuymFMS68cD-mY95vH-G5dtaxDkJY2A@mail.gmail.com>

On 11 September 2012 22:24, Jason Grout <jason-sage at creativetrax.com> wrote:
> * transformations are stateless (they only depend on a single line, and
> shouldn't store state) --- what is the rationale behind this?  For
> example, we'd like to do a dedenting transformation (useful if we are
> pasting lots of text, after we turn off autoindenting, or even if we are
> pasting a lot of single commands with autoindenting).  Of course, to do
> a dedenting transformation, we need to know the indentation of the first
> line, so it is not stateless.  Looking at the IPython code, it would be
> trivial to pass the line number in with the line string to transformations.

Input transformations are still 'in progress' under IPEP 2. Sorry, I
haven't had much time to work on it lately. Please keep telling me
about use cases like this - I am building up ideas for how the
transformation machinery should work, although it's not in a concrete
form just yet.

Thanks,
Thomas


From takowl at gmail.com  Tue Sep 11 18:41:32 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 11 Sep 2012 23:41:32 +0100
Subject: [IPython-dev] API documentation
In-Reply-To: <CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>
References: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>
	<CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>
Message-ID: <CAOvn4qjZ5kXCPabCe6EZfaxdRdAnYRat=Ka6rf_8JOcmGgNynA@mail.gmail.com>

On 11 September 2012 22:42, Carl Smith <carl.input at gmail.com> wrote:
> In short, I now need to raise
> ?3,000 in thirty days or miss out on a chance to start a small firm
> providing remote, time-share access to humanoid robots!

Cool, good luck with that! And don't worry, I'm sure there'll always
be stuff to do on IPython whenever you have time. ;-)

Thomas


From jason-sage at creativetrax.com  Tue Sep 11 18:55:30 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 11 Sep 2012 17:55:30 -0500
Subject: [IPython-dev] Sage customizations
In-Reply-To: <CAOvn4qjA8paLRSX9NhFeuymFMS68cD-mY95vH-G5dtaxDkJY2A@mail.gmail.com>
References: <504FABF3.6030503@creativetrax.com>
	<CAOvn4qjA8paLRSX9NhFeuymFMS68cD-mY95vH-G5dtaxDkJY2A@mail.gmail.com>
Message-ID: <504FC162.6010102@creativetrax.com>

On 9/11/12 4:59 PM, Thomas Kluyver wrote:
> On 11 September 2012 22:24, Jason Grout <jason-sage at creativetrax.com> wrote:
>> * transformations are stateless (they only depend on a single line, and
>> shouldn't store state) --- what is the rationale behind this?  For
>> example, we'd like to do a dedenting transformation (useful if we are
>> pasting lots of text, after we turn off autoindenting, or even if we are
>> pasting a lot of single commands with autoindenting).  Of course, to do
>> a dedenting transformation, we need to know the indentation of the first
>> line, so it is not stateless.  Looking at the IPython code, it would be
>> trivial to pass the line number in with the line string to transformations.
>
> Input transformations are still 'in progress' under IPEP 2. Sorry, I
> haven't had much time to work on it lately. Please keep telling me
> about use cases like this - I am building up ideas for how the
> transformation machinery should work, although it's not in a concrete
> form just yet.
>

Thanks.  I know you've been working on this.

I submitted a pull request illustrating the changes I'm talking about. 
I'm sure there are other places in the code that would need to be 
changed for this to be a comprehensive patch.

https://github.com/ipython/ipython/pull/2402

Thanks,

Jason




From michael.k.bilow at gmail.com  Wed Sep 12 13:17:58 2012
From: michael.k.bilow at gmail.com (Michael Bilow)
Date: Wed, 12 Sep 2012 10:17:58 -0700
Subject: [IPython-dev] Starting an IPython cluster from within a Python
	script
Message-ID: <CAGMfGm18X3eLAmb3dRs-Z8O=1K-ERhHGJ7JmC8K3CYMf3tNMGw@mail.gmail.com>

Quick question:

When I try to start an IPython cluster from a python script, for example,
using os.system("ipcluster start -n 6"), the process just seems to hang.  I
think this is because the process is waiting for a return value from the
call to os.system, which never comes.  Starting an ipcluster in another
shell window works fine.  Is there a way (or is it not recommended) to
start ipclusters within a Python script?

Thanks!

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120912/e7cd2e25/attachment.html>

From brad.froehle at gmail.com  Wed Sep 12 13:37:59 2012
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Wed, 12 Sep 2012 10:37:59 -0700
Subject: [IPython-dev] Starting an IPython cluster from within a Python
	script
In-Reply-To: <CAGMfGm18X3eLAmb3dRs-Z8O=1K-ERhHGJ7JmC8K3CYMf3tNMGw@mail.gmail.com>
References: <CAGMfGm18X3eLAmb3dRs-Z8O=1K-ERhHGJ7JmC8K3CYMf3tNMGw@mail.gmail.com>
Message-ID: <CAHXv-MgNksBoTK9M2sC64Qq8F++mh42kAPRZS8L43f1dDdRZMA@mail.gmail.com>

Try using the subprocess module.  For example:

import subprocess
child = subprocess.Popen("ipcluster start -n 6", shell=True)

You can read the help for information on how to avoid printing output
from the script, waiting for the process to complete, etc.

import os
import subprocess
devnull = open(os.devnull, 'w')
child = subprocess.Popen(['ipcluster', 'start', '-n', '6'], stdout=devnull)

-Brad

On Wed, Sep 12, 2012 at 10:17 AM, Michael Bilow
<michael.k.bilow at gmail.com> wrote:
> Quick question:
>
> When I try to start an IPython cluster from a python script, for example,
> using os.system("ipcluster start -n 6"), the process just seems to hang.  I
> think this is because the process is waiting for a return value from the
> call to os.system, which never comes.  Starting an ipcluster in another
> shell window works fine.  Is there a way (or is it not recommended) to start
> ipclusters within a Python script?
>
> Thanks!
>
> Michael
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From epatters at enthought.com  Wed Sep 12 16:14:39 2012
From: epatters at enthought.com (Evan Patterson)
Date: Wed, 12 Sep 2012 15:14:39 -0500
Subject: [IPython-dev] IPython In-process kernel support
In-Reply-To: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
References: <CAOvn4qh5aKWPFdM=PwpvBLTz=TMYii_NFtJ-=t6G8yqr6Zmkiw@mail.gmail.com>
Message-ID: <CAFAPEOVdmoUuip7vD-7PN6s7xZLs9cxEsYivWSx7j5vB10jzVA@mail.gmail.com>

Hi everyone,

A revised pull request for in-process support has been submitted:

https://github.com/ipython/ipython/pull/2397

This PR is much less intrusive than my previous one, so I hope that it will
be merged shortly. Feedback and testing would be appreciated.

Thanks,
Evan

On Thu, Sep 6, 2012 at 3:34 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> Hi Pierre,
>
> There's a pull request open to allow the IPython Qt console to use an
> in-process kernel. I thought you might be interested in using something
> like this in Spyder. If you'd like to take a look, we'd welcome any
> feedback:
>
> https://github.com/ipython/ipython/pull/2382
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120912/b9f6601d/attachment.html>

From carl.input at gmail.com  Wed Sep 12 16:31:32 2012
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 12 Sep 2012 21:31:32 +0100
Subject: [IPython-dev] API documentation
In-Reply-To: <CAOvn4qjZ5kXCPabCe6EZfaxdRdAnYRat=Ka6rf_8JOcmGgNynA@mail.gmail.com>
References: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>
	<CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>
	<CAOvn4qjZ5kXCPabCe6EZfaxdRdAnYRat=Ka6rf_8JOcmGgNynA@mail.gmail.com>
Message-ID: <CAP-uhDdrQVeLm5RUR-5wVgFYBn1az-T8=YSHW4DMJUBF88A1Rw@mail.gmail.com>

Cheers Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120912/8ab4eb5a/attachment.html>

From fperez.net at gmail.com  Thu Sep 13 01:41:13 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 12 Sep 2012 22:41:13 -0700
Subject: [IPython-dev] API documentation
In-Reply-To: <CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>
References: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>
	<CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>
Message-ID: <CAHAreOp0DHi-pMdGnrCNe16UE3=QhsZJHCSm3ptsb+jomHnJXg@mail.gmail.com>

On Tue, Sep 11, 2012 at 2:42 PM, Carl Smith <carl.input at gmail.com> wrote:
> Sorry to step up, then step back again, but I am still keen to
> contribute in this area.

No worries, as Thomas said, best of luck and we'll always be here for
when you have some bandwidth.

Cheers,

f


From carl.input at gmail.com  Thu Sep 13 05:12:37 2012
From: carl.input at gmail.com (Carl Smith)
Date: Thu, 13 Sep 2012 10:12:37 +0100
Subject: [IPython-dev] API documentation
In-Reply-To: <CAHAreOp0DHi-pMdGnrCNe16UE3=QhsZJHCSm3ptsb+jomHnJXg@mail.gmail.com>
References: <CAOvn4qhTn=jf-T3cT8ss8J6X2ZjywE1jGvHWZfp1p=0U4rMPSg@mail.gmail.com>
	<CAP-uhDfYrPbBGAti-2_pBNnW2mvkP2MEz12kY78AnAoUwxffRw@mail.gmail.com>
	<CAHAreOp0DHi-pMdGnrCNe16UE3=QhsZJHCSm3ptsb+jomHnJXg@mail.gmail.com>
Message-ID: <CAP-uhDfzDZOLo7ti6ubTg8-EBzpSZvLQeV-0cqqsmsnBdLvWnQ@mail.gmail.com>

Nice one. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120913/524ee8b2/attachment.html>

From jason-sage at creativetrax.com  Thu Sep 13 07:14:36 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 13 Sep 2012 06:14:36 -0500
Subject: [IPython-dev] Scipy 2012 Kudos
In-Reply-To: <CAM-+wY8jR8xwHO1FTmifn1div4yg_Tmw0Lesf-M59hniPsD9zQ@mail.gmail.com>
References: <CAM-+wY9eMvtqXc04CFxkEpiusxtZeFNHW9HPXsEqV5nLMSpr+w@mail.gmail.com>
	<502FC3E3.9050504@creativetrax.com>
	<CAM-+wY8jR8xwHO1FTmifn1div4yg_Tmw0Lesf-M59hniPsD9zQ@mail.gmail.com>
Message-ID: <5051C01C.9020108@creativetrax.com>

On 8/18/12 12:04 PM, Warren Weckesser wrote:
>     I just checked the scipy 2012 website, and while many talks have the
>     videos up, I can't find a video for the IPython tutorial [1].  Are there
>     plans to put up the video for that?  (no pressure, I was just
>     wondering...)
>
>
>
> Yes, the missing videos for several of the tutorials and talks will be
> available /real soon now/?.  There will also be an index for the talks
> at pyvideo.org <http://pyvideo.org>.

I'm curious if finishing posting the videos (in particular, the IPython 
parallelism one [1]) is still on the radar for someone.  No pressure---I 
just really wanted to send a link to the IPython tutorial to someone.

I'm happy to help transcode/post videos, if it helps.

Thanks,

Jason

[1] 
http://conference.scipy.org/scipy2012/schedule/tutor_schedule_2.php#ti-73



From fperez.net at gmail.com  Thu Sep 13 14:47:41 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 13 Sep 2012 11:47:41 -0700
Subject: [IPython-dev] Scipy 2012 Kudos
In-Reply-To: <5051C01C.9020108@creativetrax.com>
References: <CAM-+wY9eMvtqXc04CFxkEpiusxtZeFNHW9HPXsEqV5nLMSpr+w@mail.gmail.com>
	<502FC3E3.9050504@creativetrax.com>
	<CAM-+wY8jR8xwHO1FTmifn1div4yg_Tmw0Lesf-M59hniPsD9zQ@mail.gmail.com>
	<5051C01C.9020108@creativetrax.com>
Message-ID: <CAHAreOrX_8TWq6Hma_0Ed8SjYFstMcvJa1YDJ8UGW5dQKvV_aQ@mail.gmail.com>

Hi Jason,

On Thu, Sep 13, 2012 at 4:14 AM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> I'm curious if finishing posting the videos (in particular, the IPython
> parallelism one [1]) is still on the radar for someone.  No pressure---I
> just really wanted to send a link to the IPython tutorial to someone.

They did get posted, but the pyvideo index hasn't been updated.
Thanks for reminding us, at least we can advertise these links while
pyvideo catches up.

Here are the links:

IPython: tools for the entire lifecycle of research computing
20 minute talk at Scipy'12
Video: http://www.youtube.com/watch?v=iwVvqwLDsJo&feature=plcp
Slides: https://speakerdeck.com/u/fperez/p/ipython-tools-for-the-lifecycle-of-research-computing

IPython in-depth: Interactive Tools for Scientific Computing
2 hour tutorial at Scipy'12
Video: http://www.youtube.com/watch?v=AvknbGH01FE&feature=plcp
Repository with all materials: https://github.com/ipython/ipython-in-depth


Cheers,

f


From aron at ahmadia.net  Thu Sep 13 14:53:31 2012
From: aron at ahmadia.net (Aron Ahmadia)
Date: Thu, 13 Sep 2012 21:53:31 +0300
Subject: [IPython-dev] Scipy 2012 Kudos
In-Reply-To: <5051C01C.9020108@creativetrax.com>
References: <CAM-+wY9eMvtqXc04CFxkEpiusxtZeFNHW9HPXsEqV5nLMSpr+w@mail.gmail.com>
	<502FC3E3.9050504@creativetrax.com>
	<CAM-+wY8jR8xwHO1FTmifn1div4yg_Tmw0Lesf-M59hniPsD9zQ@mail.gmail.com>
	<5051C01C.9020108@creativetrax.com>
Message-ID: <CAPhiW4i_ZYh8U7bzYnCVFnij2h9bcCNNH6VLN+zddRUztODyMw@mail.gmail.com>

Let me take advantage of the attention on parallelism in IPython to
point out that Andy, Travis Oliphant, and I will be leading a tutorial
at Supercomputing this year that will be taught out of IPython
notebooks with a focus on performance and scalability.  I encourage
anybody who wants to know more details about how to participate
(either as a student or a volunteer) to ping me or Andy directly.

Thanks,
Aron

http://sc12.supercomputing.org/schedule/event_detail.php?evid=tut176

On Thu, Sep 13, 2012 at 2:14 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> On 8/18/12 12:04 PM, Warren Weckesser wrote:
>>     I just checked the scipy 2012 website, and while many talks have the
>>     videos up, I can't find a video for the IPython tutorial [1].  Are there
>>     plans to put up the video for that?  (no pressure, I was just
>>     wondering...)
>>
>>
>>
>> Yes, the missing videos for several of the tutorials and talks will be
>> available /real soon now/?.  There will also be an index for the talks
>> at pyvideo.org <http://pyvideo.org>.
>
> I'm curious if finishing posting the videos (in particular, the IPython
> parallelism one [1]) is still on the radar for someone.  No pressure---I
> just really wanted to send a link to the IPython tutorial to someone.
>
> I'm happy to help transcode/post videos, if it helps.
>
> Thanks,
>
> Jason
>
> [1]
> http://conference.scipy.org/scipy2012/schedule/tutor_schedule_2.php#ti-73
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From fperez.net at gmail.com  Thu Sep 13 15:02:13 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 13 Sep 2012 12:02:13 -0700
Subject: [IPython-dev] Scipy 2012 Kudos
In-Reply-To: <CAPhiW4i_ZYh8U7bzYnCVFnij2h9bcCNNH6VLN+zddRUztODyMw@mail.gmail.com>
References: <CAM-+wY9eMvtqXc04CFxkEpiusxtZeFNHW9HPXsEqV5nLMSpr+w@mail.gmail.com>
	<502FC3E3.9050504@creativetrax.com>
	<CAM-+wY8jR8xwHO1FTmifn1div4yg_Tmw0Lesf-M59hniPsD9zQ@mail.gmail.com>
	<5051C01C.9020108@creativetrax.com>
	<CAPhiW4i_ZYh8U7bzYnCVFnij2h9bcCNNH6VLN+zddRUztODyMw@mail.gmail.com>
Message-ID: <CAHAreOquesKmONFP9V0wirEkfugPHZX2bf9DmcqE75UrhvLuqA@mail.gmail.com>

On Thu, Sep 13, 2012 at 11:53 AM, Aron Ahmadia <aron at ahmadia.net> wrote:
> Let me take advantage of the attention on parallelism in IPython to
> point out that Andy, Travis Oliphant, and I will be leading a tutorial
> at Supercomputing this year that will be taught out of IPython
> notebooks with a focus on performance and scalability.  I encourage
> anybody who wants to know more details about how to participate
> (either as a student or a volunteer) to ping me or Andy directly.

And as a reminder, if you need to show any colleague a notebook that's
publicly available on the internet, you can use the IPython nbviewer
that Matthias created, simply prepend

http://nbviewer.ipython.org/urls/

to your URL and you're good to go.  Example, using the intro tour from our repo:

http://nbviewer.ipython.org/urls/raw.github.com/ipython/ipython/master/docs/examples/notebooks/00_notebook_tour.ipynb

Cheers,

f


From aron at ahmadia.net  Sat Sep 15 15:35:38 2012
From: aron at ahmadia.net (Aron Ahmadia)
Date: Sat, 15 Sep 2012 22:35:38 +0300
Subject: [IPython-dev] embedded image urls (markdown,
	others) and nbviewer/print
Message-ID: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>

We're "printing" our slides out for SC12 (they're due today), and I
noticed that images don't show up properly through nbviewer if they
are coming from Markdown.  They also appear to be broken from a normal
dialogue.  What's the correct way to export an IPython notebook that
uses Markdown images?

Cheers,
A


From aron at ahmadia.net  Sat Sep 15 15:39:56 2012
From: aron at ahmadia.net (Aron Ahmadia)
Date: Sat, 15 Sep 2012 22:39:56 +0300
Subject: [IPython-dev] embedded image urls (markdown,
	others) and nbviewer/print
In-Reply-To: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>
References: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>
Message-ID: <CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>

Apparently, normal markdown/html is insensitive to a "/files" URL vs
"files", however the preview markup requires:

"/files"

I'm not sure what the intended behavior is.

A

On Sat, Sep 15, 2012 at 10:35 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
> We're "printing" our slides out for SC12 (they're due today), and I
> noticed that images don't show up properly through nbviewer if they
> are coming from Markdown.  They also appear to be broken from a normal
> dialogue.  What's the correct way to export an IPython notebook that
> uses Markdown images?
>
> Cheers,
> A


From benjaminrk at gmail.com  Sat Sep 15 15:50:26 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 15 Sep 2012 12:50:26 -0700
Subject: [IPython-dev] embedded image urls (markdown,
	others) and nbviewer/print
In-Reply-To: <CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>
References: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>
	<CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>
Message-ID: <CAHNn8BVoqgww0o3j+d5vKtZ5SupOyj-76vrwh+H-MK=+BoMwfA@mail.gmail.com>

On Sat, Sep 15, 2012 at 12:39 PM, Aron Ahmadia <aron at ahmadia.net> wrote:

> Apparently, normal markdown/html is insensitive to a "/files" URL vs
> "files", however the preview markup requires:
>
> "/files"
>
> I'm not sure what the intended behavior is.
>

This should be considered a bug.  "files" (relative) is correct because
"/files" (absolute) will break if/when url schemes change such that the
notebook is served from non-root url (as is likely when multi-user is
introduced).


>
> A
>
> On Sat, Sep 15, 2012 at 10:35 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
> > We're "printing" our slides out for SC12 (they're due today), and I
> > noticed that images don't show up properly through nbviewer if they
> > are coming from Markdown.  They also appear to be broken from a normal
> > dialogue.  What's the correct way to export an IPython notebook that
> > uses Markdown images?
> >
> > Cheers,
> > A
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120915/bb91e570/attachment.html>

From bussonniermatthias at gmail.com  Sat Sep 15 18:02:08 2012
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Sun, 16 Sep 2012 00:02:08 +0200
Subject: [IPython-dev] embedded image urls (markdown,
	others) and nbviewer/print
In-Reply-To: <CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>
References: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>
	<CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>
Message-ID: <CANJQusWy+pxUUJQ3RVm79Qh7e7vxXap9w0a37n+D0zmMjcGRBg@mail.gmail.com>

Due to nbviewer architecture, i would say that only embedded images or
images with full url will show.  I think you'll have to use nbconvert from
the command line.

You could open an issue on github about that if you wish.

You can also look at vIPer by damian avila if my memory is correct that
allows to export slides.from notebook. You'll have to search a little by
yourself, i'm on my phone right now, sorry.

Good luck.

-- 
Matthias
Le 15 sept. 2012 21:40, "Aron Ahmadia" <aron at ahmadia.net> a ?crit :

> Apparently, normal markdown/html is insensitive to a "/files" URL vs
> "files", however the preview markup requires:
>
> "/files"
>
> I'm not sure what the intended behavior is.
>
> A
>
> On Sat, Sep 15, 2012 at 10:35 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
> > We're "printing" our slides out for SC12 (they're due today), and I
> > noticed that images don't show up properly through nbviewer if they
> > are coming from Markdown.  They also appear to be broken from a normal
> > dialogue.  What's the correct way to export an IPython notebook that
> > uses Markdown images?
> >
> > Cheers,
> > A
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120916/de04bdec/attachment.html>

From aron at ahmadia.net  Sat Sep 15 18:40:14 2012
From: aron at ahmadia.net (Aron Ahmadia)
Date: Sun, 16 Sep 2012 01:40:14 +0300
Subject: [IPython-dev] embedded image urls (markdown,
	others) and nbviewer/print
In-Reply-To: <CANJQusWy+pxUUJQ3RVm79Qh7e7vxXap9w0a37n+D0zmMjcGRBg@mail.gmail.com>
References: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>
	<CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>
	<CANJQusWy+pxUUJQ3RVm79Qh7e7vxXap9w0a37n+D0zmMjcGRBg@mail.gmail.com>
Message-ID: <CAPhiW4gh49vBwnsirkgMLAx4Cu4sgvD=fpg6WhwTdpyYKX=vhQ@mail.gmail.com>

Thanks for the tips, I ended up getting it submitted by doing manual
print view (it looked decent from Chrome).

A

On Sun, Sep 16, 2012 at 1:02 AM, Matthias Bussonnier
<bussonniermatthias at gmail.com> wrote:
> Due to nbviewer architecture, i would say that only embedded images or
> images with full url will show.  I think you'll have to use nbconvert from
> the command line.
>
> You could open an issue on github about that if you wish.
>
> You can also look at vIPer by damian avila if my memory is correct that
> allows to export slides.from notebook. You'll have to search a little by
> yourself, i'm on my phone right now, sorry.
>
> Good luck.
>
> --
> Matthias
>
> Le 15 sept. 2012 21:40, "Aron Ahmadia" <aron at ahmadia.net> a ?crit :
>>
>> Apparently, normal markdown/html is insensitive to a "/files" URL vs
>> "files", however the preview markup requires:
>>
>> "/files"
>>
>> I'm not sure what the intended behavior is.
>>
>> A
>>
>> On Sat, Sep 15, 2012 at 10:35 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
>> > We're "printing" our slides out for SC12 (they're due today), and I
>> > noticed that images don't show up properly through nbviewer if they
>> > are coming from Markdown.  They also appear to be broken from a normal
>> > dialogue.  What's the correct way to export an IPython notebook that
>> > uses Markdown images?
>> >
>> > Cheers,
>> > A
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From bussonniermatthias at gmail.com  Sun Sep 16 08:11:18 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 16 Sep 2012 14:11:18 +0200
Subject: [IPython-dev] embedded image urls (markdown,
	others) and nbviewer/print
In-Reply-To: <CAHNn8BVoqgww0o3j+d5vKtZ5SupOyj-76vrwh+H-MK=+BoMwfA@mail.gmail.com>
References: <CAPhiW4juewmrArAW_Us8YOCSJCTAVdPqG0N0AzgY9zPGaVn3_g@mail.gmail.com>
	<CAPhiW4hrP_JMXAcTVb0__TvNz2hBuV7W-i6EvWs8LX+1rSHoog@mail.gmail.com>
	<CAHNn8BVoqgww0o3j+d5vKtZ5SupOyj-76vrwh+H-MK=+BoMwfA@mail.gmail.com>
Message-ID: <78BB3CE7-2179-4430-8E8C-9846C0C71B4C@gmail.com>


Le 15 sept. 2012 ? 21:50, MinRK a ?crit :

> 
> 
> On Sat, Sep 15, 2012 at 12:39 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
> Apparently, normal markdown/html is insensitive to a "/files" URL vs
> "files", however the preview markup requires:
> 
> "/files"
> 
> I'm not sure what the intended behavior is.
> 
> This should be considered a bug.  "files" (relative) is correct because "/files" (absolute) will break if/when url schemes change such that the notebook is served from non-root url (as is likely when multi-user is introduced). 

The problem with nbviewer is that you can ask it to render almost any url, 
it has no way to know wether "files" in the requested url is legitimate or not. 

assuming I am rendering the file 

http://my/super/project/with/a/notebook.ipynb
through the url 
http://nbviewer.ipython.orl/url/my/super/project/with/a/notebook.ipynb

I have a link to the image <img src='./my/image.png'>

nbviewer is asked to get

http://nbviewer.ipython.orl/url/my/super/priject/with/a/files/my/image.png
(dev version is smart enough to know that this is png and don't crash and return raw data, which current version does not) 

But `files` can be anywhere in the url. Blindly stripping it would prevent any url containing legitimates 'files' part from working. 

Though it is still a bug which should be easier to fix once nbconvert has been refactored.
-- 
Matthias

> 
> A
> 
> On Sat, Sep 15, 2012 at 10:35 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
> > We're "printing" our slides out for SC12 (they're due today), and I
> > noticed that images don't show up properly through nbviewer if they
> > are coming from Markdown.  They also appear to be broken from a normal
> > dialogue.  What's the correct way to export an IPython notebook that
> > uses Markdown images?
> >
> > Cheers,
> > A
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120916/a77136a6/attachment.html>

From dchichkov at gmail.com  Tue Sep 18 18:42:06 2012
From: dchichkov at gmail.com (Dmitry Chichkov)
Date: Tue, 18 Sep 2012 15:42:06 -0700
Subject: [IPython-dev] Anyone encountering scrolling/"jumpy" issues in the
	notebook?
Message-ID: <CADeuJh6LZ=sqPCeWex1SvBVggxT3ktYh4fhZFhR5zjj2zdyL-g@mail.gmail.com>

Hello,

I'm consistently encountering a problem that a notebook (dev .13) is very
'jumpy'.  That is, after I'm executing some cell and then returning the
focus to that cell teh notebook scrolls to the start of the cell.

Can be reproduced in the following way:
 1. add like 100 empty lines to a code cell (so it wouldn't fit into the
screen);
 2. add "print" statement;
 3. run it with ctrl-enter;
 4. select this print statement with a mouse;

As a result the notebook will scroll to the beginning of the code cell.
And _selected_ print statement would _completely_ disappear  from view.
Could anyone reproduce that? Or is it just me?

If yes, well, as a rule anything selected should never dissapear from view.
And in general it is a bad idea to scroll, unless user explicitly scrolls...

Best,
Dmitry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120918/c7ee9297/attachment.html>

From tjhnson at gmail.com  Wed Sep 19 03:49:37 2012
From: tjhnson at gmail.com (T J)
Date: Wed, 19 Sep 2012 02:49:37 -0500
Subject: [IPython-dev] IPython Notebook on Webfaction
Message-ID: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>

Hi,

I'm looking to set up an ipython notebook over SSL on Webfaction [1], a
shared-hosting service.  There was a previous inquiry [2] to this list, but
that thread did not involve SSL.

With Webfaction, all SSL negotiations happen on the frontend Nginx server.
 Once that finishes, all requests are forwarded to the app as regular HTTP.
 Practically, this means there is no need to tell IPython about my
key/certificate.  All I do is configure my application in Webfaction's
control panel so that it is served on HTTPS and then run the IPython
notebook "unencrypted".  [Correct me if this is wrong.]

After doing this, I was able to connect without receiving any 502 errors,
but then I was receiving "404: Not Found", just as in [2].  The problem
seems to be that nginx does not support the websocket protocol...and the
IPython notebook uses this protocol through tornado.

As in [2], the situation is fixable.  Webfaction will provide an IP address
and provide a single port that receives HTTPS requests directly (without a
proxy).  Note that in this situation it is necessary to provide my
key/certificate to IPython so that communications are encrypted.

So now we come to the last of it:  Webfaction provides a single port.  On:

    http://community.webfaction.com/questions/9320/ipython-notebook

the last response from the OP mentions that each open notebook uses 3
random ports (which differ from the port that the IPython notebook server
listens on).  Is this true?  This seems like it would be an issue for me.
 Can someone explain, in general, what IPython notebook is doing with these
other ports?

I can probably get Webfaction to open up a fixed port range for me.  Is the
behavior which governs port selection configurable? What I'd like to do is
tell IPython notebook that it can only use ports in a particular range and
if there aren't enough available, then a new notebook cannot be opened.

------
[1] http://www.webfaction.com/
[2]
http://python.6.n6.nabble.com/ipython-on-webfaction-hosting-tt4979054.html#none
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120919/91656996/attachment.html>

From takowl at gmail.com  Wed Sep 19 05:49:27 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 19 Sep 2012 10:49:27 +0100
Subject: [IPython-dev] IPython Notebook on Webfaction
In-Reply-To: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
References: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
Message-ID: <CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>

On 19 September 2012 08:49, T J <tjhnson at gmail.com> wrote:
> the last response from the OP mentions that each open notebook uses 3 random
> ports (which differ from the port that the IPython notebook server listens
> on).  Is this true?  This seems like it would be an issue for me.  Can
> someone explain, in general, what IPython notebook is doing with these other
> ports?

One of them is the main request/response channel (e.g. 'execute this
code', 'get completions for foo'), one is a publication channel,
primarily for things printed to stdout, and I think the third is to
get input for stdin (although the notebook doesn't currently implement
that).

There's a pull request open that experiments with merging the three
websocket channels into one - that might help you:
https://github.com/ipython/ipython/pull/2321

Best wishes,
Thomas


From tjhnson at gmail.com  Wed Sep 19 12:32:11 2012
From: tjhnson at gmail.com (T J)
Date: Wed, 19 Sep 2012 11:32:11 -0500
Subject: [IPython-dev] IPython Notebook on Webfaction
In-Reply-To: <CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>
References: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
	<CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>
Message-ID: <CAJ8p+o8zWR+ex2YjLZNp+e=eYiHFHrOw_sjVGdBRbk-f+Z45AQ@mail.gmail.com>

On Wed, Sep 19, 2012 at 4:49 AM, Thomas Kluyver <takowl at gmail.com> wrote:

> On 19 September 2012 08:49, T J <tjhnson at gmail.com> wrote:
> > the last response from the OP mentions that each open notebook uses 3
> random
> > ports (which differ from the port that the IPython notebook server
> listens
> > on).  Is this true?  This seems like it would be an issue for me.  Can
> > someone explain, in general, what IPython notebook is doing with these
> other
> > ports?
>
> One of them is the main request/response channel (e.g. 'execute this
> code', 'get completions for foo'), one is a publication channel,
> primarily for things printed to stdout, and I think the third is to
> get input for stdin (although the notebook doesn't currently implement
> that).
>
> There's a pull request open that experiments with merging the three
> websocket channels into one - that might help you:
> https://github.com/ipython/ipython/pull/2321
>
>
>
Look promising.  This pull requests seems to make it easier to use the
notebook behind a proxy, but its unclear to me if this allows me to
restrict which the ports the notebooks will try to use.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120919/90d2e85c/attachment.html>

From ellisonbg at gmail.com  Wed Sep 19 12:34:51 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 19 Sep 2012 09:34:51 -0700
Subject: [IPython-dev] Anyone encountering scrolling/"jumpy" issues in
	the notebook?
In-Reply-To: <CADeuJh6LZ=sqPCeWex1SvBVggxT3ktYh4fhZFhR5zjj2zdyL-g@mail.gmail.com>
References: <CADeuJh6LZ=sqPCeWex1SvBVggxT3ktYh4fhZFhR5zjj2zdyL-g@mail.gmail.com>
Message-ID: <CAH4pYpR=eLd56m7Empi9aBmxz9Omt1m7JXpGGFRRUcJQecqFsQ@mail.gmail.com>

Dmitry,

Hi,

On Tue, Sep 18, 2012 at 3:42 PM, Dmitry Chichkov <dchichkov at gmail.com> wrote:
> Hello,
>
> I'm consistently encountering a problem that a notebook (dev .13) is very
> 'jumpy'.  That is, after I'm executing some cell and then returning the
> focus to that cell teh notebook scrolls to the start of the cell.
>
> Can be reproduced in the following way:
>  1. add like 100 empty lines to a code cell (so it wouldn't fit into the
> screen);
>  2. add "print" statement;
>  3. run it with ctrl-enter;
>  4. select this print statement with a mouse;
>
> As a result the notebook will scroll to the beginning of the code cell.  And
> _selected_ print statement would _completely_ disappear  from view.
> Could anyone reproduce that? Or is it just me?

This is a combination of your using ctrl-enter and a bug in how cells
are focused.  We think we know how to fix these things (there was a PR
that started to address it), but it is going to require refactoring
our model of how cells are focused and selected.

> If yes, well, as a rule anything selected should never dissapear from view.
> And in general it is a bad idea to scroll, unless user explicitly scrolls...

Yep, for sure.

Cheers,

Brian

> Best,
> Dmitry
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From d.warde.farley at gmail.com  Wed Sep 19 17:11:16 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Wed, 19 Sep 2012 17:11:16 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
Message-ID: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>

So, this is partly caused by the details of PEP 370, but I'm raising
it in an issue because it seems like there might be some way to
rectify it on IPython's side.

Essentially, our sysadmin has 0.13 installed in a site-packages
directory that he adds to the PYTHONPATH via global bashrc stuff (I
could remove it in my own ~/.bashrc, but it's a pain). I wanted to use
git master, so I installed it with "python setup.py develop --user".
What I didn't realize until just now reading PEP 370 is that
~/.local/foo is scanned *after* PYTHONPATH. So if I simply issue
"python" and "import IPython", I get 0.13.

Now, here's the fun part: because of some pkg_resources funny
business, when I do "ipython" and launch the terminal version, I get
0.14.dev. However, if I run "ipython notebook" or "ipython qtconsole",
I get all sorts of errors printing to the c onsole referencing the
version on the PYTHONPATH. I assume this has to do with the kernel
process forking off, and whatever pkg_resources funny-business not
being inherited in that process.

So, the straightforward answer is for me to just prepend ~/src/ipython
to my PYTHONPATH, but I'm a little weary of this situation of
IPython-terminal and IPython-client-server behaving differently even
when invoked with the same script.

So, is there any way we can ensure that both processes are loading the
same version of the IPython module?

Cheers,

David


From takowl at gmail.com  Wed Sep 19 17:30:25 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 19 Sep 2012 22:30:25 +0100
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
Message-ID: <CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>

On 19 September 2012 22:11, David Warde-Farley <d.warde.farley at gmail.com> wrote:
> So, is there any way we can ensure that both processes are loading the
> same version of the IPython module?

I'm sure it's possible - for instance, the parent process could
serialise its sys.path, to be recreated in the child process. I'm not
sure whether we want to introduce that sort of measure, though - the
situation where 'import IPython' doesn't get the IPython version
you're using seems a bit pathological.

Thomas


From d.warde.farley at gmail.com  Wed Sep 19 17:49:49 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Wed, 19 Sep 2012 17:49:49 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
Message-ID: <CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>

On Wed, Sep 19, 2012 at 5:30 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> I'm sure it's possible - for instance, the parent process could
> serialise its sys.path, to be recreated in the child process. I'm not
> sure whether we want to introduce that sort of measure, though - the
> situation where 'import IPython' doesn't get the IPython version
> you're using seems a bit pathological.

That isn't the situation, not quite. In a normal Python REPL:

>>> import IPython
>>> IPython.__version__
'0.13'

In IPython launched from ~/.local/bin/ipython (by virtue of
pkg_resources.load_entry_point):

In [1]: import IPython

In [2]: IPython.__version__
Out[2]: '0.14.dev'

It's only when IPython spawns another process that things fall on
their face. Namely, the client remains 0.14.dev but the server imports
0.13, so the client is sending messages that the server can't
decipher, I get a lot of "Invalid signature" and "malformed message"
stuff.

David


From brad.froehle at gmail.com  Wed Sep 19 18:28:56 2012
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Wed, 19 Sep 2012 15:28:56 -0700
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
Message-ID: <CAHXv-MjKHJExEU2jriPKjJOKhrozHkfchOfiksGwN8n3df-p2Q@mail.gmail.com>

FWIW, this has bitten me enough times that I no longer use `python
setupegg.py develop` with IPython.

On Wed, Sep 19, 2012 at 2:49 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> That isn't the situation, not quite. In a normal Python REPL:
>
>>>> import IPython
>>>> IPython.__version__
> '0.13'
>
> In IPython launched from ~/.local/bin/ipython (by virtue of
> pkg_resources.load_entry_point):
>
> In [1]: import IPython
>
> In [2]: IPython.__version__
> Out[2]: '0.14.dev'
>
> It's only when IPython spawns another process that things fall on
> their face. Namely, the client remains 0.14.dev but the server imports
> 0.13, so the client is sending messages that the server can't
> decipher, I get a lot of "Invalid signature" and "malformed message"
> stuff.
>
> David
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From benjaminrk at gmail.com  Thu Sep 20 15:42:26 2012
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 20 Sep 2012 12:42:26 -0700
Subject: [IPython-dev] IPython Notebook on Webfaction
In-Reply-To: <CAJ8p+o8zWR+ex2YjLZNp+e=eYiHFHrOw_sjVGdBRbk-f+Z45AQ@mail.gmail.com>
References: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
	<CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>
	<CAJ8p+o8zWR+ex2YjLZNp+e=eYiHFHrOw_sjVGdBRbk-f+Z45AQ@mail.gmail.com>
Message-ID: <CAHNn8BWHkvvB+3HSfx=aJFuEpLqytzYPmzLMRmj0v-N_72qzdQ@mail.gmail.com>

On Wed, Sep 19, 2012 at 9:32 AM, T J <tjhnson at gmail.com> wrote:

> On Wed, Sep 19, 2012 at 4:49 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>
>> On 19 September 2012 08:49, T J <tjhnson at gmail.com> wrote:
>> > the last response from the OP mentions that each open notebook uses 3
>> random
>> > ports (which differ from the port that the IPython notebook server
>> listens
>> > on).  Is this true?  This seems like it would be an issue for me.  Can
>> > someone explain, in general, what IPython notebook is doing with these
>> other
>> > ports?
>>
>> One of them is the main request/response channel (e.g. 'execute this
>> code', 'get completions for foo'), one is a publication channel,
>> primarily for things printed to stdout, and I think the third is to
>> get input for stdin (although the notebook doesn't currently implement
>> that).
>>
>> There's a pull request open that experiments with merging the three
>> websocket channels into one - that might help you:
>> https://github.com/ipython/ipython/pull/2321
>>
>>
>>
> Look promising.  This pull requests seems to make it easier to use the
> notebook behind a proxy, but its unclear to me if this allows me to
> restrict which the ports the notebooks will try to use.
>

People get confused by the fact that the notebook communicates with
*kernels* via random localhost ports (even there, it connects, it does not
listen).  All communications with browsers happen over a single port, and
the notebook server only ever listens on that single user-configurable port.


>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120920/31f8d143/attachment.html>

From fperez.net at gmail.com  Fri Sep 21 05:37:30 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 21 Sep 2012 02:37:30 -0700
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
Message-ID: <CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>

On Wed, Sep 19, 2012 at 2:49 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> It's only when IPython spawns another process that things fall on
> their face. Namely, the client remains 0.14.dev but the server imports
> 0.13, so the client is sending messages that the server can't
> decipher, I get a lot of "Invalid signature" and "malformed message"
> stuff.

Yup.  Just a few days ago I had to fix this very problem for a
colleague (though in her case, caused by a system-wide EPD install
where setuptools was overriding any local PYTHONPATH configuration).
After some frustration, I learned of the file 'usercustomize.py':

http://docs.python.org/library/site.html

Put a file with this name *anywhere* in your PYTHONPATH, and fix
sys.path manually in there to reorder things as needed, e.g.

sys.path = [os.environ['HOME']+'.local/lib/python2.7/site-packages/'] + sys.path

and you're done.

HTH,

f


From jason-sage at creativetrax.com  Sat Sep 22 01:26:36 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 22 Sep 2012 00:26:36 -0500
Subject: [IPython-dev] slow startup due to script magic
Message-ID: <505D4C0C.50009@creativetrax.com>

I was looking into why aleph.sagemath.org is so slow in responding to 
user requests (i.e., it takes a second or so to respond to any new 
request), and it turned out that the major slowdown was starting up a 
new IPython kernel.  We eventually traced it down to the loading of the 
script extension, which by default does 6 popen calls to find the paths 
of scripts in the _script_magics_default method, which are really slow 
(something like 200 ms for the calls on my computer):

https://github.com/ipython/ipython/blob/master/IPython/core/magics/script.py#L92 
and diving deeper, 
https://github.com/ipython/ipython/blob/master/IPython/utils/_process_posix.py#L35

If I move the tests up into the module level so that they are run when 
the module is loaded, rather than every time a new kernel is created, 
the kernel creation time drops dramatically, and in the end, our 
response time for aleph is cut in 1/2 or 1/3.

What can we do to optimize the guessing of where different commands 
live?  Here are at least a few ways:

1. explicitly specify a script_paths dictionary (just curious, what key 
would I make in the ipython config file to specify that?  I'm still a 
little lost when it comes to figuring out how keys relate to actual 
classes that are invoked.)

2. Move the guessing up into the module level, cached in a module-level 
variable.  While this doesn't help in the case where you only launch one 
kernel, it dramatically speeds up things if you are launching multiple 
kernels.

Any other ideas?  It seems such a shame that loading a rather esoteric, 
but standard, extension should double or so the startup time of an 
IPython kernel.

Thanks,

Jason


From takowl at gmail.com  Sat Sep 22 06:40:39 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 22 Sep 2012 11:40:39 +0100
Subject: [IPython-dev] slow startup due to script magic
In-Reply-To: <505D4C0C.50009@creativetrax.com>
References: <505D4C0C.50009@creativetrax.com>
Message-ID: <CAOvn4qitPTtx26TzuWqpthh3Az7Sz0b6+TrBCFe6KBhU3ajHwg@mail.gmail.com>

On 22 September 2012 06:26, Jason Grout <jason-sage at creativetrax.com> wrote:
> We eventually traced it down to the loading of the
> script extension, which by default does 6 popen calls to find the paths
> of scripts in the _script_magics_default method, which are really slow
> (something like 200 ms for the calls on my computer):

Hmm, well spotted. If it's that slow, I think we should avoid doing it
during IPython start up at all. I like it to start up quickly.

Maybe we should keep it simple: rather than checking for the existence
of each interpreter, just set a standard list. It would mean that
users can tab complete e.g. %%pypy, even if they don't have pypy
installed, but I don't think it will be too surprising if they try it
and it fails.

Thomas


From takowl at gmail.com  Sat Sep 22 11:04:56 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 22 Sep 2012 16:04:56 +0100
Subject: [IPython-dev] [SciPy-User] Pylab - standard packages
In-Reply-To: <CABL7CQiVOFt3et34C=gC0+OTuT3sciMXr32iWK4bQ0P8gHkSzQ@mail.gmail.com>
References: <mailman.903.1348015544.18920.scipy-user@scipy.org>
	<wl8mx0muxdc.fsf@physics.ucf.edu>
	<CAOvn4qipdPzuUC5MVq9mNcgeOcSZ64f-EsHH27Xe=kt6fVXx=g@mail.gmail.com>
	<CAHAreOpyKx8_SUUzEKNnS6rC57coceOUrE-54iG3QRFgaXS7jw@mail.gmail.com>
	<CAOvn4qg1EaxeDm9RHviH47r8Gn9iqA9KMNL-Jyz4e_LE7=WY7w@mail.gmail.com>
	<CAHAreOqGRfz72o5AEqzBngYXfb0mfOPEXCFXHCERgp4JXK4fBg@mail.gmail.com>
	<CAPJVwBnLmOL+JMk32c2vfTaeX1Orym7bd9USJryD5Qwob4CyTA@mail.gmail.com>
	<CAHAreOpdeYiS1AK0FbLcFuum5n=FzBjvh9P5_VjzjDXobUiYBw@mail.gmail.com>
	<CAOvn4qgpT6TV0BFHLwGfSyFORpy1Y9RkHGsF05a8tMhFBca8qw@mail.gmail.com>
	<CAHAreOoJsrPeog=k1fz7cLc-Wneje+xRJo0AKjk6eVuWOf_zVA@mail.gmail.com>
	<CAB_T6=n0-nKFyTn-=4B8QxHR9KLE2TkJvZ3n93ALd-wx0QqHdg@mail.gmail.com>
	<CAHAreOpSOJbPDQE8U0TUjMCzemPiLWD+9xh6EPe=5T9mBp9Wdw@mail.gmail.com>
	<CAB_T6=kk--VK-X5eGJkZ8n+uJa8D0Q3gJ=aHh+DEh9OzPEmZkg@mail.gmail.com>
	<CAOvn4qguy1MBhZ+ZT9ibX1Ww6B3G9sD1Q-DcUS0YFjqNgPQOng@mail.gmail.com>
	<CAB_T6=m9Hr6ASPDVCgUpqUKkV8=hDwEVo0XHGwxw-+xvvJf35g@mail.gmail.com>
	<CAOvn4qirx64Pnokv2A28pCPTQQEmexW-hUtRUgmVPzaMSEVHvw@mail.gmail.com>
	<CABL7CQjHEXTsFnOQ_s+rixkw5CgSG_eeFf-_GoeP4qwtRthgkw@mail.gmail.com>
	<CAOvn4qhfzg2yVkraBDALKewG97=pnLrznLsNG5gVwqQS5Kki0w@mail.gmail.com>
	<CABL7CQiVOFt3et34C=gC0+OTuT3sciMXr32iWK4bQ0P8gHkSzQ@mail.gmail.com>
Message-ID: <CAOvn4qiZUrzit1XoPKORpPfeAKJTPLKQ=1P7ZgZiobipdJmVkQ@mail.gmail.com>

On 22 September 2012 14:46, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> I think you have forwards and backwards compatible the wrong way around.
> Forwards compatible would be that you can open version 4 files, if that's
> introduced in IPython 0.14 or later, in 0.13.

You're quite right, I had the terms backwards. Sorry about that; I do
find them confusing.

> In my opinion you need either
> forward compatibility or by default keep saving in version 3 even after you
> have version 4, if you want to make the format a standard.

Well, if we make version 4, it would be because we were making a
change that couldn't be opened by the v3 code - that's what the
version numbers are for. But, as I understand it, we've made v3
considerably more extensible, so most of the things we've considered
adding wouldn't need a version number bump.

In the future, I envisage that if/when we define a new version, we'll
have a transition period where the notebook can open version n+1, but
defaults to saving as version n. Then once that ability is widespread,
we'd make the next release save as n+1 by default.

I've CCed the ipython-dev list: Fernando, Brian, Min, can you give
some more insight on how stable the notebook format is now?

Thanks,
Thomas


From fperez.net at gmail.com  Sat Sep 22 14:33:10 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 22 Sep 2012 11:33:10 -0700
Subject: [IPython-dev] slow startup due to script magic
In-Reply-To: <CAOvn4qitPTtx26TzuWqpthh3Az7Sz0b6+TrBCFe6KBhU3ajHwg@mail.gmail.com>
References: <505D4C0C.50009@creativetrax.com>
	<CAOvn4qitPTtx26TzuWqpthh3Az7Sz0b6+TrBCFe6KBhU3ajHwg@mail.gmail.com>
Message-ID: <CAHAreOr3LVqeO70Eu81wx2yCKYLUwwFbhyXgHaeOYNExSRVO6g@mail.gmail.com>

On Sat, Sep 22, 2012 at 3:40 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Hmm, well spotted. If it's that slow, I think we should avoid doing it
> during IPython start up at all. I like it to start up quickly.

Absolutely, thanks for catching this, Jason!  Indeed, startup time is
a really important consideration, and we should be very careful with
it.

Does anyone have a tool to instrument the startup process in an easy
manner?  Jason, if you guys have any such tools for Sage that you
could share, we could track these things and prevent similar
regressions in the future.

> Maybe we should keep it simple: rather than checking for the existence
> of each interpreter, just set a standard list. It would mean that
> users can tab complete e.g. %%pypy, even if they don't have pypy
> installed, but I don't think it will be too surprising if they try it
> and it fails.

We could also cache the search path and do it only on first usage,
rather than at IPython startup.  Else we can discuss moving this to an
optional extension...

Cheers,

f


From tjhnson at gmail.com  Sat Sep 22 14:54:14 2012
From: tjhnson at gmail.com (T J)
Date: Sat, 22 Sep 2012 13:54:14 -0500
Subject: [IPython-dev] IPython Notebook on Webfaction
In-Reply-To: <CAHNn8BWHkvvB+3HSfx=aJFuEpLqytzYPmzLMRmj0v-N_72qzdQ@mail.gmail.com>
References: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
	<CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>
	<CAJ8p+o8zWR+ex2YjLZNp+e=eYiHFHrOw_sjVGdBRbk-f+Z45AQ@mail.gmail.com>
	<CAHNn8BWHkvvB+3HSfx=aJFuEpLqytzYPmzLMRmj0v-N_72qzdQ@mail.gmail.com>
Message-ID: <CAJ8p+o8VTPgYJVpG_Lzxpus+n7FUDcQ62tdeX=c3HMErJfuJ0Q@mail.gmail.com>

On Thu, Sep 20, 2012 at 2:42 PM, MinRK <benjaminrk at gmail.com> wrote:

>
>
> On Wed, Sep 19, 2012 at 9:32 AM, T J <tjhnson at gmail.com> wrote:
>
>> On Wed, Sep 19, 2012 at 4:49 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>>
>>> On 19 September 2012 08:49, T J <tjhnson at gmail.com> wrote:
>>> > the last response from the OP mentions that each open notebook uses 3
>>> random
>>> > ports (which differ from the port that the IPython notebook server
>>> listens
>>> > on).  Is this true?  This seems like it would be an issue for me.  Can
>>> > someone explain, in general, what IPython notebook is doing with these
>>> other
>>> > ports?
>>>
>>> One of them is the main request/response channel (e.g. 'execute this
>>> code', 'get completions for foo'), one is a publication channel,
>>> primarily for things printed to stdout, and I think the third is to
>>> get input for stdin (although the notebook doesn't currently implement
>>> that).
>>>
>>> There's a pull request open that experiments with merging the three
>>> websocket channels into one - that might help you:
>>> https://github.com/ipython/ipython/pull/2321
>>>
>>>
>>>
>> Look promising.  This pull requests seems to make it easier to use the
>> notebook behind a proxy, but its unclear to me if this allows me to
>> restrict which the ports the notebooks will try to use.
>>
>
> People get confused by the fact that the notebook communicates with
> *kernels* via random localhost ports (even there, it connects, it does not
> listen).  All communications with browsers happen over a single port, and
> the notebook server only ever listens on that single user-configurable port.
>


Thanks for the clarification. Very helpful.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120922/a504b1d3/attachment.html>

From jason-sage at creativetrax.com  Sat Sep 22 21:19:53 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 22 Sep 2012 20:19:53 -0500
Subject: [IPython-dev] slow startup due to script magic
In-Reply-To: <CAHAreOr3LVqeO70Eu81wx2yCKYLUwwFbhyXgHaeOYNExSRVO6g@mail.gmail.com>
References: <505D4C0C.50009@creativetrax.com>
	<CAOvn4qitPTtx26TzuWqpthh3Az7Sz0b6+TrBCFe6KBhU3ajHwg@mail.gmail.com>
	<CAHAreOr3LVqeO70Eu81wx2yCKYLUwwFbhyXgHaeOYNExSRVO6g@mail.gmail.com>
Message-ID: <505E63B9.2080607@creativetrax.com>

On 9/22/12 1:33 PM, Fernando Perez wrote:
> On Sat, Sep 22, 2012 at 3:40 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> Hmm, well spotted. If it's that slow, I think we should avoid doing it
>> during IPython start up at all. I like it to start up quickly.
>
> Absolutely, thanks for catching this, Jason!  Indeed, startup time is
> a really important consideration, and we should be very careful with
> it.
>
> Does anyone have a tool to instrument the startup process in an easy
> manner?  Jason, if you guys have any such tools for Sage that you
> could share, we could track these things and prevent similar
> regressions in the future.

We do have a sage -startuptime script that (I think) uses an import hook 
to track the time to import the many modules that Sage imports.  It 
sounds like that may not be as useful to you guys, since you don't 
import nearly as many things as we do.  And that's mainly for us to 
diagnose startup time regressions by hand.

I suppose it would be pretty straightforward to add a nose test that 
would check the time to start a full kernel, right?


>
>> Maybe we should keep it simple: rather than checking for the existence
>> of each interpreter, just set a standard list. It would mean that
>> users can tab complete e.g. %%pypy, even if they don't have pypy
>> installed, but I don't think it will be too surprising if they try it
>> and it fails.
>
> We could also cache the search path and do it only on first usage,
> rather than at IPython startup.  Else we can discuss moving this to an
> optional extension...

It's also less confusing to have a standard list that is always 
available.  Otherwise I have to try to guess why in the world I have 
%%ruby, but you don't.

I'd say that %%ruby should invoke ruby with "/bin/env ruby" and leave it 
at that.  Why do we need to cache the path?

Thanks,

Jason



From cloverevolc at yahoo.com  Sun Sep 23 05:57:42 2012
From: cloverevolc at yahoo.com (Caleb)
Date: Sun, 23 Sep 2012 02:57:42 -0700 (PDT)
Subject: [IPython-dev] nbconvert to latex with no output cell
Message-ID: <1348394262.95041.YahooMailNeo@web142304.mail.bf1.yahoo.com>

Hi all,

I am new to this mailing list, so hi to everyone.
Here is my first question:

I use nbconvert(master branch) to convert my notebook to latex file, but
somehow all my output cells are missing.

Any idea?

cheers
Cal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120923/db0e2a28/attachment.html>

From fperez.net at gmail.com  Sun Sep 23 02:11:12 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 22 Sep 2012 23:11:12 -0700
Subject: [IPython-dev] Sage customizations
In-Reply-To: <504FABF3.6030503@creativetrax.com>
References: <504FABF3.6030503@creativetrax.com>
Message-ID: <CAHAreOpObDobxOHwXE1+i=hT5rJvUPANuHJKdUjih+STGD-3HA@mail.gmail.com>

[ Finishing an old draft... ]

On Tue, Sep 11, 2012 at 2:24 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> Fernando recently posted that we should be able to do the Sage
> customizations to ipython via just config changes and extensions.  As
> I'm getting deeper into what we do, I see the following obstacles.
> Should we be able to do these changes from config changes and extensions?

Perhaps I should have hedged slightly: you should be able to do *most*
of Sage as config/extensions.  It's obviously possible that you will
come up with something that you guys want to do that we decide isn't
worth another config/extension point, or that proves more practical
via subclassing.  But at least bringing these points up here will let
us understand those boundaries and draw them deliberately, rather than
have them be whatever the vagaries of historical evolution dictate.

> * interactiveshell system_raw function: Sage messes with the
> LD_LIBRARY_PATH so that programs included in Sage work correctly.
> However, if we're calling out to a system program installed outside of
> Sage, we need to reset the LD_LIBRARY_PATH.  So right now we override
> system_raw to set the correct LD_LIBRARY_PATH if we don't recognize the
> command as a sage-provided command.  What if we could have pre- and
> post- system_raw hooks, where we could run these commands?  I can
> imagine that these would also be useful if you wanted to always set a
> current directory for raw system commands, etc.  I suppose the same
> issue applies to the system_piped command.

I don't know if adding more hooks is the right answer; interaction
with the OS is a messy affair, partly because there are subtle
differences between platforms and the results aren't the same
depending on whether you use os.system, subprocess.* or pexpect.  In
ipython, over the years all the code to manage that has grown a little
weedy and scattered (though there have been cleanups, so it's not a
complete mess either).

What I'd like to do is to have an IPEP where we rationalize the entire
design of how we call to the OS: we define the various special
syntaxes for it explicitly, indicate which tools are used under the
hood to to the job, etc.  Having such a document lay out the
boundaries of this problem explicitly, would make it much easier then
to consider what the best solutions for user customizations are.
Perhaps in the end it will be hooks, perhaps not.  But I'd like to see
the whole chessboard in one shot, including a listing of the various
needs (such as yours) before adding more knobs to tweak.

> * SageTerminalApp inherits from TerminalIPythonApp and overrides the
> crash handler to print out custom text for reporting crashes to Sage.
> Is there a way to get the app, given the shell?  If there was, I could
> override this as part of an extension, but I'm not sure if that's the
> most elegant way to do things.  Having a SageTerminalApp also lets us
> set the name to be "Sage", and just seems more natural in some ways.

apps are meant to be per-process singletons, so you can get it this way:

In [1]: from IPython.frontend.terminal.ipapp import TerminalIPythonApp

In [2]: TerminalIPythonApp.instance()
Out[2]: <IPython.frontend.terminal.ipapp.TerminalIPythonApp at 0x7f6095cc3190>


> Here are several sticky points that seem like they should be easier (or
> at least less controversial) to solve
>
> * transformations are stateless (they only depend on a single line, and
> shouldn't store state) --- what is the rationale behind this?  For
> example, we'd like to do a dedenting transformation (useful if we are
> pasting lots of text, after we turn off autoindenting, or even if we are
> pasting a lot of single commands with autoindenting).  Of course, to do
> a dedenting transformation, we need to know the indentation of the first
> line, so it is not stateless.  Looking at the IPython code, it would be
> trivial to pass the line number in with the line string to transformations.

The rationale was simplicity: if transformations are stateless and
purely functional, they can be tested in isolation and composed in
different ways without side effects.  State means potential for side
effects, weird behavior arising from seemingly innocuous problems,
etc.  Obviously some times state *is* necessary in a problem, and
objects are one approach to manage state with some degree of control
and containment.  But they also raise the complexity bar significantly
for a design.

Over the years, I've become a big fan of keeping software designs as
purely functional as possible, layering object orientation and state
only in a controlled manner.  In this case, we should look precisely
at what information would be necessary to have for your use cases, and
think about the simplest design to achieve that.  I'm not convinced
that turning the transformations system into a big state machine is a
price worth paying.

> * oinspect -- we need to monkey-patch ipython to replace these functions:
>
> IPython.core.oinspect.getdoc = sageinspect.sage_getdoc
> IPython.core.oinspect.getsource = sagedoc.my_getsource
> IPython.core.oinspect.getargspec = sageinspect.sage_getargspec
>
> because these functions are explicitly imported and used in various
> places in IPython.  Rather, could IPython use, say, something tied to an
> inspector object attached to the shell or something?  Then we could just
> set those attributes.

Well, but we already use an object, it's called a module :) I'm only
half-joking here: you're asking us to artificially bundle a group of
functions that otherwise have no reason to share state whatsoever,
whose signature doesn't need a 'self' anywhere, so that you can then
grab that object and modify it.  But python offers precisely one
abstraction for grouping related functions that otherwise need to
share no state together: the module!  And it's an object, whose
attributes can be set by users who need to modify the default behavior
of the object.  I don't see the point in creating another layer so
that you can set your changes in a new object instead of the module,
frankly.


We certainly want to make the customization and building of derivative
systems as clean as possible, so this kind of discussion is very
important.  We'll have to iterate to find a balanced design, and in
the end it may be that for something as complex as Sage a subclass is
still the right approach and not simply a rich profile.  I hope this
helps!

Cheers,

f


From bussonniermatthias at gmail.com  Sun Sep 23 06:24:37 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 23 Sep 2012 12:24:37 +0200
Subject: [IPython-dev] nbconvert to latex with no output cell
In-Reply-To: <1348394262.95041.YahooMailNeo@web142304.mail.bf1.yahoo.com>
References: <1348394262.95041.YahooMailNeo@web142304.mail.bf1.yahoo.com>
Message-ID: <E3D3D52F-99D6-458A-AD09-7F864966B7FF@gmail.com>


Le 23 sept. 2012 ? 11:57, Caleb a ?crit :

> Hi all,
> 
> I am new to this mailing list, so hi to everyone.
> Here is my first question:
> 
> I use nbconvert(master branch) to convert my notebook to latex file, but
> somehow all my output cells are missing.

Can you post any notebook example that does it ? 
Which Os ? Which notebook version ? 
-- 
Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120923/276d6e4f/attachment.html>

From d.warde.farley at gmail.com  Sun Sep 23 02:33:23 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Sun, 23 Sep 2012 02:33:23 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
Message-ID: <CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>

On Fri, Sep 21, 2012 at 5:37 AM, Fernando Perez <fperez.net at gmail.com> wrote:

> Yup.  Just a few days ago I had to fix this very problem for a
> colleague (though in her case, caused by a system-wide EPD install
> where setuptools was overriding any local PYTHONPATH configuration).
> After some frustration, I learned of the file 'usercustomize.py':
>
> http://docs.python.org/library/site.html
>
> Put a file with this name *anywhere* in your PYTHONPATH, and fix
> sys.path manually in there to reorder things as needed, e.g.
>
> sys.path = [os.environ['HOME']+'.local/lib/python2.7/site-packages/'] + sys.path

Thanks Fernando! I'll give this a try.

David


From takowl at gmail.com  Sat Sep 22 17:39:04 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 22 Sep 2012 22:39:04 +0100
Subject: [IPython-dev] Scipy central & IPython notebook.
Message-ID: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>

An idea I've had knocking around for a while is to integrate the
IPython notebook format, and the nbconvert code, with scipy-central.
As I envisage it, users could upload a notebook, and the server would
automatically generate static HTML and .py formats from it. Visitors
would see the HTML view with links to download as .ipynb or .py. One
of the first examples I clicked on [1] includes code, descriptive
text, equations, and a plot - pretty much a text book example of what
the notebook does well.

I've included here Kevin Dunn, the owner of the scipy-central codebase
on Github. Kevin, does this idea sound interesting? I appreciate that
you might not have the time to work on it (it looks like the
scipy-central repository is a bit dead), but if someone were keen to
tackle it, would you have time to guide them around the architecture
of scipy-central, and would you be willing to merge and deploy that
feature when it was complete?

[1] http://scipy-central.org/item/12/3/integrating-an-initial-value-problem-single-ode

Thanks,
Thomas


From fperez.net at gmail.com  Sun Sep 23 00:00:15 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 22 Sep 2012 21:00:15 -0700
Subject: [IPython-dev] The site needs an update... Web team, anyone?
Message-ID: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>

Hi folks,

We've had a bunch of new stuff lately that isn't even linked
*anywhere* on the site, the two major ones being the awesome nbviewer
by Matthias and notebookcloud by Carl.  These are both services that
make ipython better for everyone, and we're not even linking them!
But rather than just throw yet two more links on the front page, I was
thinking it would be really nice to have a bit of a redesign there, so
that key items are easier to find.  Some ideas:

- the main description should perhaps be a set of links to a short,
single page with a screenshot and a small amount of text about each
major feature.  I'm thinking that the terminal, qt console, notebook,
etc bullets should each be links (or perhaps a visual grid of
thumbnails/icons?) to concise but visually clean descriptions of each.
 Maybe short pages, maybe even clean JS popups that show a screenshot
and a small amount of text?

- the news section should be a lot easier to update... Right now it
takes a full build of the site.  Rolling twitter feeds? Should we make
an account that several people can control?

- our 'projects using IPython' page
(http://wiki.ipython.org/Projects_using_IPython) is great as a
resource and should be kept, modulo a move to the new wiki on GH when
that happens.  But for key related projects that directly expose to
general users ipython functionality, such as nbviewer and nbcloud, I
think more prominent and visible links should be available.

I'm not sure how to best go about the layout, not being a web designer
*at all* myself.  If anyone wants to pitch in on this, keep in mind
that our website is a nice, small and separate repo on github
(https://github.com/ipython/ipython-website) so we can put interested
contributors on the website team and you don't need to worry about
having to touch the core code repos.

Any takers?

Cheers,

f


From bussonniermatthias at gmail.com  Sun Sep 23 07:52:28 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 23 Sep 2012 13:52:28 +0200
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
Message-ID: <F2B9A935-CBFF-4BA0-B69C-7B117D38C76D@gmail.com>


Le 22 sept. 2012 ? 23:39, Thomas Kluyver a ?crit :

> An idea I've had knocking around for a while is to integrate the
> IPython notebook format, and the nbconvert code, with scipy-central.
> As I envisage it, users could upload a notebook, and the server would
> automatically generate static HTML and .py formats from it. Visitors
> would see the HTML view with links to download as .ipynb or .py. One
> of the first examples I clicked on [1] includes code, descriptive
> text, equations, and a plot - pretty much a text book example of what
> the notebook does well.

If needed, I'm working on a server that generate thumbnails of notebooks for nbviewer.
The prototype is working right now, I just need to make the logic to spawn the rendering to multiple workers,
and cache result to avoid useless rendering.

Right now here is what I do with it :
http://i.imgur.com/dMEZL.jpg
(with a slight hint of what the next version of nbviewer would look like)
here tiles size where big and random to see how the server would handle it,
but you can get whatever size you want

Tell me if you are interested.
-- 
Matthias

> I've included here Kevin Dunn, the owner of the scipy-central codebase
> on Github. Kevin, does this idea sound interesting? I appreciate that
> you might not have the time to work on it (it looks like the
> scipy-central repository is a bit dead), but if someone were keen to
> tackle it, would you have time to guide them around the architecture
> of scipy-central, and would you be willing to merge and deploy that
> feature when it was complete?
> 
> [1] http://scipy-central.org/item/12/3/integrating-an-initial-value-problem-single-ode
> 
> Thanks,
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From ellisonbg at gmail.com  Sun Sep 23 13:46:41 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 10:46:41 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
Message-ID: <CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>

Thomas,

Originally I was thinking exactly the same that that you are
proposing.  But once I saw nbviewer, my thinking changed:

* It makes no sense to reinvent a website for actually hosting
notebooks - github and friends work perfectly.  There are multiple
problems with having separate uploads onto the website:  i) the author
of the notebook has to maintain that content separately from their
original source, which is probably on github anyways ii) the uploaded
notebook won't be version controlled, so if we want to allow
versioning, we have to reinvent it, iii) when someone wants to use and
modify the notebook, they can't just fork/clone it.  In summary, we
should strongly encourage any such solution to simply use github for
hosted content.  This is part of why I so strongly support the vision
of nbvewier.
* What nbviewer should gain is the ability for users to register their
github (or elsewhere) hosted notebooks so there is a central list of
notebooks that everyone can view.  It would be nice if there could be
comments, tags, and link to different formats of the notebook.
* Given these features, what we are really talking about here is
nbviewer, not scipy-central and I think it should be developed under
that name within the IPython universe.

Cheers,

Brian

On Sat, Sep 22, 2012 at 2:39 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> An idea I've had knocking around for a while is to integrate the
> IPython notebook format, and the nbconvert code, with scipy-central.
> As I envisage it, users could upload a notebook, and the server would
> automatically generate static HTML and .py formats from it. Visitors
> would see the HTML view with links to download as .ipynb or .py. One
> of the first examples I clicked on [1] includes code, descriptive
> text, equations, and a plot - pretty much a text book example of what
> the notebook does well.
>
> I've included here Kevin Dunn, the owner of the scipy-central codebase
> on Github. Kevin, does this idea sound interesting? I appreciate that
> you might not have the time to work on it (it looks like the
> scipy-central repository is a bit dead), but if someone were keen to
> tackle it, would you have time to guide them around the architecture
> of scipy-central, and would you be willing to merge and deploy that
> feature when it was complete?
>
> [1] http://scipy-central.org/item/12/3/integrating-an-initial-value-problem-single-ode
>
> Thanks,
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sun Sep 23 14:03:14 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 11:03:14 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
Message-ID: <CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>

A few other reasons I would like this to be developed under the
IPython universe as part of nbviewer:

* I am not sure that the community as a whole is ready to commit to
having the notebook as this central and important.  The ipython dev
team is however ready for this and I think we should just lead with
code and people will follow.
* Marketing.  If it is part of the ipython universe, we can control
the messaging and marketing to promote the notebook and ipython.  As a
project, we tend to not be very outgoing with our marketing and I
think this is a mistake.  This type of site is a great opportunity for
us to be bold with our messaging = "The IPython Notebook is awesome
and you should be using it for stuff like this, here is how to get
started!"
*  I want us to be able to develop the site it in a way that has a
consistent look and feel as i) the notebook web app and ii) the
ipython website.  If we develop it ourselves, that is not a problem.
If we work with the larger community, there are other stake holders.
I think that in the long run, we will want to have consistently
branded sites (using something like twitter bootstrap with a
consistent theme).
* Any such site has to be developed in close conjunction with the
direction of the notebook file format and nbconvert.  The ipython dev
team are the primary stakeholders in that.

Cheers,

Brian


On Sun, Sep 23, 2012 at 10:46 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> Thomas,
>
> Originally I was thinking exactly the same that that you are
> proposing.  But once I saw nbviewer, my thinking changed:
>
> * It makes no sense to reinvent a website for actually hosting
> notebooks - github and friends work perfectly.  There are multiple
> problems with having separate uploads onto the website:  i) the author
> of the notebook has to maintain that content separately from their
> original source, which is probably on github anyways ii) the uploaded
> notebook won't be version controlled, so if we want to allow
> versioning, we have to reinvent it, iii) when someone wants to use and
> modify the notebook, they can't just fork/clone it.  In summary, we
> should strongly encourage any such solution to simply use github for
> hosted content.  This is part of why I so strongly support the vision
> of nbvewier.
> * What nbviewer should gain is the ability for users to register their
> github (or elsewhere) hosted notebooks so there is a central list of
> notebooks that everyone can view.  It would be nice if there could be
> comments, tags, and link to different formats of the notebook.
> * Given these features, what we are really talking about here is
> nbviewer, not scipy-central and I think it should be developed under
> that name within the IPython universe.
>
> Cheers,
>
> Brian
>
> On Sat, Sep 22, 2012 at 2:39 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>> An idea I've had knocking around for a while is to integrate the
>> IPython notebook format, and the nbconvert code, with scipy-central.
>> As I envisage it, users could upload a notebook, and the server would
>> automatically generate static HTML and .py formats from it. Visitors
>> would see the HTML view with links to download as .ipynb or .py. One
>> of the first examples I clicked on [1] includes code, descriptive
>> text, equations, and a plot - pretty much a text book example of what
>> the notebook does well.
>>
>> I've included here Kevin Dunn, the owner of the scipy-central codebase
>> on Github. Kevin, does this idea sound interesting? I appreciate that
>> you might not have the time to work on it (it looks like the
>> scipy-central repository is a bit dead), but if someone were keen to
>> tackle it, would you have time to guide them around the architecture
>> of scipy-central, and would you be willing to merge and deploy that
>> feature when it was complete?
>>
>> [1] http://scipy-central.org/item/12/3/integrating-an-initial-value-problem-single-ode
>>
>> Thanks,
>> Thomas
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From takowl at gmail.com  Sun Sep 23 14:16:09 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 23 Sep 2012 19:16:09 +0100
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
Message-ID: <CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>

On 23 September 2012 19:03, Brian Granger <ellisonbg at gmail.com> wrote:
> * Marketing.  If it is part of the ipython universe, we can control
> the messaging and marketing to promote the notebook and ipython.  As a
> project, we tend to not be very outgoing with our marketing and I
> think this is a mistake.

As a counterpoint: I'm not thinking of this as a marketing tool for
IPython notebooks. First and foremost it should be a useful repository
for scientific Python users. I also want to be able to mix examples
submitted as notebooks with examples from people who don't use the
notebook, in which case it would work like scipy-central currently
does.

In short: I see the notebook as a tool for this kind of site, not the
focus of it.

Thomas


From kgdunn at gmail.com  Sun Sep 23 14:57:19 2012
From: kgdunn at gmail.com (Kevin Dunn)
Date: Sun, 23 Sep 2012 14:57:19 -0400
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
Message-ID: <CAGD7iRa6gNk0nGy27PSkNWgzL1AsHcr7umTnWPxNJj41sxPUeQ@mail.gmail.com>

On Sun, Sep 23, 2012 at 2:16 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 23 September 2012 19:03, Brian Granger <ellisonbg at gmail.com> wrote:
>> * Marketing.  If it is part of the ipython universe, we can control
>> the messaging and marketing to promote the notebook and ipython.  As a
>> project, we tend to not be very outgoing with our marketing and I
>> think this is a mistake.
>
> As a counterpoint: I'm not thinking of this as a marketing tool for
> IPython notebooks. First and foremost it should be a useful repository
> for scientific Python users. I also want to be able to mix examples
> submitted as notebooks with examples from people who don't use the
> notebook, in which case it would work like scipy-central currently
> does.
>
> In short: I see the notebook as a tool for this kind of site, not the
> focus of it.
>
> Thomas


Hi there, thanks for these exciting ideas. However, I posted this to
the SciPy list on 26 Aug 2012. I'll simply cut and paste it here
again. I know this is the IPython list, but it is relevant to the
above discussion.

In short, I'm happy for the community to take on SciPy-Central and use
it however it could work best. However, I'm unable to contribute at
this time to make it work in the desired way.

Here's the original message. Thanks for reading. Kevin
----

Hi everyone, the scipy-central.org "maintainer" here; in inverted
commas, because I've been doing no maintenance over the past 12 to 18
months.

I'd like to offer all the scipy-central.org material over to the
community to keep going with it and improve it. I expected to have
time to do it myself, but new jobs and a different career path have
prevented me from implementing improvements that Pauli and several
others have proposed in the past.

This might be a good time to consider the issue, even just from a
branding perspective, to get it all looking coherent.

If anyone wants to get back to me on this, I can pass over the following:

1. Some/all of the domain names (those not transferred I will let
lapse on the next renewal in June 2013)

scipy-central.org(DNS)
scpyce.org(DNS)
scipy-central.com(MX)	Forward ( http://scipy-central.org )
scipy-central.net(MX)		Forward ( http://scipy-central.org )
scipycentral.com(MX)		Forward ( http://scipy-central.org )
scipycentral.net(MX)		Forward ( http://scipy-central.org )
scipycentral.org(MX)		Forward ( http://scipy-central.org )
scpyce.com(MX)		Forward ( http://scpyce.org )	
scpyce.net(MX)			Forward ( http://scpyce.org )	

2. all Django code, media files, templates, CSS and HTML, including
detailed instructions on how to install and use it are already at
https://github.com/kgdunn/SciPyCentral  Please take a look at this to
judge the complexity of the move.
3. backup scripts: local and off-site
4. the PostgreSQL database [23Mb]
5. monthly backup snapshots going back to July 2011, when the site
started [450Mb]
6. the Google Analytics account (not sure how to transfer this)
7. the original artwork files for the media

There are some interesting page hit and search analytics in raw data
(item 5) and from Google (item 6) that someone might want to look at.

I currently host the site on Webfaction. The new owner(s) will require
a similar unix-type server with:
* Python and Django,
* webserver (e.g. Apache, or nginx),
* ability to email for account creation, password resets and
submission confirmation
* rsync for remote backup,
* and other miscellaneous utilities: South, Pygments, Sphinx, Haystack
search with Xapian, PIL, etc

The new owner(s) will also inherit 2 pull requests and 45 issues on
the above-mentioned Github site. Other proposals made in the Scipy-Dev
mailing lists over the past year also exist.

I'm in no rush to hand it over; just putting it out there that someone
or some group more capable and with more time can definitely do a
better job of it than me.

Let me know how the community wants to proceed: please let's keep the
discussion public so everyone can contribute.


From ellisonbg at gmail.com  Sun Sep 23 15:53:13 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 12:53:13 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
Message-ID: <CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>

On Sun, Sep 23, 2012 at 11:16 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 23 September 2012 19:03, Brian Granger <ellisonbg at gmail.com> wrote:
>> * Marketing.  If it is part of the ipython universe, we can control
>> the messaging and marketing to promote the notebook and ipython.  As a
>> project, we tend to not be very outgoing with our marketing and I
>> think this is a mistake.
>
> As a counterpoint: I'm not thinking of this as a marketing tool for
> IPython notebooks. First and foremost it should be a useful repository
> for scientific Python users.

I don't view marketing and usefulness as mutually exclusive and I
agree with you that usefulness is the primary desired attribute.  I
(and everyone else on the dev team) would definitely not tolerate mere
marketing with no actual usefulness.  But this is the exact reason I
brought up the marketing point - we are not good at it and downplay
its importance like it is a bad thing.  This is an opportunity for us
to address that issue head on.

> I also want to be able to mix examples
> submitted as notebooks with examples from people who don't use the
> notebook, in which case it would work like scipy-central currently
> does.

The notebook is quickly growing into a full blown IDE and will
definitely be able to edit non-notebook files.  It is a completely
logic step for nbviewer to grow those abilities as well.

> In short: I see the notebook as a tool for this kind of site, not the
> focus of it.

OK, but what about my comments about using github rather than
uploading files by hand for content?

Cheers,

Brian

> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Sun Sep 23 16:11:13 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Sep 2012 13:11:13 -0700
Subject: [IPython-dev] Fwd: [IPython-User] Looking for an IPython Expert
In-Reply-To: <CAE-2DE7qY52vu1bHwASvM-T9CUJY82RD5Lizt92PUu+GE9Ckzg@mail.gmail.com>
References: <CAE-2DE7qY52vu1bHwASvM-T9CUJY82RD5Lizt92PUu+GE9Ckzg@mail.gmail.com>
Message-ID: <CAHAreOrYrkE-LAXJse1r=tbJU2ZAJ+UtS5Q7em9S2TNUhjsgqg@mail.gmail.com>

Hi folks,

forwarding this from the user list in case some dedicated developers
happen not to scan that list regularly.  As an aside, David (the
author) had the courtesy of contacting me prior to posting and running
the text by me and I was happy to see this happen. I am thrilled to
see IPython-specific job posts start appearing!

For future reference in similar cases, all I asked of David was that
he follow the guidelines the Bay Piggies list uses:
http://www.baypiggies.net/job-listings

Cheers,

f

---------- Forwarded message ----------
From: David Himrod <david.himrod at gmail.com>
Date: Sun, Sep 23, 2012 at 11:12 AM
Subject: [IPython-User] Looking for an IPython Expert
To: ipython-user at scipy.org



IPython users and developers,

My name is Dave and I run the Optimization and Analytics team at Appnexus.
 The team started using iPython and Pandas around 2 years ago as a way to
quickly turn data research into production systems.  Not surprisingly,
these tools have picked up traction throughout the company.  Many other
groups (spanning engineering and business) are using them to do analysis
and compliment their development practices.  I would say we have 20-30
regular users, and estimate it will be closer to 100 in the next 6 months.
 We are looking for an IPython expert (in New York) to help us create a
deeply integrated development and research environment for a growing
organization.  We would be interested in giving back to IPython if we do
any development that makes sense to share with the community.

In addition, we have open positions for Quants and Engineers within the
optimization team.  If you are interested please reach out to me at
dhimrod at appnexus.com.  I would also consider consulting engagements, but
note Wes Mckinney **name drop** has set the bar pretty damn high.

You can read more about the team here
http://careers.appnexus.com/teams/optimization/

Thanks,
Dave



_______________________________________________
IPython-User mailing list
IPython-User at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-user


From takowl at gmail.com  Sun Sep 23 16:19:44 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 23 Sep 2012 21:19:44 +0100
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
Message-ID: <CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>

On 23 September 2012 20:53, Brian Granger <ellisonbg at gmail.com> wrote:
> OK, but what about my comments about using github rather than
> uploading files by hand for content?

I assume that would include things like gists, which seem easier for
relatively small examples? So long as people don't need to go through
setting up a git repository and pushing stuff, using external storage
is fine by me.

(From an earlier message)
> * Given these features, what we are really talking about here is
> nbviewer, not scipy-central and I think it should be developed under
> that name within the IPython universe.

>From my perspective, nbviewer is a poor name for this. The focus
should be on the content (scientific Python example code), not the
tools (the notebook). Within that, we can promote the Notebook, but
naming the site after it would be putting the cart before the horse.

Thomas


From benjaminrk at gmail.com  Sun Sep 23 16:52:59 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 23 Sep 2012 13:52:59 -0700
Subject: [IPython-dev] slow startup due to script magic
In-Reply-To: <505E63B9.2080607@creativetrax.com>
References: <505D4C0C.50009@creativetrax.com>
	<CAOvn4qitPTtx26TzuWqpthh3Az7Sz0b6+TrBCFe6KBhU3ajHwg@mail.gmail.com>
	<CAHAreOr3LVqeO70Eu81wx2yCKYLUwwFbhyXgHaeOYNExSRVO6g@mail.gmail.com>
	<505E63B9.2080607@creativetrax.com>
Message-ID: <CAHNn8BX_GkiUxBPe4kZtiRype9HuA-7XQEhx3826bL=ooiKzzA@mail.gmail.com>

On Sat, Sep 22, 2012 at 6:19 PM, Jason Grout <jason-sage at creativetrax.com>wrote:

> On 9/22/12 1:33 PM, Fernando Perez wrote:
> > On Sat, Sep 22, 2012 at 3:40 AM, Thomas Kluyver <takowl at gmail.com>
> wrote:
> >> Hmm, well spotted. If it's that slow, I think we should avoid doing it
> >> during IPython start up at all. I like it to start up quickly.
> >
> > Absolutely, thanks for catching this, Jason!  Indeed, startup time is
> > a really important consideration, and we should be very careful with
> > it.
> >
> > Does anyone have a tool to instrument the startup process in an easy
> > manner?  Jason, if you guys have any such tools for Sage that you
> > could share, we could track these things and prevent similar
> > regressions in the future.
>
> We do have a sage -startuptime script that (I think) uses an import hook
> to track the time to import the many modules that Sage imports.  It
> sounds like that may not be as useful to you guys, since you don't
> import nearly as many things as we do.  And that's mainly for us to
> diagnose startup time regressions by hand.
>
> I suppose it would be pretty straightforward to add a nose test that
> would check the time to start a full kernel, right?
>
>
> >
> >> Maybe we should keep it simple: rather than checking for the existence
> >> of each interpreter, just set a standard list. It would mean that
> >> users can tab complete e.g. %%pypy, even if they don't have pypy
> >> installed, but I don't think it will be too surprising if they try it
> >> and it fails.
> >
> > We could also cache the search path and do it only on first usage,
> > rather than at IPython startup.  Else we can discuss moving this to an
> > optional extension...
>
> It's also less confusing to have a standard list that is always
> available.  Otherwise I have to try to guess why in the world I have
> %%ruby, but you don't.
>
> I'd say that %%ruby should invoke ruby with "/bin/env ruby" and leave it
> at that.  Why do we need to cache the path?
>

https://github.com/ipython/ipython/pull/2424



>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120923/11727f7c/attachment.html>

From fperez.net at gmail.com  Sun Sep 23 16:56:06 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Sep 2012 13:56:06 -0700
Subject: [IPython-dev] slow startup due to script magic
In-Reply-To: <505E63B9.2080607@creativetrax.com>
References: <505D4C0C.50009@creativetrax.com>
	<CAOvn4qitPTtx26TzuWqpthh3Az7Sz0b6+TrBCFe6KBhU3ajHwg@mail.gmail.com>
	<CAHAreOr3LVqeO70Eu81wx2yCKYLUwwFbhyXgHaeOYNExSRVO6g@mail.gmail.com>
	<505E63B9.2080607@creativetrax.com>
Message-ID: <CAHAreOp1RMmOEZTOPErPUCHaCf5FnKWJe0YwtY1CowVSujuedg@mail.gmail.com>

On Sat, Sep 22, 2012 at 6:19 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> I suppose it would be pretty straightforward to add a nose test that
> would check the time to start a full kernel, right?

Yes, and we should build a way to track that over time on a
standardized machine.  I think the ideal quantity to track would be a
distribution of the ratio of starting ipython to starting python
itself, over a set of configurations (local linux/osx/windows,
machines with NFS filesystems, etc).  That's the ratio we don't want
diverging and which is meaningful, not really the absolute startup
time by itself (if python takes longer to start, there's nothing we
can do about that).

>>> Maybe we should keep it simple: rather than checking for the existence
>>> of each interpreter, just set a standard list. It would mean that
>>> users can tab complete e.g. %%pypy, even if they don't have pypy
>>> installed, but I don't think it will be too surprising if they try it
>>> and it fails.
>>
>> We could also cache the search path and do it only on first usage,
>> rather than at IPython startup.  Else we can discuss moving this to an
>> optional extension...
>
> It's also less confusing to have a standard list that is always
> available.  Otherwise I have to try to guess why in the world I have
> %%ruby, but you don't.
>
> I'd say that %%ruby should invoke ruby with "/bin/env ruby" and leave it
> at that.  Why do we need to cache the path?

And what about windows?  Remember, IPython is 100% windows compatible,
so unix-y solutions aren't good enough for us.  But there may be a
similar solution for windows that we can rely on.

I think, however, that the idea was to only provide the %%foo scripts
for which interepreters were actually found, hence the (expensive)
checks on startup.  I'm leaning more towards:

- finding a valid way to make a 'env foo'-style call across platforms
- defining a hardcoded set of defaults we'll call that way
- simply giving a good error message if the user happens not to
actually have 'foo' installed on their system.

That would mean no caching involved, no expensive early searches, and
I think an acceptable user experience (you could argue an even better
one, in that they wouldn't have to wonder why %%ruby is an undefined
magic for them, instead simply knowing that %%ruby happens to call
ruby which they may need to install).

... aand as usual, while we talk, Min codes:

https://github.com/ipython/ipython/pull/2424

:)

Further discussion should continue on the PR.

Cheers,

f


From ellisonbg at gmail.com  Sun Sep 23 17:28:22 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 14:28:22 -0700
Subject: [IPython-dev] Fwd: [IPython-User] Looking for an IPython Expert
In-Reply-To: <CAHAreOrYrkE-LAXJse1r=tbJU2ZAJ+UtS5Q7em9S2TNUhjsgqg@mail.gmail.com>
References: <CAE-2DE7qY52vu1bHwASvM-T9CUJY82RD5Lizt92PUu+GE9Ckzg@mail.gmail.com>
	<CAHAreOrYrkE-LAXJse1r=tbJU2ZAJ+UtS5Q7em9S2TNUhjsgqg@mail.gmail.com>
Message-ID: <CAH4pYpRK975QNKpLNFaSvJOV-vk-7cVmEpr43cdahZvQLOZUjA@mail.gmail.com>

FWIW, I am strongly +1 on allowing job postings to our lists.  This is great!

Cheers,

Brian

On Sun, Sep 23, 2012 at 1:11 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> forwarding this from the user list in case some dedicated developers
> happen not to scan that list regularly.  As an aside, David (the
> author) had the courtesy of contacting me prior to posting and running
> the text by me and I was happy to see this happen. I am thrilled to
> see IPython-specific job posts start appearing!
>
> For future reference in similar cases, all I asked of David was that
> he follow the guidelines the Bay Piggies list uses:
> http://www.baypiggies.net/job-listings
>
> Cheers,
>
> f
>
> ---------- Forwarded message ----------
> From: David Himrod <david.himrod at gmail.com>
> Date: Sun, Sep 23, 2012 at 11:12 AM
> Subject: [IPython-User] Looking for an IPython Expert
> To: ipython-user at scipy.org
>
>
>
> IPython users and developers,
>
> My name is Dave and I run the Optimization and Analytics team at Appnexus.
>  The team started using iPython and Pandas around 2 years ago as a way to
> quickly turn data research into production systems.  Not surprisingly,
> these tools have picked up traction throughout the company.  Many other
> groups (spanning engineering and business) are using them to do analysis
> and compliment their development practices.  I would say we have 20-30
> regular users, and estimate it will be closer to 100 in the next 6 months.
>  We are looking for an IPython expert (in New York) to help us create a
> deeply integrated development and research environment for a growing
> organization.  We would be interested in giving back to IPython if we do
> any development that makes sense to share with the community.
>
> In addition, we have open positions for Quants and Engineers within the
> optimization team.  If you are interested please reach out to me at
> dhimrod at appnexus.com.  I would also consider consulting engagements, but
> note Wes Mckinney **name drop** has set the bar pretty damn high.
>
> You can read more about the team here
> http://careers.appnexus.com/teams/optimization/
>
> Thanks,
> Dave
>
>
>
> _______________________________________________
> IPython-User mailing list
> IPython-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Sun Sep 23 17:49:59 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 14:49:59 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
Message-ID: <CAH4pYpQ6QyMzokum2tqtXgzmzinXgru3dQdXaFabrrx1=P1iAw@mail.gmail.com>

On Sun, Sep 23, 2012 at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 23 September 2012 20:53, Brian Granger <ellisonbg at gmail.com> wrote:
>> OK, but what about my comments about using github rather than
>> uploading files by hand for content?
>
> I assume that would include things like gists, which seem easier for
> relatively small examples? So long as people don't need to go through
> setting up a git repository and pushing stuff, using external storage
> is fine by me.

Yes, it will definitely work with gists for that reason.

> (From an earlier message)
>> * Given these features, what we are really talking about here is
>> nbviewer, not scipy-central and I think it should be developed under
>> that name within the IPython universe.
>
> >From my perspective, nbviewer is a poor name for this. The focus
> should be on the content (scientific Python example code), not the
> tools (the notebook). Within that, we can promote the Notebook, but
> naming the site after it would be putting the cart before the horse.

Yes, I agree that as the capabilities of nbviewer expand, it will
quickly outgrow that name and that we will probably want to reconsider
that.

Cheers,

Brian

> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Sun Sep 23 19:13:49 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Sep 2012 16:13:49 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
Message-ID: <CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>

On Sun, Sep 23, 2012 at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> >From my perspective, nbviewer is a poor name for this. The focus
> should be on the content (scientific Python example code), not the
> tools (the notebook). Within that, we can promote the Notebook, but
> naming the site after it would be putting the cart before the horse.

Jumping in a little late here...  Here's my perspective on this:
IPython is an important piece of the pylab ecosystem, and as I argued
vociferously on the scipy-user discussion, I think the notebook should
play an even more prominent role in there.  But it's also very
important to keep this in mind: ipython and the notebook are also used
by, and important to, communities that have nothing to do at all with
scientific work.

And in the spirit of pylab being a federation of collaborating
projects, I think the right solution here is to find a way for say
'pylab-central' (were it to exist) to use something *like* nbviewer,
or nbviewer itself, but without having to tightly couple it to pylab
central, so that it can be equally used by other projects and
communities that don't have scientific uses in mind.

Pylab-central could, for notebooks, simply embed (perhaps in an
iframe) the nbviewer-rendered version of a notebook.  It will be up to
the pylab central developers to decide whether they want to get into
the business of hosting notebooks or they point to people's gists or
full-fledged repos. nbviewer doesn't care where things are: as long as
there's a public URL for an ipynb file, it can render it.

This loose-coupling model seems to me better both in terms of
acknowledging the structure of the pylab/ipython Venn diagram as well
as in the spirit of federated collaboration we're trying to establish
with pylab.

Cheers,

f


From ellisonbg at gmail.com  Sun Sep 23 19:34:40 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 16:34:40 -0700
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
	team, anyone?
In-Reply-To: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
Message-ID: <CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>

> We've had a bunch of new stuff lately that isn't even linked
> *anywhere* on the site, the two major ones being the awesome nbviewer
> by Matthias and notebookcloud by Carl.  These are both services that
> make ipython better for everyone, and we're not even linking them!
> But rather than just throw yet two more links on the front page, I was
> thinking it would be really nice to have a bit of a redesign there, so
> that key items are easier to find.  Some ideas:

This bring up a discussion that we started to have earlier in the
summer, but never finished:

How do we decide what subprojects to include under the ipython
universe and what are the benefits/responsibilities associated with
inclusion?

The background is that nbviewer is more or less part of the ipython
universe, but notebook cloud is not.  Here is what I propose:

To be part of the ipython universe a subproject must:

* Be hosted as a github repo under the ipython org.
* Be released under a BSD license.
* Not charge users in any way or be funded directly through a for profit entity.
* Ideally, any funding would go through the ipython
foundation/numfocus.  This would allow for profit entities to sponsor
ipython development, but it would allow the foundation to make sure it
was being done in a way that was consistent with the goals of the
project.
* Participate in our governance model (although we haven't formalized this).
* Have someone who is committed to maintaining the project.
* Be approved for inclusion by a vote of the devs who have commit rights.

In return, the subproject would get:

* Possible funding through numfocus/ipython foundation and other grants.
* Promotion on the ipython website as an official ipython subproject.
* The participation of the ipython dev team in helping with development.
* Name recognition from being part of ipython.

This would allow us to distinguish, in a fair manner, between projects
that we promote on the website as "This project uses IPython" (for
that we should post anything and everything) and these approved
subprojects that are actually parts of the ipython universe.

What do people think about something like this?

What do people think about notebook cloud in this respect?

> - the main description should perhaps be a set of links to a short,
> single page with a screenshot and a small amount of text about each
> major feature.  I'm thinking that the terminal, qt console, notebook,
> etc bullets should each be links (or perhaps a visual grid of
> thumbnails/icons?) to concise but visually clean descriptions of each.
>  Maybe short pages, maybe even clean JS popups that show a screenshot
> and a small amount of text?

Yes, don't forget the parallel computing support.

> - the news section should be a lot easier to update... Right now it
> takes a full build of the site.  Rolling twitter feeds? Should we make
> an account that several people can control?

Yes,

> - our 'projects using IPython' page
> (http://wiki.ipython.org/Projects_using_IPython) is great as a
> resource and should be kept, modulo a move to the new wiki on GH when
> that happens.  But for key related projects that directly expose to
> general users ipython functionality, such as nbviewer and nbcloud, I
> think more prominent and visible links should be available.

Yes I just want to make sure that we all agree what those projects are.

> I'm not sure how to best go about the layout, not being a web designer
> *at all* myself.  If anyone wants to pitch in on this, keep in mind
> that our website is a nice, small and separate repo on github
> (https://github.com/ipython/ipython-website) so we can put interested
> contributors on the website team and you don't need to worry about
> having to touch the core code repos.
>
> Any takers?
>
> Cheers,
>
> f
> _______________________________________________
> IPython-User mailing list
> IPython-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From matthewturk at gmail.com  Sun Sep 23 19:52:52 2012
From: matthewturk at gmail.com (Matthew Turk)
Date: Sun, 23 Sep 2012 19:52:52 -0400
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
Message-ID: <CALO3=5ExBaxcGy37sT=AOoYFw_dgOUJhkPxCxrOxkPKo3XPN7w@mail.gmail.com>

[snip]

> Pylab-central could, for notebooks, simply embed (perhaps in an
> iframe) the nbviewer-rendered version of a notebook.  It will be up to
> the pylab central developers to decide whether they want to get into
> the business of hosting notebooks or they point to people's gists or
> full-fledged repos. nbviewer doesn't care where things are: as long as
> there's a public URL for an ipynb file, it can render it.

For what it's worth, in the yt project we have a data hub where
projects get listed and data uploaded for remote viz.  We just added a
notebook upload function, which accepts (like a pastebin, more than
like a forkable gist) a notebook file and then generates links to the
nbviewer from that upload.

-Matt


From fperez.net at gmail.com  Sun Sep 23 19:57:53 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Sep 2012 16:57:53 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CALO3=5ExBaxcGy37sT=AOoYFw_dgOUJhkPxCxrOxkPKo3XPN7w@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<CALO3=5ExBaxcGy37sT=AOoYFw_dgOUJhkPxCxrOxkPKo3XPN7w@mail.gmail.com>
Message-ID: <CAHAreOqvMPccCJDUmq3mA9uULoFrNA2cGweG68wfUQEWsfFm-g@mail.gmail.com>

On Sun, Sep 23, 2012 at 4:52 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>
> For what it's worth, in the yt project we have a data hub where
> projects get listed and data uploaded for remote viz.  We just added a
> notebook upload function, which accepts (like a pastebin, more than
> like a forkable gist) a notebook file and then generates links to the
> nbviewer from that upload.

Excellent data point, and I think it reinforces my view: we want other
communities to take advantage of this resource, with a loose coupling
between them.  nbviewer should be a service, central to ipython itself
but also easy to weave into another project's workflow.  Obviously
other projects can decide to run *their own* nbviewer if they so
desire, but the main one at ipython should work well with minimal
coupling, as it seems to be doing for you guys.

Thanks a lot for this feedback!

f


From carl.input at gmail.com  Sun Sep 23 21:38:04 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 24 Sep 2012 02:38:04 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
Message-ID: <CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>

Hi all

I just saw Brian's post and wanted to offer my initial thoughts as the
author of nbcloud.

> To be part of the ipython universe a subproject must:
>
> * Be hosted as a github repo under the ipython org.

This seems fine. New projects can be developed independently, then
move to the ipython org once they're mature enough.

> * Be released under a BSD license.

Seems fine.

> * Not charge users in any way or be funded directly through a for profit entity.

This might be problematic. NotebookCloud is designed to work with the
user's EC2 machines, but I had considered providing a more integrated
offering, where nbcloud would bill users directly and pay Amazon
separately. For me, this just wasn't workable, but other projects may
need to charge users, even if only to recover some or all of its
costs. Web service provision normally costs money, sometimes lots of
it. Someone has to pay.

> * Ideally, any funding would go through the ipython
> foundation/numfocus.  This would allow for profit entities to sponsor
> ipython development, but it would allow the foundation to make sure it
> was being done in a way that was consistent with the goals of the
> project.

I'm a fan of sponsorship. If a project had a sponsor that paid the
bills, should this prevent it from becoming an official sub-project?
It's also unlikely that a company that's willing to do a deal with one
project is going to be equally happy to donate to a third party that
wont offer anything like the same package.

> * Participate in our governance model (although we haven't formalized this).

Would this be a requirement, or an open offer?

> * Have someone who is committed to maintaining the project.

This is a difficult thing for an open source developer to commit to
long term, but makes perfect sense nonetheless.

> * Be approved for inclusion by a vote of the devs who have commit rights.

Seems very sensible.

> In return, the subproject would get:
>
> * Possible funding through numfocus/ipython foundation and other grants.

This sounds cool, but not if your project is forced to depend on this
type of funding exclusively. It discourages commercial involvement and
isn't really in keeping with the spirit of BSD licensing, as I
understand it.

> * Promotion on the ipython website as an official ipython subproject.

This'd be cool.

> * The participation of the ipython dev team in helping with development.

I don't see how this would work. If anyone wanted to send me a patch,
they can now. Being an officially recognised sub-project would only
make this a little more likely. There's no way IPython can say that,
if a project's official, our volunteers will work on it.

> * Name recognition from being part of ipython.

Sounds good.

P.S. As an aside: I'm currently looking to do some stuff with IPython
and the NAO robots. This will cost me at least ?3,000 up front. I
can't personally donate that kind of money to the public good. I'd
need a way to earn it back. I think your proposals make a lot of
sense, but I can see the financial requirements being too restrictive,
possibly making official sub-projects a very cliquey affair.

Personally, I'd probably always opt for unofficial status on every
project, just because I don't want to agree to something I might
regret down the line. It's a lot of restrictions to accept,
perpetually, with potentially nothing much in return.

Sorry, if this comes across as negative. It's not meant to at all. I
just wanted to offer a different perspective on some of these points.
Ultimately, the question has nothing to do with whether nbcloud would
'get in' or not, but how IPython wants to go about adopting projects.

All the best

Carl


From asmeurer at gmail.com  Sun Sep 23 23:09:03 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sun, 23 Sep 2012 21:09:03 -0600
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
Message-ID: <CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>

On Sun, Sep 23, 2012 at 7:38 PM, Carl Smith <carl.input at gmail.com> wrote:
> Hi all
>
> I just saw Brian's post and wanted to offer my initial thoughts as the
> author of nbcloud.
>
>> To be part of the ipython universe a subproject must:
>>
>> * Be hosted as a github repo under the ipython org.
>
> This seems fine. New projects can be developed independently, then
> move to the ipython org once they're mature enough.
>
>> * Be released under a BSD license.
>
> Seems fine.
>
>> * Not charge users in any way or be funded directly through a for profit entity.
>
> This might be problematic. NotebookCloud is designed to work with the
> user's EC2 machines, but I had considered providing a more integrated
> offering, where nbcloud would bill users directly and pay Amazon
> separately. For me, this just wasn't workable, but other projects may
> need to charge users, even if only to recover some or all of its
> costs. Web service provision normally costs money, sometimes lots of
> it. Someone has to pay.
>
>> * Ideally, any funding would go through the ipython
>> foundation/numfocus.  This would allow for profit entities to sponsor
>> ipython development, but it would allow the foundation to make sure it
>> was being done in a way that was consistent with the goals of the
>> project.
>
> I'm a fan of sponsorship. If a project had a sponsor that paid the
> bills, should this prevent it from becoming an official sub-project?
> It's also unlikely that a company that's willing to do a deal with one
> project is going to be equally happy to donate to a third party that
> wont offer anything like the same package.
>
>> * Participate in our governance model (although we haven't formalized this).
>
> Would this be a requirement, or an open offer?

Wouldn't this have to be a requirement?  By making it a part of the
IPython organization, you are surrendering the governing rights of the
project, so to speak.  For example, the admins of the IPython
organization will be free to give people push access to the
repository.

Of course, perhaps the governance model can be written so as to give
the maintainer (i.e., possibly the original author) some higher powers
over that particular project.  I don't really know what your
governance model looks like or what you want it to look like, so I
can't really say much on this point. I'm assuming you have a BDFL
model with Fernando the BDFL.  If so, maybe the maintainers could be
like "mini-BDFLs" of their sub-projects.  Or maybe that's a bad idea.

>
>> * Have someone who is committed to maintaining the project.
>
> This is a difficult thing for an open source developer to commit to
> long term, but makes perfect sense nonetheless.
>
>> * Be approved for inclusion by a vote of the devs who have commit rights.
>
> Seems very sensible.
>
>> In return, the subproject would get:
>>
>> * Possible funding through numfocus/ipython foundation and other grants.
>
> This sounds cool, but not if your project is forced to depend on this
> type of funding exclusively. It discourages commercial involvement and
> isn't really in keeping with the spirit of BSD licensing, as I
> understand it.
>
>> * Promotion on the ipython website as an official ipython subproject.
>
> This'd be cool.
>
>> * The participation of the ipython dev team in helping with development.
>
> I don't see how this would work. If anyone wanted to send me a patch,
> they can now. Being an officially recognised sub-project would only
> make this a little more likely. There's no way IPython can say that,
> if a project's official, our volunteers will work on it.
>
>> * Name recognition from being part of ipython.
>
> Sounds good.
>
> P.S. As an aside: I'm currently looking to do some stuff with IPython
> and the NAO robots. This will cost me at least ?3,000 up front. I
> can't personally donate that kind of money to the public good. I'd
> need a way to earn it back. I think your proposals make a lot of
> sense, but I can see the financial requirements being too restrictive,
> possibly making official sub-projects a very cliquey affair.
>
> Personally, I'd probably always opt for unofficial status on every
> project, just because I don't want to agree to something I might
> regret down the line. It's a lot of restrictions to accept,
> perpetually, with potentially nothing much in return.
>
> Sorry, if this comes across as negative. It's not meant to at all. I
> just wanted to offer a different perspective on some of these points.
> Ultimately, the question has nothing to do with whether nbcloud would
> 'get in' or not, but how IPython wants to go about adopting projects.
>
> All the best
>
> Carl

So I guess the question to ask is, what is the benefit from being
"officially supported" by IPython?  I guess it would result in more
people finding your project, because of all the official links
(webpage, possibly in the IPython software itself, etc.).  Other than
that, the only thing I can think of is that if you have code that
might potentially be useful to include in IPython itself, then you
would want to include it just to prevent code duplication and
unnecessary wheel reinvention.  Like you said, if you are already open
source and on GitHub, then you can receive pull requests. And if
IPython core devs help out, you can give them push access, so there's
little difference from the technical point of view.

Aaron Meurer


From ellisonbg at gmail.com  Mon Sep 24 01:38:15 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 22:38:15 -0700
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
Message-ID: <CAH4pYpTdymB4FYX0yb-B34SmBSoNS2etzZRT-SuawLVaPhpKSQ@mail.gmail.com>

Carl,

On Sun, Sep 23, 2012 at 6:38 PM, Carl Smith <carl.input at gmail.com> wrote:
> Hi all
>
> I just saw Brian's post and wanted to offer my initial thoughts as the
> author of nbcloud.
>
>> To be part of the ipython universe a subproject must:
>>
>> * Be hosted as a github repo under the ipython org.
>
> This seems fine. New projects can be developed independently, then
> move to the ipython org once they're mature enough.
>
>> * Be released under a BSD license.
>
> Seems fine.
>
>> * Not charge users in any way or be funded directly through a for profit entity.
>
> This might be problematic. NotebookCloud is designed to work with the
> user's EC2 machines, but I had considered providing a more integrated
> offering, where nbcloud would bill users directly and pay Amazon
> separately. For me, this just wasn't workable, but other projects may
> need to charge users, even if only to recover some or all of its
> costs. Web service provision normally costs money, sometimes lots of
> it. Someone has to pay.

Essentially, what you are talking about is running a business.  It
might not be a for-profit business, but it is still a legal business
entity with interests that are outside the ipython project.  We are
completely fine if businesses want to develop software based on
ipython and charge users for usage of it.  In fact, our choice of the
BSD license is precisely to encourage this type of thing.  But, I
don't think we want to tie the open source project to such entities.
The reason is that in the long run, it is likely that there will be
many businesses that offer IPython related services for $.  We want to
ensure the open source ipython project remains a neutral playing
ground for such entities.  Allowing subprojects that are directly
sponsored by a one particular business would erode that neutrality.

If you want to start to charge users for using notebook cloud and
create a legal entity to manage those revenues and expenses, I think
it should remain completely independent of ipython.  That not to say
we don't welcome the development, but it should be separate.

>> * Ideally, any funding would go through the ipython
>> foundation/numfocus.  This would allow for profit entities to sponsor
>> ipython development, but it would allow the foundation to make sure it
>> was being done in a way that was consistent with the goals of the
>> project.
>
> I'm a fan of sponsorship. If a project had a sponsor that paid the
> bills, should this prevent it from becoming an official sub-project?

Again, the goal is neutrality in situations where there are multiple
independent businesses in the ipython ecosystem.  It depends on how
the sponsorship was setup.  Let's say that nbviewer grows and we
eventually have more hosting costs.  If a company (let's say Enthought
for the sake of argument) wanted to cover those costs, they should
donate to the foundation and run the money through there.  That would
be fine.  What I don't want to get into is having subprojects that are
essentially commercial products that users pay for and that sponsors
also try to fund through the foundation.  That creates too much
entanglement between the open source project and external businesses
that may or may not have goals that line up with the project.

> It's also unlikely that a company that's willing to do a deal with one
> project is going to be equally happy to donate to a third party that
> wont offer anything like the same package.
>
>> * Participate in our governance model (although we haven't formalized this).
>
> Would this be a requirement, or an open offer?

Even though we don't have a formal governance model, I think it would
need to be required.  My rough summary of our governance model is
this:

* Fernando is BDFL - theoretically he has final say on everything.
* In practice, he almost never exercises his right as dictator.
Instead, he acts benevolently by deferring i) to community vote and
ii) to others who are closer to different parts of the code base.

Thus, if you or anyone wanted to contribute a subproject, you would
need to be willing to agree that ultimately Fernando has final say and
that the community would be the primary guide on a daily, practical
basis.

>> * Have someone who is committed to maintaining the project.
>
> This is a difficult thing for an open source developer to commit to
> long term, but makes perfect sense nonetheless.

Yes, no one can make long term promises, but that is part of what the
dev team would have to vote on, i.e., do we want this project to be
part of ipython even if the primary maintainer stops contributing.

>> * Be approved for inclusion by a vote of the devs who have commit rights.
>
> Seems very sensible.
>
>> In return, the subproject would get:
>>
>> * Possible funding through numfocus/ipython foundation and other grants.
>
> This sounds cool, but not if your project is forced to depend on this
> type of funding exclusively. It discourages commercial involvement and
> isn't really in keeping with the spirit of BSD licensing, as I
> understand it.

The term "commercial involvement" can mean different things here:

* A company like Enthought or Microsoft funding the development of IPython.
* Employees of a particular company contributing to the project.
* A company using IPython as part of their own commercial product.
This is the spirit of the BSD part you mention above.

All of these things have happened many times over the years and we
strongly encourage this type of commercial involvement.  The type of
commercial involvement that I am personally not OK with is a
commercial entity or external non-profit trying to run one of their
"for pay" products under the IPython umbrella.  I feel that
people/businesses/non-profits who want to build products on top of
ipython that cost users money should do that on their own.  There are
also good legal and financial reasons to keep these efforts separate
as well.  Please don't misunderstand me - I am happy to see this type
of thing happening, it just needs to be separate.

>> * Promotion on the ipython website as an official ipython subproject.
>
> This'd be cool.
>
>> * The participation of the ipython dev team in helping with development.
>
> I don't see how this would work. If anyone wanted to send me a patch,
> they can now. Being an officially recognised sub-project would only
> make this a little more likely. There's no way IPython can say that,
> if a project's official, our volunteers will work on it.
>
>> * Name recognition from being part of ipython.
>
> Sounds good.
>
> P.S. As an aside: I'm currently looking to do some stuff with IPython
> and the NAO robots. This will cost me at least ?3,000 up front. I
> can't personally donate that kind of money to the public good. I'd
> need a way to earn it back. I think your proposals make a lot of
> sense, but I can see the financial requirements being too restrictive,
> possibly making official sub-projects a very cliquey affair.

I am not quite following you, but it sounds like you need/want to make
money off of notebook cloud.  If this is the case, in my mind, you
should just do it separately from ipython.  We will support and
encourage you, but I think the constraints of the open source project
will only hinder you.

> Personally, I'd probably always opt for unofficial status on every
> project, just because I don't want to agree to something I might
> regret down the line. It's a lot of restrictions to accept,
> perpetually, with potentially nothing much in return.
>
> Sorry, if this comes across as negative. It's not meant to at all. I
> just wanted to offer a different perspective on some of these points.
> Ultimately, the question has nothing to do with whether nbcloud would
> 'get in' or not, but how IPython wants to go about adopting projects.

No, this doesn't come across as negative.  Part of the challenge here
is to try to foresee the future and come up with guidelines that we
can apply in a consistent manner moving forward.  Personally, I think
things like notebook cloud and nbviewer are just the beginning.

Would love to hear what the other core devs think about all of this.

Cheers,

Brian

> All the best
>
> Carl
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Mon Sep 24 01:49:29 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 23 Sep 2012 22:49:29 -0700
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
Message-ID: <CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>

Aaron,

On Sun, Sep 23, 2012 at 8:09 PM, Aaron Meurer <asmeurer at gmail.com> wrote:
> On Sun, Sep 23, 2012 at 7:38 PM, Carl Smith <carl.input at gmail.com> wrote:
>> Hi all
>>
>> I just saw Brian's post and wanted to offer my initial thoughts as the
>> author of nbcloud.
>>
>>> To be part of the ipython universe a subproject must:
>>>
>>> * Be hosted as a github repo under the ipython org.
>>
>> This seems fine. New projects can be developed independently, then
>> move to the ipython org once they're mature enough.
>>
>>> * Be released under a BSD license.
>>
>> Seems fine.
>>
>>> * Not charge users in any way or be funded directly through a for profit entity.
>>
>> This might be problematic. NotebookCloud is designed to work with the
>> user's EC2 machines, but I had considered providing a more integrated
>> offering, where nbcloud would bill users directly and pay Amazon
>> separately. For me, this just wasn't workable, but other projects may
>> need to charge users, even if only to recover some or all of its
>> costs. Web service provision normally costs money, sometimes lots of
>> it. Someone has to pay.
>>
>>> * Ideally, any funding would go through the ipython
>>> foundation/numfocus.  This would allow for profit entities to sponsor
>>> ipython development, but it would allow the foundation to make sure it
>>> was being done in a way that was consistent with the goals of the
>>> project.
>>
>> I'm a fan of sponsorship. If a project had a sponsor that paid the
>> bills, should this prevent it from becoming an official sub-project?
>> It's also unlikely that a company that's willing to do a deal with one
>> project is going to be equally happy to donate to a third party that
>> wont offer anything like the same package.
>>
>>> * Participate in our governance model (although we haven't formalized this).
>>
>> Would this be a requirement, or an open offer?
>
> Wouldn't this have to be a requirement?  By making it a part of the
> IPython organization, you are surrendering the governing rights of the
> project, so to speak.  For example, the admins of the IPython
> organization will be free to give people push access to the
> repository.

Yes.

> Of course, perhaps the governance model can be written so as to give
> the maintainer (i.e., possibly the original author) some higher powers
> over that particular project.  I don't really know what your
> governance model looks like or what you want it to look like, so I
> can't really say much on this point. I'm assuming you have a BDFL
> model with Fernando the BDFL.  If so, maybe the maintainers could be
> like "mini-BDFLs" of their sub-projects.  Or maybe that's a bad idea.

While it is completely informal, we effectively have mini-BDFLs over
parts of the IPython code base.  It is a pretty big code base and it
is tough for any one person to be deeply invested in all of it
(although some of our devs do an amazing job at that!).  But in my
mind, the master BDFL still has final say on everything.  Fernando is
extremely civil and gentle, but there have been times where he has
gone head-to-head with me over things related to the notebook.  I feel
that this is well within his BDFL rights and anytime I sense he is
putting his foot down, I am more than happy to submit (even when we
disagree, I respect him deeply).  But even in the middle of this, I
think that most of us feel like it is the community that drives the
process.  That too is a testament to Fernando's benevolence.

>>
>>> * Have someone who is committed to maintaining the project.
>>
>> This is a difficult thing for an open source developer to commit to
>> long term, but makes perfect sense nonetheless.
>>
>>> * Be approved for inclusion by a vote of the devs who have commit rights.
>>
>> Seems very sensible.
>>
>>> In return, the subproject would get:
>>>
>>> * Possible funding through numfocus/ipython foundation and other grants.
>>
>> This sounds cool, but not if your project is forced to depend on this
>> type of funding exclusively. It discourages commercial involvement and
>> isn't really in keeping with the spirit of BSD licensing, as I
>> understand it.
>>
>>> * Promotion on the ipython website as an official ipython subproject.
>>
>> This'd be cool.
>>
>>> * The participation of the ipython dev team in helping with development.
>>
>> I don't see how this would work. If anyone wanted to send me a patch,
>> they can now. Being an officially recognised sub-project would only
>> make this a little more likely. There's no way IPython can say that,
>> if a project's official, our volunteers will work on it.
>>
>>> * Name recognition from being part of ipython.
>>
>> Sounds good.
>>
>> P.S. As an aside: I'm currently looking to do some stuff with IPython
>> and the NAO robots. This will cost me at least ?3,000 up front. I
>> can't personally donate that kind of money to the public good. I'd
>> need a way to earn it back. I think your proposals make a lot of
>> sense, but I can see the financial requirements being too restrictive,
>> possibly making official sub-projects a very cliquey affair.
>>
>> Personally, I'd probably always opt for unofficial status on every
>> project, just because I don't want to agree to something I might
>> regret down the line. It's a lot of restrictions to accept,
>> perpetually, with potentially nothing much in return.
>>
>> Sorry, if this comes across as negative. It's not meant to at all. I
>> just wanted to offer a different perspective on some of these points.
>> Ultimately, the question has nothing to do with whether nbcloud would
>> 'get in' or not, but how IPython wants to go about adopting projects.
>>
>> All the best
>>
>> Carl
>
> So I guess the question to ask is, what is the benefit from being
> "officially supported" by IPython?  I guess it would result in more
> people finding your project, because of all the official links
> (webpage, possibly in the IPython software itself, etc.).  Other than
> that, the only thing I can think of is that if you have code that
> might potentially be useful to include in IPython itself, then you
> would want to include it just to prevent code duplication and
> unnecessary wheel reinvention.  Like you said, if you are already open
> source and on GitHub, then you can receive pull requests. And if
> IPython core devs help out, you can give them push access, so there's
> little difference from the technical point of view.

I agree that the main benefit is mostly marketing.

> Aaron Meurer
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Mon Sep 24 03:49:10 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 24 Sep 2012 09:49:10 +0200
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
Message-ID: <88474389-8F06-452A-81C9-8231802FC05B@gmail.com>


Le 24 sept. 2012 ? 01:13, Fernando Perez a ?crit :

> On Sun, Sep 23, 2012 at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>> From my perspective, nbviewer is a poor name for this. The focus
>> should be on the content (scientific Python example code), not the
>> tools (the notebook). Within that, we can promote the Notebook, but
>> naming the site after it would be putting the cart before the horse.
> 
> Jumping in a little late here...  Here's my perspective on this:
> IPython is an important piece of the pylab ecosystem, and as I argued
> vociferously on the scipy-user discussion, I think the notebook should
> play an even more prominent role in there.  But it's also very
> important to keep this in mind: ipython and the notebook are also used
> by, and important to, communities that have nothing to do at all with
> scientific work.
> 
> And in the spirit of pylab being a federation of collaborating
> projects, I think the right solution here is to find a way for say
> 'pylab-central' (were it to exist) to use something *like* nbviewer,
> or nbviewer itself, but without having to tightly couple it to pylab
> central, so that it can be equally used by other projects and
> communities that don't have scientific uses in mind.
> 
> Pylab-central could, for notebooks, simply embed (perhaps in an
> iframe) the nbviewer-rendered version of a notebook.

Certainly not as is !
Nbviewer embed remote javascript which would be high security risk for any website 
or user that **trust** ipython.org 

This is one of the reason that I'm reluctant to add github (or other) authentication on nbviewer.
The day we add something like that to nbviewer of if so embed it in another website without proper
care, it is the golden goose for anyone that want to do javascript injection.

In the long term it would be nice to have an api.nbviewer.ipython.org that serve something clean, 
but I just don't want people to try this kind of things for now. 
-- 
Matthias


>  It will be up tow
> the pylab central developers to decide whether they want to get into
> the business of hosting notebooks or they point to people's gists or
> full-fledged repos. nbviewer doesn't care where things are: as long as
> there's a public URL for an ipynb file, it can render it.
> 
> This loose-coupling model seems to me better both in terms of
> acknowledging the structure of the pylab/ipython Venn diagram as well
> as in the spirit of federated collaboration we're trying to establish
> with pylab.
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From carl.input at gmail.com  Mon Sep 24 06:18:44 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 24 Sep 2012 11:18:44 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
Message-ID: <CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>

Thanks Brian for taking the time to explain. I agree with what you're
saying, pretty much entirely; I can see much more clearly now what
kind of mess you're trying to avoid.

Just for the record, I will *never* commercialise the NotebookCloud
service, nor use of the AMI. These are free to all, in every sense of
free. There's no issue there. However, I was considering was the
creation of a library of notebooks, and would be looking to have these
sponsored, so a fully commercial interest would essentially pay to
have their logo at the top right of each notebook. These sponsored
notebooks would be tied to nbcloud; they would always assume the user
was on an nbcloud instance. I'd be using revenue from sponsorship to
finance things like bounties for stuff I need help with on the web app
and AMI side of things. It would all be one, single project, with
sponsorship.

This is all something I've strongly considered doing, so I would want
to keep commercial options open for the project in general. Again, the
nbcloud service is to remain free forever, I'm staunch on that, but
the library of notebooks that build off of it would likely be
commercialised. The closeness of the notebook library to the web
service would make it tricky at my end to separate things completely,
which would make it tricky for you guys to accommodate nbcloud as a
part of IPython.

If IPython want to adopt nbcloud completely, I'd be very happy to see
that happen, on the terms you've set out above, but it would make *me*
less able to commit to it.


From carl.input at gmail.com  Mon Sep 24 06:32:48 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 24 Sep 2012 11:32:48 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
Message-ID: <CAP-uhDcVouBd+Ww1V=ajs-fR6NT7byfWkhiVcu-vm21DSKsONw@mail.gmail.com>

P.S. The robotics stuff is a totally separate project. I just
mentioned it here as it's another example of an open source, IPython
based project that'd need financial support.


From tjhnson at gmail.com  Mon Sep 24 12:19:01 2012
From: tjhnson at gmail.com (T J)
Date: Mon, 24 Sep 2012 11:19:01 -0500
Subject: [IPython-dev] IPython Notebook on Webfaction
In-Reply-To: <CAJ8p+o8VTPgYJVpG_Lzxpus+n7FUDcQ62tdeX=c3HMErJfuJ0Q@mail.gmail.com>
References: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
	<CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>
	<CAJ8p+o8zWR+ex2YjLZNp+e=eYiHFHrOw_sjVGdBRbk-f+Z45AQ@mail.gmail.com>
	<CAHNn8BWHkvvB+3HSfx=aJFuEpLqytzYPmzLMRmj0v-N_72qzdQ@mail.gmail.com>
	<CAJ8p+o8VTPgYJVpG_Lzxpus+n7FUDcQ62tdeX=c3HMErJfuJ0Q@mail.gmail.com>
Message-ID: <CAJ8p+o8jWBHtKuUQr+oiDZH-Z-ig_=9_++Kb_vnkZ5xbB9HDdg@mail.gmail.com>

On Sat, Sep 22, 2012 at 1:54 PM, T J <tjhnson at gmail.com> wrote:

> On Thu, Sep 20, 2012 at 2:42 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>>
>>
>> On Wed, Sep 19, 2012 at 9:32 AM, T J <tjhnson at gmail.com> wrote:
>>
>>> On Wed, Sep 19, 2012 at 4:49 AM, Thomas Kluyver <takowl at gmail.com>wrote:
>>>
>>>> On 19 September 2012 08:49, T J <tjhnson at gmail.com> wrote:
>>>> > the last response from the OP mentions that each open notebook uses 3
>>>> random
>>>> > ports (which differ from the port that the IPython notebook server
>>>> listens
>>>> > on).  Is this true?  This seems like it would be an issue for me.  Can
>>>> > someone explain, in general, what IPython notebook is doing with
>>>> these other
>>>> > ports?
>>>>
>>>> One of them is the main request/response channel (e.g. 'execute this
>>>> code', 'get completions for foo'), one is a publication channel,
>>>> primarily for things printed to stdout, and I think the third is to
>>>> get input for stdin (although the notebook doesn't currently implement
>>>> that).
>>>>
>>>> There's a pull request open that experiments with merging the three
>>>> websocket channels into one - that might help you:
>>>> https://github.com/ipython/ipython/pull/2321
>>>>
>>>>
>>>>
>>> Look promising.  This pull requests seems to make it easier to use the
>>> notebook behind a proxy, but its unclear to me if this allows me to
>>> restrict which the ports the notebooks will try to use.
>>>
>>
>> People get confused by the fact that the notebook communicates with
>> *kernels* via random localhost ports (even there, it connects, it does not
>> listen).  All communications with browsers happen over a single port, and
>> the notebook server only ever listens on that single user-configurable port.
>>
>
>
> Thanks for the clarification. Very helpful.
>

I've given this another try now (still behind the Webfaction proxy), using
pull request #2321, but I'm still getting "404: Not Found".  In order to
use SockJS, must I do any configuration or should it have been automatic?
 Is there a way that I can make IPython provide more detailed log
information so that I can pinpoint why I'm getting the 404s.  Here is what
I see at the command line:

    WARNING:root:404 GET / (127.0.0.1) 0.64ms

My config file is:

c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.base_project_url = '/notebook/'
c.NotebookApp.base_kernel_url = '/notebook/'
c.NotebookApp.port = 26000
c.NotebookApp.open_browser = False
c.NotebookApp.webapp_settings = {'static_url_prefix':'/notebook/static/'}
c.IPKernelApp.pylab = 'inline'
c.IPKernelApp.pylab_import_all = False
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120924/24f653e9/attachment.html>

From jason-sage at creativetrax.com  Mon Sep 24 12:31:08 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Mon, 24 Sep 2012 11:31:08 -0500
Subject: [IPython-dev] IPython Notebook on Webfaction
In-Reply-To: <CAJ8p+o8jWBHtKuUQr+oiDZH-Z-ig_=9_++Kb_vnkZ5xbB9HDdg@mail.gmail.com>
References: <CAJ8p+o-sZ6YPnbbAZVTKk9e7ZaQfBfY0qXysMo7dmVU-1vSF_A@mail.gmail.com>
	<CAOvn4qj40Pt5ZsRKygbgA+waUZkQfQ2a9DHL4UqxikLG7jDHoQ@mail.gmail.com>
	<CAJ8p+o8zWR+ex2YjLZNp+e=eYiHFHrOw_sjVGdBRbk-f+Z45AQ@mail.gmail.com>
	<CAHNn8BWHkvvB+3HSfx=aJFuEpLqytzYPmzLMRmj0v-N_72qzdQ@mail.gmail.com>
	<CAJ8p+o8VTPgYJVpG_Lzxpus+n7FUDcQ62tdeX=c3HMErJfuJ0Q@mail.gmail.com>
	<CAJ8p+o8jWBHtKuUQr+oiDZH-Z-ig_=9_++Kb_vnkZ5xbB9HDdg@mail.gmail.com>
Message-ID: <50608ACC.5040100@creativetrax.com>

On 9/24/12 11:19 AM, T J wrote:
> On Sat, Sep 22, 2012 at 1:54 PM, T J <tjhnson at gmail.com
> <mailto:tjhnson at gmail.com>> wrote:
>
>     On Thu, Sep 20, 2012 at 2:42 PM, MinRK <benjaminrk at gmail.com
>     <mailto:benjaminrk at gmail.com>> wrote:
>
>
>
>         On Wed, Sep 19, 2012 at 9:32 AM, T J <tjhnson at gmail.com
>         <mailto:tjhnson at gmail.com>> wrote:
>
>             On Wed, Sep 19, 2012 at 4:49 AM, Thomas Kluyver
>             <takowl at gmail.com <mailto:takowl at gmail.com>> wrote:
>
>                 On 19 September 2012 08:49, T J <tjhnson at gmail.com
>                 <mailto:tjhnson at gmail.com>> wrote:
>                  > the last response from the OP mentions that each open
>                 notebook uses 3 random
>                  > ports (which differ from the port that the IPython
>                 notebook server listens
>                  > on).  Is this true?  This seems like it would be an
>                 issue for me.  Can
>                  > someone explain, in general, what IPython notebook is
>                 doing with these other
>                  > ports?
>
>                 One of them is the main request/response channel (e.g.
>                 'execute this
>                 code', 'get completions for foo'), one is a publication
>                 channel,
>                 primarily for things printed to stdout, and I think the
>                 third is to
>                 get input for stdin (although the notebook doesn't
>                 currently implement
>                 that).
>
>                 There's a pull request open that experiments with
>                 merging the three
>                 websocket channels into one - that might help you:
>                 https://github.com/ipython/ipython/pull/2321
>
>
>
>             Look promising.  This pull requests seems to make it easier
>             to use the notebook behind a proxy, but its unclear to me if
>             this allows me to restrict which the ports the notebooks
>             will try to use.
>
>
>         People get confused by the fact that the notebook communicates
>         with *kernels* via random localhost ports (even there, it
>         connects, it does not listen).  All communications with browsers
>         happen over a single port, and the notebook server only ever
>         listens on that single user-configurable port.
>
>
>
>     Thanks for the clarification. Very helpful.
>
>
> I've given this another try now (still behind the Webfaction proxy),
> using pull request #2321, but I'm still getting "404: Not Found".  In
> order to use SockJS, must I do any configuration or should it have been
> automatic?

I haven't noticed #2321 before.  Just FYI, we use SockJS in the Sage 
Cell server to communicate with an IPython kernel.  Quick links to our code:

javascript:

https://github.com/sagemath/sagecell/blob/master/static/compute_server.js#L85

https://github.com/sagemath/sagecell/blob/master/static/compute_server.js#L989

server:
https://github.com/sagemath/sagecell/blob/master/handlers.py#L105

Thanks,

Jason



From ellisonbg at gmail.com  Mon Sep 24 15:19:08 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 24 Sep 2012 12:19:08 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
Message-ID: <CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>

On Mon, Sep 24, 2012 at 12:49 AM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 24 sept. 2012 ? 01:13, Fernando Perez a ?crit :
>
>> On Sun, Sep 23, 2012 at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>>> From my perspective, nbviewer is a poor name for this. The focus
>>> should be on the content (scientific Python example code), not the
>>> tools (the notebook). Within that, we can promote the Notebook, but
>>> naming the site after it would be putting the cart before the horse.
>>
>> Jumping in a little late here...  Here's my perspective on this:
>> IPython is an important piece of the pylab ecosystem, and as I argued
>> vociferously on the scipy-user discussion, I think the notebook should
>> play an even more prominent role in there.  But it's also very
>> important to keep this in mind: ipython and the notebook are also used
>> by, and important to, communities that have nothing to do at all with
>> scientific work.
>>
>> And in the spirit of pylab being a federation of collaborating
>> projects, I think the right solution here is to find a way for say
>> 'pylab-central' (were it to exist) to use something *like* nbviewer,
>> or nbviewer itself, but without having to tightly couple it to pylab
>> central, so that it can be equally used by other projects and
>> communities that don't have scientific uses in mind.
>>
>> Pylab-central could, for notebooks, simply embed (perhaps in an
>> iframe) the nbviewer-rendered version of a notebook.
>
> Certainly not as is !
> Nbviewer embed remote javascript which would be high security risk for any website
> or user that **trust** ipython.org

I am beginning to think we should remove <script> tags from markdown
cells because of this.

> This is one of the reason that I'm reluctant to add github (or other) authentication on nbviewer.
> The day we add something like that to nbviewer of if so embed it in another website without proper
> care, it is the golden goose for anyone that want to do javascript injection.

Yep, this is a huge issue.

> In the long term it would be nice to have an api.nbviewer.ipython.org that serve something clean,
> but I just don't want people to try this kind of things for now.
> --
> Matthias
>
>
>>  It will be up tow
>> the pylab central developers to decide whether they want to get into
>> the business of hosting notebooks or they point to people's gists or
>> full-fledged repos. nbviewer doesn't care where things are: as long as
>> there's a public URL for an ipynb file, it can render it.
>>
>> This loose-coupling model seems to me better both in terms of
>> acknowledging the structure of the pylab/ipython Venn diagram as well
>> as in the spirit of federated collaboration we're trying to establish
>> with pylab.
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Mon Sep 24 15:21:50 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 24 Sep 2012 12:21:50 -0700
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
Message-ID: <CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>

Carl,

How about this - why don't you develop notebook cloud separately from
ipython for now.  If in the future it looks like it would make sense
to bring in as an official ipython subproject, we can consider it.  I
think we needs to see where exactly it goes first.

Cheers,

Brian

On Mon, Sep 24, 2012 at 3:18 AM, Carl Smith <carl.input at gmail.com> wrote:
> Thanks Brian for taking the time to explain. I agree with what you're
> saying, pretty much entirely; I can see much more clearly now what
> kind of mess you're trying to avoid.
>
> Just for the record, I will *never* commercialise the NotebookCloud
> service, nor use of the AMI. These are free to all, in every sense of
> free. There's no issue there. However, I was considering was the
> creation of a library of notebooks, and would be looking to have these
> sponsored, so a fully commercial interest would essentially pay to
> have their logo at the top right of each notebook. These sponsored
> notebooks would be tied to nbcloud; they would always assume the user
> was on an nbcloud instance. I'd be using revenue from sponsorship to
> finance things like bounties for stuff I need help with on the web app
> and AMI side of things. It would all be one, single project, with
> sponsorship.
>
> This is all something I've strongly considered doing, so I would want
> to keep commercial options open for the project in general. Again, the
> nbcloud service is to remain free forever, I'm staunch on that, but
> the library of notebooks that build off of it would likely be
> commercialised. The closeness of the notebook library to the web
> service would make it tricky at my end to separate things completely,
> which would make it tricky for you guys to accommodate nbcloud as a
> part of IPython.
>
> If IPython want to adopt nbcloud completely, I'd be very happy to see
> that happen, on the terms you've set out above, but it would make *me*
> less able to commit to it.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From d.warde.farley at gmail.com  Mon Sep 24 15:25:44 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Mon, 24 Sep 2012 15:25:44 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
Message-ID: <CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>

On Sun, Sep 23, 2012 at 2:33 AM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> On Fri, Sep 21, 2012 at 5:37 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>> Yup.  Just a few days ago I had to fix this very problem for a
>> colleague (though in her case, caused by a system-wide EPD install
>> where setuptools was overriding any local PYTHONPATH configuration).
>> After some frustration, I learned of the file 'usercustomize.py':
>>
>> http://docs.python.org/library/site.html
>>
>> Put a file with this name *anywhere* in your PYTHONPATH, and fix
>> sys.path manually in there to reorder things as needed, e.g.
>>
>> sys.path = [os.environ['HOME']+'.local/lib/python2.7/site-packages/'] + sys.path
>
> Thanks Fernando! I'll give this a try.

Hmm, this doesn't seem to work, unfortunately. At least, a whole lot
of .egg directories get added to the front of sys.path after
usercustomize.py in ~/.local/lib/site-packages is executed, including
the IPython egg directory. How very annoying.

David


From jason-sage at creativetrax.com  Mon Sep 24 15:31:01 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Mon, 24 Sep 2012 14:31:01 -0500
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
Message-ID: <5060B4F5.9000005@creativetrax.com>

On 9/24/12 2:19 PM, Brian Granger wrote:
>> Certainly not as is !
>> >Nbviewer embed remote javascript which would be high security risk for any website
>> >or user that **trust** ipython.org
> I am beginning to think we should remove <script> tags from markdown
> cells because of this.
>

Don't serve user-generated content from ipython.org.  Serve 
user-generated content from something like pylab-central.org or 
something.  Some time ago, someone (William Stein maybe?) forwarded to 
me a talk from someone at google which said something to the effect that 
taking care of all the vulnerabilities is *hard*, and google finally 
just decided to serve any untrusted content from a different domain. 
(yeah, I know---that chain of hearsay is not extremely inspiring...). 
I'm CCing William in hopes that maybe he was the one that forwarded the 
story and can find it (I've looked but can't find it).

But the end result was---don't server untrusted material from a trusted 
domain.

That said, I guess we're breaking that rule with interact.sagemath.org 
(Sage's answer to something like scipy central, at least for small 
snippets).

Thanks,

Jason



From bussonniermatthias at gmail.com  Mon Sep 24 15:37:56 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 24 Sep 2012 21:37:56 +0200
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
Message-ID: <EA44F306-E5D6-42D1-B245-00975E63AE10@gmail.com>


Le 24 sept. 2012 ? 21:19, Brian Granger a ?crit :

> On Mon, Sep 24, 2012 at 12:49 AM, Matthias BUSSONNIER
> <bussonniermatthias at gmail.com> wrote:
>> 
>> Le 24 sept. 2012 ? 01:13, Fernando Perez a ?crit :
>> 
>>> On Sun, Sep 23, 2012 at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>>>> From my perspective, nbviewer is a poor name for this. The focus
>>>> should be on the content (scientific Python example code), not the
>>>> tools (the notebook). Within that, we can promote the Notebook, but
>>>> naming the site after it would be putting the cart before the horse.
>>> 
>>> Jumping in a little late here...  Here's my perspective on this:
>>> IPython is an important piece of the pylab ecosystem, and as I argued
>>> vociferously on the scipy-user discussion, I think the notebook should
>>> play an even more prominent role in there.  But it's also very
>>> important to keep this in mind: ipython and the notebook are also used
>>> by, and important to, communities that have nothing to do at all with
>>> scientific work.
>>> 
>>> And in the spirit of pylab being a federation of collaborating
>>> projects, I think the right solution here is to find a way for say
>>> 'pylab-central' (were it to exist) to use something *like* nbviewer,
>>> or nbviewer itself, but without having to tightly couple it to pylab
>>> central, so that it can be equally used by other projects and
>>> communities that don't have scientific uses in mind.
>>> 
>>> Pylab-central could, for notebooks, simply embed (perhaps in an
>>> iframe) the nbviewer-rendered version of a notebook.
>> 
>> Certainly not as is !
>> Nbviewer embed remote javascript which would be high security risk for any website
>> or user that **trust** ipython.org
> 
> I am beginning to think we should remove <script> tags from markdown
> cells because of this.

This is **not** an issues of <script> tag in markdown cell. 
you can insert html almost everywhere, we should totally assumed 
that people can forge a false notebook file on their own.

So script in html output, or maybe even in code cell source. 

One way I see is iframe sandboxing with recent browser. 
-- 
Matthias


> 
>> This is one of the reason that I'm reluctant to add github (or other) authentication on nbviewer.
>> The day we add something like that to nbviewer of if so embed it in another website without proper
>> care, it is the golden goose for anyone that want to do javascript injection.
> 
> Yep, this is a huge issue.
> 
>> In the long term it would be nice to have an api.nbviewer.ipython.org that serve something clean,
>> but I just don't want people to try this kind of things for now.
>> --
>> Matthias
>> 
>> 
>>> It will be up tow
>>> the pylab central developers to decide whether they want to get into
>>> the business of hosting notebooks or they point to people's gists or
>>> full-fledged repos. nbviewer doesn't care where things are: as long as
>>> there's a public URL for an ipynb file, it can render it.
>>> 
>>> This loose-coupling model seems to me better both in terms of
>>> acknowledging the structure of the pylab/ipython Venn diagram as well
>>> as in the spirit of federated collaboration we're trying to establish
>>> with pylab.
>>> 
>>> Cheers,
>>> 
>>> f
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From carl.input at gmail.com  Mon Sep 24 15:41:33 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 24 Sep 2012 20:41:33 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
	<CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>
Message-ID: <CAP-uhDcw0_oVukf1_tOn9QQsyYY=CEHYUgiyLQ6O=cwhSmXOgg@mail.gmail.com>

On 24 September 2012 20:21, Brian Granger <ellisonbg at gmail.com> wrote:
> Carl,
>
> How about this - why don't you develop notebook cloud separately from
> ipython for now.  If in the future it looks like it would make sense
> to bring in as an official ipython subproject, we can consider it.  I
> think we needs to see where exactly it goes first.
>
> Cheers,
>
> Brian

No problem at all. I wasn't actually pushing for NotebookCloud to be
adopted; I just saw it mentioned in this thread and wanted to reply to
those comments. I only want what's best for IPython.

I would like to see NotebookCloud added to the list of projects that
use IPython, but actual adoption is something you guys can do any time
you think it'd be good for IPython. It's open source after all.

Thanks for taking the time to discuss this with me.

All the best

Carl


From bussonniermatthias at gmail.com  Mon Sep 24 15:55:33 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 24 Sep 2012 21:55:33 +0200
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <5060B4F5.9000005@creativetrax.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
	<5060B4F5.9000005@creativetrax.com>
Message-ID: <D1623FD1-8926-4A59-9FA5-3EE5453428E1@gmail.com>


Le 24 sept. 2012 ? 21:31, Jason Grout a ?crit :

> On 9/24/12 2:19 PM, Brian Granger wrote:
>>> Certainly not as is !
>>>> Nbviewer embed remote javascript which would be high security risk for any website
>>>> or user that **trust** ipython.org
>> I am beginning to think we should remove <script> tags from markdown
>> cells because of this.
>> 
> 
> Don't serve user-generated content from ipython.org.  Serve 
> user-generated content from something like pylab-central.org or 
> something.  Some time ago, someone (William Stein maybe?) forwarded to 
> me a talk from someone at google which said something to the effect that 
> taking care of all the vulnerabilities is *hard*, and google finally 
> just decided to serve any untrusted content from a different domain. 
> (yeah, I know---that chain of hearsay is not extremely inspiring...). 
> I'm CCing William in hopes that maybe he was the one that forwarded the 
> story and can find it (I've looked but can't find it).
> 
> But the end result was---don't server untrusted material from a trusted 
> domain.

Thanks, 
I don't quite see how we could do that as ipynb is transformed to html/css/javascript on the server side, 
and we would like to allow some kind of integration with github like possibility for users to post comment directly from nbviewer.
But i'm not an expert on web technologies, and i'm sure this problem has been encounters elsewhere and has a solution. 

Obviously with the nbviewer, we can restrict javascript execution, but this is more an issue with notebook collaboration where user will expect to be able to publish javascript.
I think html5 sandbox could be a solution, but I haven't  found any good resources to clearly understand what could be done.

Still, thanks for the suggestion, i'll see what i can do with difference domains.
-- 
Matthias



From takowl at gmail.com  Mon Sep 24 16:11:30 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 24 Sep 2012 21:11:30 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAP-uhDcw0_oVukf1_tOn9QQsyYY=CEHYUgiyLQ6O=cwhSmXOgg@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
	<CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>
	<CAP-uhDcw0_oVukf1_tOn9QQsyYY=CEHYUgiyLQ6O=cwhSmXOgg@mail.gmail.com>
Message-ID: <CAOvn4qh12aWySh_RSU=Q4=vFevmiGMtgRHh3B5PGCJ9hN+8Tag@mail.gmail.com>

On 24 September 2012 20:41, Carl Smith <carl.input at gmail.com> wrote:
> I would like to see NotebookCloud added to the list of projects that
> use IPython, but actual adoption is something you guys can do any time
> you think it'd be good for IPython. It's open source after all.

To be clear, it's absolutely fine to make something like NotebookCloud
without it being an official IPython project.

You're welcome to add NotebookCloud - and any relevant project - to
the list of projects using IPython. That page is kind of a vanity
thing (I think Fernando uses it to demonstrate the impact of IPython
to potential funders), so we're happy to see projects added to the
list.

Thanks,
Thomas


From d.warde.farley at gmail.com  Mon Sep 24 16:35:19 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Mon, 24 Sep 2012 16:35:19 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
Message-ID: <CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>

On Mon, Sep 24, 2012 at 3:25 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:

> Hmm, this doesn't seem to work, unfortunately. At least, a whole lot
> of .egg directories get added to the front of sys.path after
> usercustomize.py in ~/.local/lib/site-packages is executed, including
> the IPython egg directory. How very annoying.

Great, and setuptools path mangling means that all that garbage is
also added *before* whatever's added by $PYTHONPATH, so I can't put
~/src/ipython at the front of the PYTHONPATH to fix it.

Thanks a lot, setuptools. So am I cooked?

David


From carl.input at gmail.com  Mon Sep 24 16:36:41 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 24 Sep 2012 21:36:41 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAOvn4qh12aWySh_RSU=Q4=vFevmiGMtgRHh3B5PGCJ9hN+8Tag@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
	<CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>
	<CAP-uhDcw0_oVukf1_tOn9QQsyYY=CEHYUgiyLQ6O=cwhSmXOgg@mail.gmail.com>
	<CAOvn4qh12aWySh_RSU=Q4=vFevmiGMtgRHh3B5PGCJ9hN+8Tag@mail.gmail.com>
Message-ID: <CAP-uhDd49jH9Awj5q9NOubv8b+tUJF6ORDRqyu5fP6ecx5_kSg@mail.gmail.com>

Cool Thomas. Cheers.
On Sep 24, 2012 9:12 PM, "Thomas Kluyver" <takowl at gmail.com> wrote:

> On 24 September 2012 20:41, Carl Smith <carl.input at gmail.com> wrote:
> > I would like to see NotebookCloud added to the list of projects that
> > use IPython, but actual adoption is something you guys can do any time
> > you think it'd be good for IPython. It's open source after all.
>
> To be clear, it's absolutely fine to make something like NotebookCloud
> without it being an official IPython project.
>
> You're welcome to add NotebookCloud - and any relevant project - to
> the list of projects using IPython. That page is kind of a vanity
> thing (I think Fernando uses it to demonstrate the impact of IPython
> to potential funders), so we're happy to see projects added to the
> list.
>
> Thanks,
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120924/71b850b5/attachment.html>

From carl.input at gmail.com  Mon Sep 24 17:02:32 2012
From: carl.input at gmail.com (Carl Smith)
Date: Mon, 24 Sep 2012 22:02:32 +0100
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAP-uhDd49jH9Awj5q9NOubv8b+tUJF6ORDRqyu5fP6ecx5_kSg@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
	<CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>
	<CAP-uhDcw0_oVukf1_tOn9QQsyYY=CEHYUgiyLQ6O=cwhSmXOgg@mail.gmail.com>
	<CAOvn4qh12aWySh_RSU=Q4=vFevmiGMtgRHh3B5PGCJ9hN+8Tag@mail.gmail.com>
	<CAP-uhDd49jH9Awj5q9NOubv8b+tUJF6ORDRqyu5fP6ecx5_kSg@mail.gmail.com>
Message-ID: <CAP-uhDcBcs2x2q=J=J8kDNBxGUeTDbz2kOyUNpGhNNGV6CS-_A@mail.gmail.com>

I added NotebookCloud to the wiki page mentioned above.


From ellisonbg at gmail.com  Mon Sep 24 17:11:12 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 24 Sep 2012 14:11:12 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <5060B4F5.9000005@creativetrax.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
	<5060B4F5.9000005@creativetrax.com>
Message-ID: <CAH4pYpQBrLsmejvzOs5UR6o9wpjwPdLunSU+6=bcczR0cRVEaA@mail.gmail.com>

On Mon, Sep 24, 2012 at 12:31 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> On 9/24/12 2:19 PM, Brian Granger wrote:
>>> Certainly not as is !
>>> >Nbviewer embed remote javascript which would be high security risk for any website
>>> >or user that **trust** ipython.org
>> I am beginning to think we should remove <script> tags from markdown
>> cells because of this.
>>
>
> Don't serve user-generated content from ipython.org.  Serve
> user-generated content from something like pylab-central.org or
> something.  Some time ago, someone (William Stein maybe?) forwarded to
> me a talk from someone at google which said something to the effect that
> taking care of all the vulnerabilities is *hard*, and google finally
> just decided to serve any untrusted content from a different domain.
> (yeah, I know---that chain of hearsay is not extremely inspiring...).
> I'm CCing William in hopes that maybe he was the one that forwarded the
> story and can find it (I've looked but can't find it).

For us it is not as simple as using a different domain because of the
way we load and run javascript code.  In short we:

* Get the JS code embedded in a string inside a JSON message.
* We unpack it and then eval it in the context of the cells output area.

The domain tricks don't work in a setting like this.

> But the end result was---don't server untrusted material from a trusted
> domain.
>
> That said, I guess we're breaking that rule with interact.sagemath.org
> (Sage's answer to something like scipy central, at least for small
> snippets).
>
> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Mon Sep 24 17:20:34 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 24 Sep 2012 23:20:34 +0200
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <CAH4pYpQBrLsmejvzOs5UR6o9wpjwPdLunSU+6=bcczR0cRVEaA@mail.gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
	<5060B4F5.9000005@creativetrax.com>
	<CAH4pYpQBrLsmejvzOs5UR6o9wpjwPdLunSU+6=bcczR0cRVEaA@mail.gmail.com>
Message-ID: <55FDA49A-068F-49C6-9A71-AD49D0EC3527@gmail.com>


Le 24 sept. 2012 ? 23:11, Brian Granger a ?crit :

> On Mon, Sep 24, 2012 at 12:31 PM, Jason Grout
> <jason-sage at creativetrax.com> wrote:
>> On 9/24/12 2:19 PM, Brian Granger wrote:
>>>> Certainly not as is !
>>>>> Nbviewer embed remote javascript which would be high security risk for any website
>>>>> or user that **trust** ipython.org
>>> I am beginning to think we should remove <script> tags from markdown
>>> cells because of this.
>>> 
>> 
>> Don't serve user-generated content from ipython.org.  Serve
>> user-generated content from something like pylab-central.org or
>> something.  Some time ago, someone (William Stein maybe?) forwarded to
>> me a talk from someone at google which said something to the effect that
>> taking care of all the vulnerabilities is *hard*, and google finally
>> just decided to serve any untrusted content from a different domain.
>> (yeah, I know---that chain of hearsay is not extremely inspiring...).
>> I'm CCing William in hopes that maybe he was the one that forwarded the
>> story and can find it (I've looked but can't find it).
> 
> For us it is not as simple as using a different domain because of the
> way we load and run javascript code.  In short we:
> 
> * Get the JS code embedded in a string inside a JSON message.
> * We unpack it and then eval it in the context of the cells output area.

For notebook, yes, not for nbviewer.
Displayed js is not evaluated in nbviewer, only script tags. 

Even displayed js is not evaluated when loading notebooks. 
-- 
Matthias

From ellisonbg at gmail.com  Mon Sep 24 17:22:27 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 24 Sep 2012 14:22:27 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <55FDA49A-068F-49C6-9A71-AD49D0EC3527@gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
	<5060B4F5.9000005@creativetrax.com>
	<CAH4pYpQBrLsmejvzOs5UR6o9wpjwPdLunSU+6=bcczR0cRVEaA@mail.gmail.com>
	<55FDA49A-068F-49C6-9A71-AD49D0EC3527@gmail.com>
Message-ID: <CAH4pYpQzrdv+F6m7stXmB_2cdgAa1NZBoMy-za=qebEv0U89Vw@mail.gmail.com>

On Mon, Sep 24, 2012 at 2:20 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 24 sept. 2012 ? 23:11, Brian Granger a ?crit :
>
>> On Mon, Sep 24, 2012 at 12:31 PM, Jason Grout
>> <jason-sage at creativetrax.com> wrote:
>>> On 9/24/12 2:19 PM, Brian Granger wrote:
>>>>> Certainly not as is !
>>>>>> Nbviewer embed remote javascript which would be high security risk for any website
>>>>>> or user that **trust** ipython.org
>>>> I am beginning to think we should remove <script> tags from markdown
>>>> cells because of this.
>>>>
>>>
>>> Don't serve user-generated content from ipython.org.  Serve
>>> user-generated content from something like pylab-central.org or
>>> something.  Some time ago, someone (William Stein maybe?) forwarded to
>>> me a talk from someone at google which said something to the effect that
>>> taking care of all the vulnerabilities is *hard*, and google finally
>>> just decided to serve any untrusted content from a different domain.
>>> (yeah, I know---that chain of hearsay is not extremely inspiring...).
>>> I'm CCing William in hopes that maybe he was the one that forwarded the
>>> story and can find it (I've looked but can't find it).
>>
>> For us it is not as simple as using a different domain because of the
>> way we load and run javascript code.  In short we:
>>
>> * Get the JS code embedded in a string inside a JSON message.
>> * We unpack it and then eval it in the context of the cells output area.
>
> For notebook, yes, not for nbviewer.
> Displayed js is not evaluated in nbviewer, only script tags.
>
> Even displayed js is not evaluated when loading notebooks.

Right, thanks for the clarification.

Cheers,

Brian

> --
> Matthias
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Mon Sep 24 17:25:39 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 24 Sep 2012 14:25:39 -0700
Subject: [IPython-dev] Scipy central & IPython notebook.
In-Reply-To: <EA44F306-E5D6-42D1-B245-00975E63AE10@gmail.com>
References: <CAOvn4qhHjifoZXsMxh_TXh44FhneN3hTJMPodwJX_bER0SR-QQ@mail.gmail.com>
	<CAH4pYpQWmQxVUYRG6C9Q2fcwU5hojBhc+Lk7jM7Kc2dJnxzdWQ@mail.gmail.com>
	<CAH4pYpQwbBXdz5Ssor39Uc5kMPZxjdTwHzk-s8izzwZqMMf9-g@mail.gmail.com>
	<CAOvn4qhGraqnmmktOx4YJWiDqENkQqMvHwKn9L9bEPce45uM-w@mail.gmail.com>
	<CAH4pYpTSFCTrdLUJe884WBGMv=y=XG2Xe=WQRnUcg8UvHNUvHQ@mail.gmail.com>
	<CAOvn4qhSdRt21JptEtsWVH1eY3eQO77Bfwc4YGv5VG=7OFi3sA@mail.gmail.com>
	<CAHAreOr7RhMjOvmUHi7i-5eHLtYADVtt5KS9g1S+r3bW0RCM1g@mail.gmail.com>
	<88474389-8F06-452A-81C9-8231802FC05B@gmail.com>
	<CAH4pYpQuOQ117D3=PJGBftR4ECyFQ9kt866wjHxrxiSANq5NbQ@mail.gmail.com>
	<EA44F306-E5D6-42D1-B245-00975E63AE10@gmail.com>
Message-ID: <CAH4pYpTGncn6E4gKZLDjY-D8g6iJksCn5U-9LrO69O8_ngtkXw@mail.gmail.com>

On Mon, Sep 24, 2012 at 12:37 PM, Matthias BUSSONNIER
<bussonniermatthias at gmail.com> wrote:
>
> Le 24 sept. 2012 ? 21:19, Brian Granger a ?crit :
>
>> On Mon, Sep 24, 2012 at 12:49 AM, Matthias BUSSONNIER
>> <bussonniermatthias at gmail.com> wrote:
>>>
>>> Le 24 sept. 2012 ? 01:13, Fernando Perez a ?crit :
>>>
>>>> On Sun, Sep 23, 2012 at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>>>>> From my perspective, nbviewer is a poor name for this. The focus
>>>>> should be on the content (scientific Python example code), not the
>>>>> tools (the notebook). Within that, we can promote the Notebook, but
>>>>> naming the site after it would be putting the cart before the horse.
>>>>
>>>> Jumping in a little late here...  Here's my perspective on this:
>>>> IPython is an important piece of the pylab ecosystem, and as I argued
>>>> vociferously on the scipy-user discussion, I think the notebook should
>>>> play an even more prominent role in there.  But it's also very
>>>> important to keep this in mind: ipython and the notebook are also used
>>>> by, and important to, communities that have nothing to do at all with
>>>> scientific work.
>>>>
>>>> And in the spirit of pylab being a federation of collaborating
>>>> projects, I think the right solution here is to find a way for say
>>>> 'pylab-central' (were it to exist) to use something *like* nbviewer,
>>>> or nbviewer itself, but without having to tightly couple it to pylab
>>>> central, so that it can be equally used by other projects and
>>>> communities that don't have scientific uses in mind.
>>>>
>>>> Pylab-central could, for notebooks, simply embed (perhaps in an
>>>> iframe) the nbviewer-rendered version of a notebook.
>>>
>>> Certainly not as is !
>>> Nbviewer embed remote javascript which would be high security risk for any website
>>> or user that **trust** ipython.org
>>
>> I am beginning to think we should remove <script> tags from markdown
>> cells because of this.
>
> This is **not** an issues of <script> tag in markdown cell.
> you can insert html almost everywhere, we should totally assumed
> that people can forge a false notebook file on their own.

As far as I know a valid notebook file can only load <script> tags in
two locations:

* In markdown cells.
* In JS/HTML output.

Are there others that I am missing?

Here is my current thinking about this issue:

1. We should *never* allow JS code to be run without the user of the
notebook explicitly asking for it.  This mean that merely
loading/rendering a notebook should never cause JS code to run.
2.  This means that we should strip <script> tags out of markdown
cells completely as markdown is rendered to HTML and displayed at load
time.
3.  I don't don't what to do about JS/HTML output - it one of the most
useful parts of the notebook.  We should probably investigate
sandboxing techniques...but minimally we should probably warn users
before we run this code (although what a pain from the UI
perspective).

Cheers,

Brian

> So script in html output, or maybe even in code cell source.
>
> One way I see is iframe sandboxing with recent browser.
> --
> Matthias
>
>
>>
>>> This is one of the reason that I'm reluctant to add github (or other) authentication on nbviewer.
>>> The day we add something like that to nbviewer of if so embed it in another website without proper
>>> care, it is the golden goose for anyone that want to do javascript injection.
>>
>> Yep, this is a huge issue.
>>
>>> In the long term it would be nice to have an api.nbviewer.ipython.org that serve something clean,
>>> but I just don't want people to try this kind of things for now.
>>> --
>>> Matthias
>>>
>>>
>>>> It will be up tow
>>>> the pylab central developers to decide whether they want to get into
>>>> the business of hosting notebooks or they point to people's gists or
>>>> full-fledged repos. nbviewer doesn't care where things are: as long as
>>>> there's a public URL for an ipynb file, it can render it.
>>>>
>>>> This loose-coupling model seems to me better both in terms of
>>>> acknowledging the structure of the pylab/ipython Venn diagram as well
>>>> as in the spirit of federated collaboration we're trying to establish
>>>> with pylab.
>>>>
>>>> Cheers,
>>>>
>>>> f
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Mon Sep 24 17:26:52 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 24 Sep 2012 14:26:52 -0700
Subject: [IPython-dev] [IPython-User] The site needs an update... Web
 team, anyone?
In-Reply-To: <CAOvn4qh12aWySh_RSU=Q4=vFevmiGMtgRHh3B5PGCJ9hN+8Tag@mail.gmail.com>
References: <CAHAreOroq23URESsWwSmUnRjtz3utKNMyzeuTnZTL9CzS=sH2g@mail.gmail.com>
	<CAH4pYpQOs-55Hs-jMkPthDSiaL5F83Dp6H5WujDNBBEZwGGbPw@mail.gmail.com>
	<CAP-uhDcdqUs0b1QQ-r7S-Sxk2x=m30CAsemxj=-fxrv-88FBxQ@mail.gmail.com>
	<CAKgW=6L4juAbPMG8f7-qrNSUyjv8dDu2f_J42Ob81wjxHMA+RQ@mail.gmail.com>
	<CAH4pYpQOS+cLykRT67wX5EuW650jA9PReRYi-aMgkegJOoXN-A@mail.gmail.com>
	<CAP-uhDfZScVaFJTvaD01+O_Ua+eFQO+xhuy3+TamHs78ZmyGwg@mail.gmail.com>
	<CAH4pYpS2iSs7zf7Hbe3fYWsY7PhrC2vxR_vyng+S8asj+Ua49g@mail.gmail.com>
	<CAP-uhDcw0_oVukf1_tOn9QQsyYY=CEHYUgiyLQ6O=cwhSmXOgg@mail.gmail.com>
	<CAOvn4qh12aWySh_RSU=Q4=vFevmiGMtgRHh3B5PGCJ9hN+8Tag@mail.gmail.com>
Message-ID: <CAH4pYpRCNANfBRFWMUJO7vP9svTRYeTGibQTWeuof6dmmdnJcw@mail.gmail.com>

On Mon, Sep 24, 2012 at 1:11 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 24 September 2012 20:41, Carl Smith <carl.input at gmail.com> wrote:
>> I would like to see NotebookCloud added to the list of projects that
>> use IPython, but actual adoption is something you guys can do any time
>> you think it'd be good for IPython. It's open source after all.
>
> To be clear, it's absolutely fine to make something like NotebookCloud
> without it being an official IPython project.

+1, sorry if this wasn't clear.  Thanks Thomas.

> You're welcome to add NotebookCloud - and any relevant project - to
> the list of projects using IPython. That page is kind of a vanity
> thing (I think Fernando uses it to demonstrate the impact of IPython
> to potential funders), so we're happy to see projects added to the
> list.
>
> Thanks,
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Mon Sep 24 18:40:43 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 24 Sep 2012 15:40:43 -0700
Subject: [IPython-dev] [ANN] New release of PythonTutor
Message-ID: <CAHAreOobHFDfMLQ19cwuWxmDsnUK5M4fSMYLn3BdN9Cotiu1pw@mail.gmail.com>

Hi folks,

many of you probably have seen Philip Guo's awesome PythonTutor, he
just put out a new release:

www.pythontutor.com
https://plus.google.com/u/0/+ResearchatGoogle/posts/cseo9qi7LWq

this is something that we've talked about at least with some folks in
private, and ideas for integration with the notebook are worth
considering.  Incidentally, I'll be meeting Philip at Google in a few
weeks (we've talked several times in the past and have had very
productive discussions), so if there's anything you can think of at
the ipython/pythontutor intersection, let us know here and Philip and
I will hash it out in person and can report back.

Cheers,

f


From fperez.net at gmail.com  Mon Sep 24 20:08:29 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 24 Sep 2012 17:08:29 -0700
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
Message-ID: <CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>

On Mon, Sep 24, 2012 at 1:35 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> Great, and setuptools path mangling means that all that garbage is
> also added *before* whatever's added by $PYTHONPATH, so I can't put
> ~/src/ipython at the front of the PYTHONPATH to fix it.
>
> Thanks a lot, setuptools. So am I cooked?

Mmh, that's weird: in the system we were, the usercustomize did happen
after the setuptools nonsense, so we were able to work around that.

Setuptools is simply an abomination, period.  It breaks fundamental
constructs of how python is supposed to work and makes certain things
that are defined in the design of Python impossible.  It's a shame
that it's so widely used, and this is why in ipython we have a strict
policy of avoiding its use as much as possible.

I can only suggest one of two ugly solutions:

- patch your startup scripts for any tool you want
- patch the python call itself: put ~/.local/bin/python to be your own
python script that fixes sys.path for you, and then executes whatever
else was in sys.argv.

Either of these is ugly as hell, but there is no elegant way to slay
setuptools.  It digs itself so deep into the system, undermining every
natural and clean design with underhanded tactics, that only pretty
brutal and ham-fisted solutions seem to be possible.

Sorry you're having to deal with this. Over the years, setuptools has
cost me so much wasted time (precisely because it does stuff like
this) that I have absolutely no tolerance for it.  I can only hope one
day the python world will be rid of that abomination.

Cheers,

f


From vanderplas at astro.washington.edu  Mon Sep 24 21:36:22 2012
From: vanderplas at astro.washington.edu (Jake Vanderplas)
Date: Mon, 24 Sep 2012 18:36:22 -0700
Subject: [IPython-dev] notebooks embedded in sphinx
Message-ID: <50610A96.20507@astro.washington.edu>

Hi all,
I've been thinking about the best way to embed notebooks in sphinx 
documentation using nbconvert. I've opened an initial PR at 
http://github.com/ipython/nbconvert/pull/35/
The current version is a simple sphinx extension based on the raw html 
directive, but results in non-standard html that may not work in some 
browsers (i.e., style tags outside the html header).

I'm still not sure the best way to progress, but I'd appreciate any 
feedback or ideas.
Thanks!
    Jake


From damianavila at gmail.com  Mon Sep 24 23:09:28 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Tue, 25 Sep 2012 00:09:28 -0300
Subject: [IPython-dev] notebooks embedded in sphinx
In-Reply-To: <50610A96.20507@astro.washington.edu>
References: <50610A96.20507@astro.washington.edu>
Message-ID: <50612068.40109@gmail.com>

El 24/09/12 22:36, Jake Vanderplas escribi?:
> Hi all,
> I've been thinking about the best way to embed notebooks in sphinx
> documentation using nbconvert. I've opened an initial PR at
> http://github.com/ipython/nbconvert/pull/35/
> The current version is a simple sphinx extension based on the raw html
> directive, but results in non-standard html that may not work in some
> browsers (i.e., style tags outside the html header).
>
> I'm still not sure the best way to progress, but I'd appreciate any
> feedback or ideas.
> Thanks!
>      Jake
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Hi Jake,

I have been playing with nbconvert to embed the static html of the 
notebook into a static site generator (nikola).
I have also used the raw:: html directive and the css styling provided 
by the html-header from nbconvert.
I have also modified the template html provided by nikola.
I am testing some other things, but you can see an example here: 
http://damianavila.github.com/Testing 
<http://damianavila.github.com/Testing/>
Maybe, as you suggest in the github discussion, you could write a 
template in Sphinx to embed correctly the notebook.

Cheers.

Dami?n.





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120925/d6591c15/attachment.html>

From jason-sage at creativetrax.com  Tue Sep 25 08:22:50 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 25 Sep 2012 07:22:50 -0500
Subject: [IPython-dev] splitter not transforming a line
Message-ID: <5061A21A.5020701@creativetrax.com>

In the input splitter at 
https://github.com/ipython/ipython/blob/master/IPython/core/inputsplitter.py#L907, 
it appears that the test makes sure that the "def" line in the below 
code is not run through the transformer:

@interact
def f(n=1):
     print n


Since it is not complete and the buffer has the @interact line in it and 
the @interact line does not end with ':' or ',', the transform is never 
run.  This is a big problem for Sage, since Sage wants to transform that 
1 to Integer(1) in the preparser.

Is this intentional?  There seem to be enough comments like 
https://github.com/ipython/ipython/blob/master/IPython/core/inputsplitter.py#L883 
that indicate that maybe this is an oversight in a tricky part of the code.

Thanks,

Jason


From takowl at gmail.com  Tue Sep 25 08:35:36 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 25 Sep 2012 13:35:36 +0100
Subject: [IPython-dev] splitter not transforming a line
In-Reply-To: <5061A21A.5020701@creativetrax.com>
References: <5061A21A.5020701@creativetrax.com>
Message-ID: <CAOvn4qi2J0ttOFK_+ci=v75j2Z2GYmDqgyBOF7mw_X9DbvuH+w@mail.gmail.com>

On 25 September 2012 13:22, Jason Grout <jason-sage at creativetrax.com> wrote:
> Is this intentional?  There seem to be enough comments like
> https://github.com/ipython/ipython/blob/master/IPython/core/inputsplitter.py#L883
> that indicate that maybe this is an oversight in a tricky part of the code.

It is quite possibly an oversight - all of that part of the code is
complex. Sorry, my work on IPEP2 has been sidelined while I've been
pushing Pylab forwards.

> Since it is not complete and the buffer has the @interact line in it and
> the @interact line does not end with ':' or ',', the transform is never
> run.  This is a big problem for Sage, since Sage wants to transform that
> 1 to Integer(1) in the preparser.

Once it's in place, I would encourage you to do this with the new AST
transformation hook:
https://github.com/ipython/ipython/pull/2301

This was inspired by Sympy wanting to do exactly the same
transformation. I still need to do a bit of work, but I think I can
see how to complete it.

Thanks,
Thomas


From tomspur at fedoraproject.org  Tue Sep 25 08:41:43 2012
From: tomspur at fedoraproject.org (Thomas Spura)
Date: Tue, 25 Sep 2012 14:41:43 +0200
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
	<CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
Message-ID: <CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>

On Tue, Sep 25, 2012 at 2:08 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Mon, Sep 24, 2012 at 1:35 PM, David Warde-Farley
> <d.warde.farley at gmail.com> wrote:
>> Great, and setuptools path mangling means that all that garbage is
>> also added *before* whatever's added by $PYTHONPATH, so I can't put
>> ~/src/ipython at the front of the PYTHONPATH to fix it.
>>
>> Thanks a lot, setuptools. So am I cooked?
>
> Mmh, that's weird: in the system we were, the usercustomize did happen
> after the setuptools nonsense, so we were able to work around that.
>
> Setuptools is simply an abomination, period.  It breaks fundamental
> constructs of how python is supposed to work and makes certain things
> that are defined in the design of Python impossible.  It's a shame
> that it's so widely used, and this is why in ipython we have a strict
> policy of avoiding its use as much as possible.
>
> I can only suggest one of two ugly solutions:
>
> - patch your startup scripts for any tool you want
> - patch the python call itself: put ~/.local/bin/python to be your own
> python script that fixes sys.path for you, and then executes whatever
> else was in sys.argv.

- install anything into a virtualenv instead of some ~/.local/ stuff.

I never had a problem with this setup so far, unless you count the
circumvention of the sysadmin a problem...
I think it's easier to install anything I need in userspace _again_
and you can be sure, that it's consistent and working, but a small
overhead is added of course.

Greetings,
    Tom


From jason-sage at creativetrax.com  Tue Sep 25 08:43:18 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 25 Sep 2012 07:43:18 -0500
Subject: [IPython-dev] splitter not transforming a line
In-Reply-To: <CAOvn4qi2J0ttOFK_+ci=v75j2Z2GYmDqgyBOF7mw_X9DbvuH+w@mail.gmail.com>
References: <5061A21A.5020701@creativetrax.com>
	<CAOvn4qi2J0ttOFK_+ci=v75j2Z2GYmDqgyBOF7mw_X9DbvuH+w@mail.gmail.com>
Message-ID: <5061A6E6.7000205@creativetrax.com>

On 9/25/12 7:35 AM, Thomas Kluyver wrote:
> On 25 September 2012 13:22, Jason Grout <jason-sage at creativetrax.com> wrote:
>> Is this intentional?  There seem to be enough comments like
>> https://github.com/ipython/ipython/blob/master/IPython/core/inputsplitter.py#L883
>> that indicate that maybe this is an oversight in a tricky part of the code.
>
> It is quite possibly an oversight - all of that part of the code is
> complex. Sorry, my work on IPEP2 has been sidelined while I've been
> pushing Pylab forwards.
>
>> Since it is not complete and the buffer has the @interact line in it and
>> the @interact line does not end with ':' or ',', the transform is never
>> run.  This is a big problem for Sage, since Sage wants to transform that
>> 1 to Integer(1) in the preparser.
>
> Once it's in place, I would encourage you to do this with the new AST
> transformation hook:
> https://github.com/ipython/ipython/pull/2301
>
> This was inspired by Sympy wanting to do exactly the same
> transformation. I still need to do a bit of work, but I think I can
> see how to complete it.

Do you think the AST hook will make it into the next version, or should 
I submit a quick fix for this issue (i.e., check to see if buf[-1] 
starts with '@' as well as ends with ':' or ',').

We'll still have to do a string transformation since we introduce new 
syntax as well (e.g., [1..4] means [1,2,3,4]).  But many of our 
transforms will work more cleanly on an AST, so thanks for your work on 
this!

Jason



From takowl at gmail.com  Tue Sep 25 09:01:51 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 25 Sep 2012 14:01:51 +0100
Subject: [IPython-dev] splitter not transforming a line
In-Reply-To: <5061A6E6.7000205@creativetrax.com>
References: <5061A21A.5020701@creativetrax.com>
	<CAOvn4qi2J0ttOFK_+ci=v75j2Z2GYmDqgyBOF7mw_X9DbvuH+w@mail.gmail.com>
	<5061A6E6.7000205@creativetrax.com>
Message-ID: <CAOvn4qiCesauJferpqx+38cJEbZ3zyog51zEvLv7C7-s8xkJTg@mail.gmail.com>

On 25 September 2012 13:43, Jason Grout <jason-sage at creativetrax.com> wrote:
> Do you think the AST hook will make it into the next version, or should
> I submit a quick fix for this issue (i.e., check to see if buf[-1]
> starts with '@' as well as ends with ':' or ',').

It should make it into 0.14, but if the 'next version' you're thinking
of is 0.13.1, I don't think it will be backported - it is a new
feature, after all.

> We'll still have to do a string transformation since we introduce new
> syntax as well (e.g., [1..4] means [1,2,3,4]).  But many of our
> transforms will work more cleanly on an AST, so thanks for your work on
> this!

Yep, AST transformations are only possible if the syntax is already
valid Python, so the revised input transformation system will
definitely support string transformations. But where it's possible, an
AST transformation should be easier to write and more reliable.

Thomas


From jason-sage at creativetrax.com  Tue Sep 25 09:28:41 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 25 Sep 2012 08:28:41 -0500
Subject: [IPython-dev] splitter not transforming a line
In-Reply-To: <CAOvn4qiCesauJferpqx+38cJEbZ3zyog51zEvLv7C7-s8xkJTg@mail.gmail.com>
References: <5061A21A.5020701@creativetrax.com>
	<CAOvn4qi2J0ttOFK_+ci=v75j2Z2GYmDqgyBOF7mw_X9DbvuH+w@mail.gmail.com>
	<5061A6E6.7000205@creativetrax.com>
	<CAOvn4qiCesauJferpqx+38cJEbZ3zyog51zEvLv7C7-s8xkJTg@mail.gmail.com>
Message-ID: <5061B189.60105@creativetrax.com>

On 9/25/12 8:01 AM, Thomas Kluyver wrote:
> On 25 September 2012 13:43, Jason Grout <jason-sage at creativetrax.com> wrote:
>> Do you think the AST hook will make it into the next version, or should
>> I submit a quick fix for this issue (i.e., check to see if buf[-1]
>> starts with '@' as well as ends with ':' or ',').
>
> It should make it into 0.14, but if the 'next version' you're thinking
> of is 0.13.1, I don't think it will be backported - it is a new
> feature, after all.

I'll submit a pull request for inclusion in 0.13.1, then.

https://github.com/ipython/ipython/pull/2428

Thanks,

Jason



From fperez.net at gmail.com  Tue Sep 25 14:18:00 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 25 Sep 2012 11:18:00 -0700
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
	<CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
	<CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>
Message-ID: <CAHAreOp-rv5NZW2DyU3z35vy7YNgQLj0PqrUPHL_wC1M1K+3kA@mail.gmail.com>

On Tue, Sep 25, 2012 at 5:41 AM, Thomas Spura <tomspur at fedoraproject.org> wrote:
> I never had a problem with this setup so far, unless you count the
> circumvention of the sysadmin a problem...
> I think it's easier to install anything I need in userspace _again_
> and you can be sure, that it's consistent and working, but a small
> overhead is added of course.

But is it possible with a virtualenv to benefit from system-wide
packages while overriding a few yourself, if the system-wide ones use
the underhanded setutpools behavior?

Because that's the problem that I've found to be hard to manage with
setuptools: wanting to keep the system-wide stuff but overlay a few
personally managed packages to override system ones.  When the system
ones use regular distutils, that's trivial to do.  But with setuptools
messing with sys.path at runtime, you end up with the conundrum David
is in...

Maybe virtualenvs are the solution, it's still unfortunate an entire
new system has to be devised to protect against setuptools
aggressively malicious behavior.

Cheers,

f


From d.warde.farley at gmail.com  Tue Sep 25 21:50:56 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Tue, 25 Sep 2012 21:50:56 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAHAreOp-rv5NZW2DyU3z35vy7YNgQLj0PqrUPHL_wC1M1K+3kA@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
	<CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
	<CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>
	<CAHAreOp-rv5NZW2DyU3z35vy7YNgQLj0PqrUPHL_wC1M1K+3kA@mail.gmail.com>
Message-ID: <CALc6Xo4pPdpC_1dE=Lux3P2N20MG7Dhrodv2rsFrA=mVXqy8yA@mail.gmail.com>

On Tue, Sep 25, 2012 at 2:18 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Tue, Sep 25, 2012 at 5:41 AM, Thomas Spura <tomspur at fedoraproject.org> wrote:
>> I never had a problem with this setup so far, unless you count the
>> circumvention of the sysadmin a problem...
>> I think it's easier to install anything I need in userspace _again_
>> and you can be sure, that it's consistent and working, but a small
>> overhead is added of course.
>
> But is it possible with a virtualenv to benefit from system-wide
> packages while overriding a few yourself, if the system-wide ones use
> the underhanded setutpools behavior?
>
> Because that's the problem that I've found to be hard to manage with
> setuptools: wanting to keep the system-wide stuff but overlay a few
> personally managed packages to override system ones.  When the system
> ones use regular distutils, that's trivial to do.  But with setuptools
> messing with sys.path at runtime, you end up with the conundrum David
> is in...
>
> Maybe virtualenvs are the solution, it's still unfortunate an entire
> new system has to be devised to protect against setuptools
> aggressively malicious behavior.

Indeed. It turns out, our sysadmin had run into this problem before
with other users but didn't know how to solve it. I told him to
replace easy_install with pip in his site-packages generation script,
and he did so with his install of ipython (planning to fix other
packages when he updates them or when people wish to override them).
Problem solved. Down with setuptools!

David

David


From erik.m.bray at gmail.com  Thu Sep 27 08:59:05 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 27 Sep 2012 08:59:05 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CALc6Xo4pPdpC_1dE=Lux3P2N20MG7Dhrodv2rsFrA=mVXqy8yA@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
	<CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
	<CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>
	<CAHAreOp-rv5NZW2DyU3z35vy7YNgQLj0PqrUPHL_wC1M1K+3kA@mail.gmail.com>
	<CALc6Xo4pPdpC_1dE=Lux3P2N20MG7Dhrodv2rsFrA=mVXqy8yA@mail.gmail.com>
Message-ID: <CAOTD34YGTDA=SH_m+EOG2+op-W32z-V5a73j0D5C_QpLbTe=0Q@mail.gmail.com>

On Tue, Sep 25, 2012 at 9:50 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> On Tue, Sep 25, 2012 at 2:18 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>> On Tue, Sep 25, 2012 at 5:41 AM, Thomas Spura <tomspur at fedoraproject.org> wrote:
>>> I never had a problem with this setup so far, unless you count the
>>> circumvention of the sysadmin a problem...
>>> I think it's easier to install anything I need in userspace _again_
>>> and you can be sure, that it's consistent and working, but a small
>>> overhead is added of course.
>>
>> But is it possible with a virtualenv to benefit from system-wide
>> packages while overriding a few yourself, if the system-wide ones use
>> the underhanded setutpools behavior?
>>
>> Because that's the problem that I've found to be hard to manage with
>> setuptools: wanting to keep the system-wide stuff but overlay a few
>> personally managed packages to override system ones.  When the system
>> ones use regular distutils, that's trivial to do.  But with setuptools
>> messing with sys.path at runtime, you end up with the conundrum David
>> is in...
>>
>> Maybe virtualenvs are the solution, it's still unfortunate an entire
>> new system has to be devised to protect against setuptools
>> aggressively malicious behavior.
>
> Indeed. It turns out, our sysadmin had run into this problem before
> with other users but didn't know how to solve it. I told him to
> replace easy_install with pip in his site-packages generation script,
> and he did so with his install of ipython (planning to fix other
> packages when he updates them or when people wish to override them).
> Problem solved. Down with setuptools!

I don't really see this so much as a setuptools problem though.  Your
sysadmin shouldn't be using PYTHONPATH for this.  I've had to battle
this a bit at my own institution.  The problem with using PYTHONPATH
to add site-specific site-packages directories are many-fold.  For
one, if the user then messes with their PYTHONPATH they suddenly can't
`import numpy` anymore and come complaining that numpy broke or
something like that.  PYTHONPATH should be there as an easy way for
the user to optionally add additional paths to the front of their
sys.path.  So when it's used like this it basically takes away your
freedom to customize anything on your path at all since the contents
of PYTHONPATH come first.  Sure you can always add your stuff to the
front of PYTHONPATH too, but a lot of users get confused by that.  Or
it doesn't work with virtualenv or buildout, or other such problems.

What the sysadmin *should* be doing is adding a sitecustomize.py
somewhere on the standard sys.path (in lib/pythonX.Y/site-packages for
example) and adding any custom paths there, while leaving your
PYTHONPATH alone.  As mentioned elsewhere in this thread
usercustomize.py can also be used.

Don't get me wrong--setuptools can cause its share of problems. But I
really don't see this as a setuptools fault so much as a site
configuration mess.

Erik


From d.warde.farley at gmail.com  Thu Sep 27 15:12:28 2012
From: d.warde.farley at gmail.com (David Warde-Farley)
Date: Thu, 27 Sep 2012 15:12:28 -0400
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAOTD34YGTDA=SH_m+EOG2+op-W32z-V5a73j0D5C_QpLbTe=0Q@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
	<CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
	<CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>
	<CAHAreOp-rv5NZW2DyU3z35vy7YNgQLj0PqrUPHL_wC1M1K+3kA@mail.gmail.com>
	<CALc6Xo4pPdpC_1dE=Lux3P2N20MG7Dhrodv2rsFrA=mVXqy8yA@mail.gmail.com>
	<CAOTD34YGTDA=SH_m+EOG2+op-W32z-V5a73j0D5C_QpLbTe=0Q@mail.gmail.com>
Message-ID: <CALc6Xo56AWo_gm675bX5P=jDt6Qs7KCq0EW_pUm1X=y_5jZZWw@mail.gmail.com>

On Thu, Sep 27, 2012 at 8:59 AM, Erik Bray <erik.m.bray at gmail.com> wrote:

> I don't really see this so much as a setuptools problem though.  Your
> sysadmin shouldn't be using PYTHONPATH for this.  I've had to battle
> this a bit at my own institution.  The problem with using PYTHONPATH
> to add site-specific site-packages directories are many-fold.  For
> one, if the user then messes with their PYTHONPATH they suddenly can't
> `import numpy` anymore and come complaining that numpy broke or
> something like that.  PYTHONPATH should be there as an easy way for
> the user to optionally add additional paths to the front of their
> sys.path.  So when it's used like this it basically takes away your
> freedom to customize anything on your path at all since the contents
> of PYTHONPATH come first.  Sure you can always add your stuff to the
> front of PYTHONPATH too, but a lot of users get confused by that.

Prepending to an environment variable to override things is a pretty
standard UNIXy thing to do. Same thing appears in Java with CLASSPATH,
etc., and it pretty much mirrors PATH in that respect. I guess it
could confuse novices, you're right.

> What the sysadmin *should* be doing is adding a sitecustomize.py
> somewhere on the standard sys.path (in lib/pythonX.Y/site-packages for
> example) and adding any custom paths there, while leaving your
> PYTHONPATH alone.  As mentioned elsewhere in this thread
> usercustomize.py can also be used.

Are you sure this would solve the issue? If all of the stuff in all of
the site-packages directories is processed before PYTHONPATH is even
touched, then I guess it might.

David


From fperez.net at gmail.com  Thu Sep 27 15:16:06 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 27 Sep 2012 12:16:06 -0700
Subject: [IPython-dev] Weird PYTHONPATH, etc. issue
In-Reply-To: <CAOTD34YGTDA=SH_m+EOG2+op-W32z-V5a73j0D5C_QpLbTe=0Q@mail.gmail.com>
References: <CALc6Xo7a4Eu8mBe-_Jrgm0GPs9oFXEYAp=jbc5qTdYGGve+ucA@mail.gmail.com>
	<CAOvn4qh+wVD6fZnwBU6xGPE4xY070Gbn5bainrSWm9==M+_XAQ@mail.gmail.com>
	<CALc6Xo4f1wtPOVg5gcmh9ouW91DRegfEpqidTcikZ-MJOoW-JQ@mail.gmail.com>
	<CAHAreOpbDauDfoRtgknNT_0A8JuDwZYRuXoCgrav+sbfYRzNRQ@mail.gmail.com>
	<CALc6Xo56XLGmKkxaGByM_YGcvHsz2JZ8+i=X7p6zjFjQJ=LrMA@mail.gmail.com>
	<CALc6Xo4416ZWS_BsC6Rn0MJF-M30vUk1POq6CdcOecfG=qHiDg@mail.gmail.com>
	<CALc6Xo60S3WcRRLjgA=CNJC1c2+ycp1B1X05g_ey8AU_=dHa3g@mail.gmail.com>
	<CAHAreOqHfsyxYb2eYutihcfRKx_26jjzbZPAsDepGeoe4dJNWg@mail.gmail.com>
	<CAE4GLsu4Yyp8ER2H91vzZM9UFYnq32ZW__wcBN_3hXS+8a5btg@mail.gmail.com>
	<CAHAreOp-rv5NZW2DyU3z35vy7YNgQLj0PqrUPHL_wC1M1K+3kA@mail.gmail.com>
	<CALc6Xo4pPdpC_1dE=Lux3P2N20MG7Dhrodv2rsFrA=mVXqy8yA@mail.gmail.com>
	<CAOTD34YGTDA=SH_m+EOG2+op-W32z-V5a73j0D5C_QpLbTe=0Q@mail.gmail.com>
Message-ID: <CAHAreOrAyPFUug8d0JweSt26FZN-qz+_p9yF25Qww2y1A+Az6Q@mail.gmail.com>

On Thu, Sep 27, 2012 at 5:59 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
> I don't really see this so much as a setuptools problem though.  Your
> sysadmin shouldn't be using PYTHONPATH for this.

I think there was a misunderstanding; in the case I referred to, the
admin didn't touch pythonpath at all.  There was simply a system-wide
install of EPD, and that (because EPD uses setuptools) rewired
sys.path, making the *user* configuration of PYTHONPATH moot.
PYTHONPATH is a *user* tool, and in this case the user was trying to
use it as intended, but a system-wide setuptools-using package was
messing sys.path at runtime and overriding PYTHONPATH.

That is the behavior of setuptools which is completely unacceptable.

In our case, we were able to work around it by creating a second set
of path entries in sys.path from PYTHONPATH via the usercustomize.py
file.  Ugly as hell, but it got the job done.  I'm not sure why that
didn't work in David's case.

So again, the problem can occur even when sysadmins don't touch
PYTHONPATH at all, simply by virtue of installing any package that
uses setuptools.

Cheers,

f


From takowl at gmail.com  Fri Sep 28 13:42:01 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 28 Sep 2012 18:42:01 +0100
Subject: [IPython-dev] Plugin system deprecated
Message-ID: <CAOvn4qimKVs+aOrotPTJJfUBzBCQVQUds71FbBYNkt0eC0qEqg@mail.gmail.com>

Hi all,

We're planning to drop the plugin system (extensions inheriting from
the Plugin class), as we don't think anyone is using it. If you know
of any extensions using the plugin system, please let us know.

https://github.com/ipython/ipython/pull/2291

The extension system will still exist, and you're encouraged to use
that for new and existing extensions.

Thanks,
Thomas


From fperez.net at gmail.com  Fri Sep 28 15:28:42 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 28 Sep 2012 12:28:42 -0700
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
Message-ID: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>

https://sites.google.com/site/larchenv/

obviously lots of similarities with the notebook, and some very
interesting ideas about rich input that seem to be well implemented.
It's a java/jython tool internally.  I hadn't seen it before.

Cheers,

f


From jason-sage at creativetrax.com  Fri Sep 28 15:57:02 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 28 Sep 2012 14:57:02 -0500
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
Message-ID: <5066010E.7050905@creativetrax.com>

On 9/28/12 2:28 PM, Fernando Perez wrote:
> https://sites.google.com/site/larchenv/
>
> obviously lots of similarities with the notebook, and some very
> interesting ideas about rich input that seem to be well implemented.
> It's a java/jython tool internally.  I hadn't seen it before.

This is pretty amazing.  I've been thinking a lot about these ideas over 
the last few months/year, and particularly about the thing he showed 
about representing the code as a graphical object in the code (and 
particularly how to do it over the web).

It looks like he's done a really nice job.  I'm really impressed.

Thanks,

Jason




From fperez.net at gmail.com  Fri Sep 28 16:07:37 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 28 Sep 2012 13:07:37 -0700
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <5066010E.7050905@creativetrax.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
Message-ID: <CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>

On Fri, Sep 28, 2012 at 12:57 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> This is pretty amazing.  I've been thinking a lot about these ideas over
> the last few months/year, and particularly about the thing he showed
> about representing the code as a graphical object in the code (and
> particularly how to do it over the web).
>
> It looks like he's done a really nice job.  I'm really impressed.

Indeed, the implementation looks very smooth.  Definitely lots of
ideas to think about.


From fperez.net at gmail.com  Fri Sep 28 16:43:57 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 28 Sep 2012 13:43:57 -0700
Subject: [IPython-dev] Fwd: [SciPy-User] PyData NYC 2012 Speakers and Talks
	announced!
In-Reply-To: <CA+NHr60rvOsBNxF5v1B26YHuhoU_n=9LUmvMWjC5LUL7EFmxUQ@mail.gmail.com>
References: <CA+NHr60rvOsBNxF5v1B26YHuhoU_n=9LUmvMWjC5LUL7EFmxUQ@mail.gmail.com>
Message-ID: <CAHAreOqg_NL03XXgYAPP9k+y0o7SFdJV1fLC4FRcqXCLsVvW3w@mail.gmail.com>

Hi folks,

for pydata NYC we have not only one, but *four* IPython talks! Brian
will be delivering one focused on the notebook and one on parallelism:

http://nyc2012.pydata.org/abstracts/#ipythonnotebook
http://nyc2012.pydata.org/abstracts/#ipythonparallel

and there will be two more, one by Michael Selik on teaching:

http://nyc2012.pydata.org/abstracts/#ipythonteaching

and one by Davin Potts on using IPython for large-scale data analysis
in combination with the powerful Disco framework:

http://nyc2012.pydata.org/abstracts/#disco

Obviously Brian's presence will be fantastic to have, but I'm
particularly excited to see newcomers from outside the core group
making these presentations, as that's the sign of a healthy project
growing outside of its core team.  I look forward to seeing these
later, as I won't be able to make it in person.

Cheers,

f


---------- Forwarded message ----------
From: Peter Wang <pwang at streamitive.com>
Date: Fri, Sep 28, 2012 at 1:31 PM
Subject: [SciPy-User] PyData NYC 2012 Speakers and Talks announced!
To: scipy-user at scipy.org


Hi everyone,

The PyData NYC team and Continuum Analytics are proud to announce the
full lineup of talks and speakers for the PyData NYC 2012 event!
We're thrilled with the exciting lineup of workshops, hands-on
tutorials, and talks about real-world uses of Python for data
analysis.

http://nyc2012.pydata.org/schedule

The list of presenters and talk abstracts are also available, and are
linked from the schedule page.

For those who will be in town on Thursday evening of October 25th,
there will be a special PyData edition of Drinks & Data at Dewey's
Flatiron.  It'll be a great chance to socialize and meet with PyData
presenters and other attendees.  Register here:
http://drinks-and-data-pydata-conf-ny.eventbrite.com/

We're also proud to be part of the NYC DataWeek:
http://oreilly.com/dataweek/?cmp=tw-strata-ev-dr.  The week of October
22nd is going to be a great time to be in New York!

Lastly, we are still looking for sponsors!  If you want to get your
company recognition in front of a few hundred Python data hackers and
hardcore developers, PyData will be a premier venue to showcase your
products or recruit exceptional talent.  Please visit
http://nyc2012.pydata.org/sponsors/becoming/ to inquire about
sponsorship.  In addition to the conference sponsorship, charter
sponsorships for dinner Friday night, as well as the Sunday
Hack-a-thon event are all open.

Please help us promote the conference! Tell your friends, email your
meetup groups, and follow @PyDataConf on Twitter.  Early registration
ends in just a few weeks, so register today!

http://pydata.eventbrite.com/


See you there!

-Peter Wang
Organizer, PyData NYC 2012
_______________________________________________
SciPy-User mailing list
SciPy-User at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user


From benjaminrk at gmail.com  Fri Sep 28 16:52:30 2012
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 28 Sep 2012 13:52:30 -0700
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
	<CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
Message-ID: <CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>

On Fri, Sep 28, 2012 at 1:07 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Fri, Sep 28, 2012 at 12:57 PM, Jason Grout
> <jason-sage at creativetrax.com> wrote:
> > This is pretty amazing.  I've been thinking a lot about these ideas over
> > the last few months/year, and particularly about the thing he showed
> > about representing the code as a graphical object in the code (and
> > particularly how to do it over the web).
> >
> > It looks like he's done a really nice job.  I'm really impressed.
>
> Indeed, the implementation looks very smooth.  Definitely lots of
> ideas to think about.
>

He specifically mentions Bret Victor, who has written
extensively<http://worrydream.com/LearnableProgramming/> on
matters of learning/interacting with code,
and has some really slick ideas.


> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120928/c7bf8e2a/attachment.html>

From vanderplas at astro.washington.edu  Fri Sep 28 17:10:27 2012
From: vanderplas at astro.washington.edu (Jake Vanderplas)
Date: Fri, 28 Sep 2012 14:10:27 -0700
Subject: [IPython-dev] Fwd: [SciPy-User] PyData NYC 2012 Speakers and
 Talks announced!
In-Reply-To: <CAHAreOqg_NL03XXgYAPP9k+y0o7SFdJV1fLC4FRcqXCLsVvW3w@mail.gmail.com>
References: <CA+NHr60rvOsBNxF5v1B26YHuhoU_n=9LUmvMWjC5LUL7EFmxUQ@mail.gmail.com>
	<CAHAreOqg_NL03XXgYAPP9k+y0o7SFdJV1fLC4FRcqXCLsVvW3w@mail.gmail.com>
Message-ID: <50661243.2030309@astro.washington.edu>

Hello,
Speaking of signs of a healthy project: don't forget to mention the fact 
that many of the sessions at PyData will likely use the ipython notebook 
as a teaching tool.  I know that will be the case for the two tutorials 
I'm preparing on matplotlib and scikit-learn.

While we're on the subject, I want to say a huge thank-you to all the 
folks who have worked on the notebook project.  It's an incredible 
contribution to the scientific python community.
    Jake


On 09/28/2012 01:43 PM, Fernando Perez wrote:
> Hi folks,
>
> for pydata NYC we have not only one, but *four* IPython talks! Brian
> will be delivering one focused on the notebook and one on parallelism:
>
> http://nyc2012.pydata.org/abstracts/#ipythonnotebook
> http://nyc2012.pydata.org/abstracts/#ipythonparallel
>
> and there will be two more, one by Michael Selik on teaching:
>
> http://nyc2012.pydata.org/abstracts/#ipythonteaching
>
> and one by Davin Potts on using IPython for large-scale data analysis
> in combination with the powerful Disco framework:
>
> http://nyc2012.pydata.org/abstracts/#disco
>
> Obviously Brian's presence will be fantastic to have, but I'm
> particularly excited to see newcomers from outside the core group
> making these presentations, as that's the sign of a healthy project
> growing outside of its core team.  I look forward to seeing these
> later, as I won't be able to make it in person.
>
> Cheers,
>
> f
>
>
> ---------- Forwarded message ----------
> From: Peter Wang <pwang at streamitive.com>
> Date: Fri, Sep 28, 2012 at 1:31 PM
> Subject: [SciPy-User] PyData NYC 2012 Speakers and Talks announced!
> To: scipy-user at scipy.org
>
>
> Hi everyone,
>
> The PyData NYC team and Continuum Analytics are proud to announce the
> full lineup of talks and speakers for the PyData NYC 2012 event!
> We're thrilled with the exciting lineup of workshops, hands-on
> tutorials, and talks about real-world uses of Python for data
> analysis.
>
> http://nyc2012.pydata.org/schedule
>
> The list of presenters and talk abstracts are also available, and are
> linked from the schedule page.
>
> For those who will be in town on Thursday evening of October 25th,
> there will be a special PyData edition of Drinks & Data at Dewey's
> Flatiron.  It'll be a great chance to socialize and meet with PyData
> presenters and other attendees.  Register here:
> http://drinks-and-data-pydata-conf-ny.eventbrite.com/
>
> We're also proud to be part of the NYC DataWeek:
> http://oreilly.com/dataweek/?cmp=tw-strata-ev-dr.  The week of October
> 22nd is going to be a great time to be in New York!
>
> Lastly, we are still looking for sponsors!  If you want to get your
> company recognition in front of a few hundred Python data hackers and
> hardcore developers, PyData will be a premier venue to showcase your
> products or recruit exceptional talent.  Please visit
> http://nyc2012.pydata.org/sponsors/becoming/ to inquire about
> sponsorship.  In addition to the conference sponsorship, charter
> sponsorships for dinner Friday night, as well as the Sunday
> Hack-a-thon event are all open.
>
> Please help us promote the conference! Tell your friends, email your
> meetup groups, and follow @PyDataConf on Twitter.  Early registration
> ends in just a few weeks, so register today!
>
> http://pydata.eventbrite.com/
>
>
> See you there!
>
> -Peter Wang
> Organizer, PyData NYC 2012
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev




From ellisonbg at gmail.com  Fri Sep 28 17:26:51 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 28 Sep 2012 14:26:51 -0700
Subject: [IPython-dev] Fwd: [SciPy-User] PyData NYC 2012 Speakers and
 Talks announced!
In-Reply-To: <50661243.2030309@astro.washington.edu>
References: <CA+NHr60rvOsBNxF5v1B26YHuhoU_n=9LUmvMWjC5LUL7EFmxUQ@mail.gmail.com>
	<CAHAreOqg_NL03XXgYAPP9k+y0o7SFdJV1fLC4FRcqXCLsVvW3w@mail.gmail.com>
	<50661243.2030309@astro.washington.edu>
Message-ID: <CAH4pYpQrpMzDRgvJkFSnAG-UikFCE8SYTuOC-h-7maj4gwL2yQ@mail.gmail.com>

Jake,

> Speaking of signs of a healthy project: don't forget to mention the fact
> that many of the sessions at PyData will likely use the ipython notebook
> as a teaching tool.  I know that will be the case for the two tutorials
> I'm preparing on matplotlib and scikit-learn.

Yes, good point, for us, that is one of the biggest validation points
of all - do people actually use it.

> While we're on the subject, I want to say a huge thank-you to all the
> folks who have worked on the notebook project.  It's an incredible
> contribution to the scientific python community.

Thanks for the feedback, we are thrilled you like it!

Brian
>     Jake
>
>
> On 09/28/2012 01:43 PM, Fernando Perez wrote:
>> Hi folks,
>>
>> for pydata NYC we have not only one, but *four* IPython talks! Brian
>> will be delivering one focused on the notebook and one on parallelism:
>>
>> http://nyc2012.pydata.org/abstracts/#ipythonnotebook
>> http://nyc2012.pydata.org/abstracts/#ipythonparallel
>>
>> and there will be two more, one by Michael Selik on teaching:
>>
>> http://nyc2012.pydata.org/abstracts/#ipythonteaching
>>
>> and one by Davin Potts on using IPython for large-scale data analysis
>> in combination with the powerful Disco framework:
>>
>> http://nyc2012.pydata.org/abstracts/#disco
>>
>> Obviously Brian's presence will be fantastic to have, but I'm
>> particularly excited to see newcomers from outside the core group
>> making these presentations, as that's the sign of a healthy project
>> growing outside of its core team.  I look forward to seeing these
>> later, as I won't be able to make it in person.
>>
>> Cheers,
>>
>> f
>>
>>
>> ---------- Forwarded message ----------
>> From: Peter Wang <pwang at streamitive.com>
>> Date: Fri, Sep 28, 2012 at 1:31 PM
>> Subject: [SciPy-User] PyData NYC 2012 Speakers and Talks announced!
>> To: scipy-user at scipy.org
>>
>>
>> Hi everyone,
>>
>> The PyData NYC team and Continuum Analytics are proud to announce the
>> full lineup of talks and speakers for the PyData NYC 2012 event!
>> We're thrilled with the exciting lineup of workshops, hands-on
>> tutorials, and talks about real-world uses of Python for data
>> analysis.
>>
>> http://nyc2012.pydata.org/schedule
>>
>> The list of presenters and talk abstracts are also available, and are
>> linked from the schedule page.
>>
>> For those who will be in town on Thursday evening of October 25th,
>> there will be a special PyData edition of Drinks & Data at Dewey's
>> Flatiron.  It'll be a great chance to socialize and meet with PyData
>> presenters and other attendees.  Register here:
>> http://drinks-and-data-pydata-conf-ny.eventbrite.com/
>>
>> We're also proud to be part of the NYC DataWeek:
>> http://oreilly.com/dataweek/?cmp=tw-strata-ev-dr.  The week of October
>> 22nd is going to be a great time to be in New York!
>>
>> Lastly, we are still looking for sponsors!  If you want to get your
>> company recognition in front of a few hundred Python data hackers and
>> hardcore developers, PyData will be a premier venue to showcase your
>> products or recruit exceptional talent.  Please visit
>> http://nyc2012.pydata.org/sponsors/becoming/ to inquire about
>> sponsorship.  In addition to the conference sponsorship, charter
>> sponsorships for dinner Friday night, as well as the Sunday
>> Hack-a-thon event are all open.
>>
>> Please help us promote the conference! Tell your friends, email your
>> meetup groups, and follow @PyDataConf on Twitter.  Early registration
>> ends in just a few weeks, so register today!
>>
>> http://pydata.eventbrite.com/
>>
>>
>> See you there!
>>
>> -Peter Wang
>> Organizer, PyData NYC 2012
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Fri Sep 28 17:42:49 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 28 Sep 2012 14:42:49 -0700
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
	<CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
	<CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>
Message-ID: <CAH4pYpTsNF4OkTe13FY+qUcVz0m-PiUjxYWD4Lwrw6DSjh-ZyA@mail.gmail.com>

On Fri, Sep 28, 2012 at 1:52 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>
> On Fri, Sep 28, 2012 at 1:07 PM, Fernando Perez <fperez.net at gmail.com>
> wrote:
>>
>> On Fri, Sep 28, 2012 at 12:57 PM, Jason Grout
>> <jason-sage at creativetrax.com> wrote:
>> > This is pretty amazing.  I've been thinking a lot about these ideas over
>> > the last few months/year, and particularly about the thing he showed
>> > about representing the code as a graphical object in the code (and
>> > particularly how to do it over the web).
>> >
>> > It looks like he's done a really nice job.  I'm really impressed.
>>
>> Indeed, the implementation looks very smooth.  Definitely lots of
>> ideas to think about.
>
>
> He specifically mentions Bret Victor, who has written extensively on matters
> of learning/interacting with code,
> and has some really slick ideas.

Oh, thanks for the link to this, definitely some really cool stuff there.

Brian

>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From takowl at gmail.com  Fri Sep 28 18:36:32 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 28 Sep 2012 23:36:32 +0100
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAH4pYpTsNF4OkTe13FY+qUcVz0m-PiUjxYWD4Lwrw6DSjh-ZyA@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
	<CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
	<CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>
	<CAH4pYpTsNF4OkTe13FY+qUcVz0m-PiUjxYWD4Lwrw6DSjh-ZyA@mail.gmail.com>
Message-ID: <CAOvn4qgPVKpARgzxT7whzrw+qZ1OeRTd=Z-oxAxYte4vAX4Prg@mail.gmail.com>

On 28 September 2012 22:42, Brian Granger <ellisonbg at gmail.com> wrote:
> Oh, thanks for the link to this, definitely some really cool stuff there.

Definitely. One thing that seems particularly interesting is
interactive input widgets - see e.g. this link:
https://sites.google.com/site/larchenv/the-larch-env/embedded_polygon.png?attredirects=0

How might something like that fit into our notebook model?

Thom


From asmeurer at gmail.com  Fri Sep 28 18:49:46 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Fri, 28 Sep 2012 16:49:46 -0600
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAOvn4qgPVKpARgzxT7whzrw+qZ1OeRTd=Z-oxAxYte4vAX4Prg@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
	<CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
	<CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>
	<CAH4pYpTsNF4OkTe13FY+qUcVz0m-PiUjxYWD4Lwrw6DSjh-ZyA@mail.gmail.com>
	<CAOvn4qgPVKpARgzxT7whzrw+qZ1OeRTd=Z-oxAxYte4vAX4Prg@mail.gmail.com>
Message-ID: <CAKgW=6LHm4M6sZMytrKY4B-D+jVw=t7MkBTXU=2=OoG2NohGog@mail.gmail.com>

I guess allow arbitrary javascript to return input to the cell.

This would also be cool if you did a similar thing in QT (if that's
possible).  And maybe if you ever get a curses console, there as well.
 It would of course be a non-trivial piece of work, but this would
open up the possibility for an equation editor as input to SymPy.

Aaron Meurer

On Fri, Sep 28, 2012 at 4:36 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 28 September 2012 22:42, Brian Granger <ellisonbg at gmail.com> wrote:
>> Oh, thanks for the link to this, definitely some really cool stuff there.
>
> Definitely. One thing that seems particularly interesting is
> interactive input widgets - see e.g. this link:
> https://sites.google.com/site/larchenv/the-larch-env/embedded_polygon.png?attredirects=0
>
> How might something like that fit into our notebook model?
>
> Thom
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ellisonbg at gmail.com  Fri Sep 28 19:23:48 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 28 Sep 2012 16:23:48 -0700
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAOvn4qgPVKpARgzxT7whzrw+qZ1OeRTd=Z-oxAxYte4vAX4Prg@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
	<CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
	<CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>
	<CAH4pYpTsNF4OkTe13FY+qUcVz0m-PiUjxYWD4Lwrw6DSjh-ZyA@mail.gmail.com>
	<CAOvn4qgPVKpARgzxT7whzrw+qZ1OeRTd=Z-oxAxYte4vAX4Prg@mail.gmail.com>
Message-ID: <CAH4pYpTnwVhPnaE3QndajLKX=Q_930Zm7h+WDLityJddt9+sXg@mail.gmail.com>

Thomas,

It is already completely possible, we are just waiting for people to
write code.  Here is a rough sketch of how it would work:

1. Create object that has a JS representation that is a nice input widget.
2. Use plays with the widget.
3. As that happens, the widget JSON data to a python class/function
running in the kernel using the Kernel JS object.
4. The Python function coverts the JSON data to Python and handles accordingly.

If I had lots of spare time, I would definitely be working on this
stuff because it is not hard to code but is super high impact.

Cheers,

Brian

On Fri, Sep 28, 2012 at 3:36 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> On 28 September 2012 22:42, Brian Granger <ellisonbg at gmail.com> wrote:
>> Oh, thanks for the link to this, definitely some really cool stuff there.
>
> Definitely. One thing that seems particularly interesting is
> interactive input widgets - see e.g. this link:
> https://sites.google.com/site/larchenv/the-larch-env/embedded_polygon.png?attredirects=0
>
> How might something like that fit into our notebook model?
>
> Thom
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From bussonniermatthias at gmail.com  Sat Sep 29 04:10:56 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sat, 29 Sep 2012 10:10:56 +0200
Subject: [IPython-dev] The Larch Environment - anyone seen this before?
In-Reply-To: <CAKgW=6LHm4M6sZMytrKY4B-D+jVw=t7MkBTXU=2=OoG2NohGog@mail.gmail.com>
References: <CAHAreOpG=FpEd5gJZm3H+Sov7zLsJmxTuaPwM=ZpqaymhWdDJg@mail.gmail.com>
	<5066010E.7050905@creativetrax.com>
	<CAHAreOpaZxPdJraagkwPMCgrNOgTX7kqn85q7RYyEdwuQ6phjA@mail.gmail.com>
	<CAHNn8BWoRoB-Ns++6=tiKZ8M3KCiuWwSW-kodHV3HOaALBtmQA@mail.gmail.com>
	<CAH4pYpTsNF4OkTe13FY+qUcVz0m-PiUjxYWD4Lwrw6DSjh-ZyA@mail.gmail.com>
	<CAOvn4qgPVKpARgzxT7whzrw+qZ1OeRTd=Z-oxAxYte4vAX4Prg@mail.gmail.com>
	<CAKgW=6LHm4M6sZMytrKY4B-D+jVw=t7MkBTXU=2=OoG2NohGog@mail.gmail.com>
Message-ID: <A56864E6-2CA9-408B-A35F-734AC6FBBF23@gmail.com>


Le 29 sept. 2012 ? 00:49, Aaron Meurer a ?crit :

> I guess allow arbitrary javascript to return input to the cell.
> 
> This would also be cool if you did a similar thing in QT (if that's
> possible).  And maybe if you ever get a curses console, there as well.
> It would of course be a non-trivial piece of work, but this would
> open up the possibility for an equation editor as input to SymPy.

You'll need a QtWebkit View. Issues is JS engine of qtengine sooooooo Slow that even an empty  notebook in
a QtWebkit view is slow to scroll and take around 30s to render Math of a notebook without math...
(hopefully this should change with Qt5 which embed V8 I think).
-- 
Matthias

> 
> Aaron Meurer
> 
> On Fri, Sep 28, 2012 at 4:36 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>> On 28 September 2012 22:42, Brian Granger <ellisonbg at gmail.com> wrote:
>>> Oh, thanks for the link to this, definitely some really cool stuff there.
>> 
>> Definitely. One thing that seems particularly interesting is
>> interactive input widgets - see e.g. this link:
>> https://sites.google.com/site/larchenv/the-larch-env/embedded_polygon.png?attredirects=0
>> 
>> How might something like that fit into our notebook model?
>> 
>> Thom
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Sat Sep 29 05:22:17 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 29 Sep 2012 10:22:17 +0100
Subject: [IPython-dev] AST transformations - ready for review
Message-ID: <CAOvn4qja_L0CoMrdSO6uwzgJT+nHKkRFHi1-V+7Zjm3-gZFDLA@mail.gmail.com>

Hi all,

I think the AST transformation hook in PR #2301 is ready for review.
https://github.com/ipython/ipython/pull/2301

I've made sure that it works with %timeit, %time and macros. I've also
added a test for the Integer() wrapper that Sage and Sympy both want
to apply. The code for that is:

class IntegerWrapper(ast.NodeTransformer):
    """Wraps all integers in a call to Integer()"""
    def visit_Num(self, node):
        if isinstance(node.n, int):
            return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()),
                            args=[node], keywords=[])


Thanks,
Thomas


From takowl at gmail.com  Sat Sep 29 11:42:25 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 29 Sep 2012 16:42:25 +0100
Subject: [IPython-dev] Extension loading
Message-ID: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>

This grows out of a discussion on PR #2430
https://github.com/ipython/ipython/pull/2430

Some of our default extensions contain a global flag to prevent the
extension being loaded more than once. This is rather ugly, and fixes
us to the singleton model of one IPython shell per process. I'd like
to clear that up before more third parties copy the pattern.

One obvious way is for the extension manager to track what extensions
are loaded, and refuse to load an extension more than once. That has a
downside, though: you might want calling %load_ext more than once to
work. For instance, if the extension adds something to the namespace,
the user overwrites it, and then realises they didn't want to, calling
%load_ext again is an obvious course of action. We have a %reload_ext
magic function, but the user may not know about it.

So, I see three options:
1. The extension manager doesn't track what's loaded, and it remains
up to the extension to avoid loading twice if it wants to.*
2. The extension manager tracks what's loaded, and refuses to load
anything more than once. It's up to the user to call %reload_ext if
they need to reload something.
3. The extension manager tracks what's loaded, and calling %load_ext a
second time works like %reload_ext (optional: get rid of %reload_ext,
which has become redundant)

* Are there any cases where running the extension-loading code more
than once without reloading the module causes something to go wrong?

I think 3 would be my preference. Any thoughts?

Thanks,
Thomas


From zvoros at gmail.com  Sat Sep 29 12:03:17 2012
From: zvoros at gmail.com (=?ISO-8859-1?Q?Zolt=E1n_V=F6r=F6s?=)
Date: Sat, 29 Sep 2012 18:03:17 +0200
Subject: [IPython-dev] Extension loading
In-Reply-To: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
References: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
Message-ID: <50671BC5.7060403@gmail.com>



On 09/29/2012 05:42 PM, Thomas Kluyver wrote:
> So, I see three options:
> 1. The extension manager doesn't track what's loaded, and it remains
> up to the extension to avoid loading twice if it wants to.*
> 2. The extension manager tracks what's loaded, and refuses to load
> anything more than once. It's up to the user to call %reload_ext if
> they need to reload something.
> 3. The extension manager tracks what's loaded, and calling %load_ext a
> second time works like %reload_ext (optional: get rid of %reload_ext,
> which has become redundant)
What about adding the flag '-f' to %load_ext, which would reload the 
extension no matter what, and without the flag, if the extension is 
already loaded, the user would just get a warning, but the extension 
wouldn't be reloaded? By using the flag explicitly, accidents can be 
prevented (e.g., in the notebook, the user calls %load_ext, and nothing 
happens, because the extension is already loaded, and then the user 
moves 3 cells down, forgets about the previous call, and calls %load_ext 
again), and you wouldn't have to keep track of how many times you called 
%load_ext. I think "overloading" the magic function in the way you 
suggested could result in some unexpected/undesired behaviour.

Cheers,
Zolt?n


From jason-sage at creativetrax.com  Sat Sep 29 13:47:58 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 29 Sep 2012 12:47:58 -0500
Subject: [IPython-dev] Extension loading
In-Reply-To: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
References: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
Message-ID: <5067344E.1010405@creativetrax.com>

On 9/29/12 10:42 AM, Thomas Kluyver wrote:
> This grows out of a discussion on PR #2430
> https://github.com/ipython/ipython/pull/2430
>
> Some of our default extensions contain a global flag to prevent the
> extension being loaded more than once. This is rather ugly, and fixes
> us to the singleton model of one IPython shell per process. I'd like
> to clear that up before more third parties copy the pattern.
>
> One obvious way is for the extension manager to track what extensions
> are loaded, and refuse to load an extension more than once. That has a
> downside, though: you might want calling %load_ext more than once to
> work. For instance, if the extension adds something to the namespace,
> the user overwrites it, and then realises they didn't want to, calling
> %load_ext again is an obvious course of action. We have a %reload_ext
> magic function, but the user may not know about it.
>
> So, I see three options:
> 1. The extension manager doesn't track what's loaded, and it remains
> up to the extension to avoid loading twice if it wants to.*
> 2. The extension manager tracks what's loaded, and refuses to load
> anything more than once. It's up to the user to call %reload_ext if
> they need to reload something.
> 3. The extension manager tracks what's loaded, and calling %load_ext a
> second time works like %reload_ext (optional: get rid of %reload_ext,
> which has become redundant)
>
> * Are there any cases where running the extension-loading code more
> than once without reloading the module causes something to go wrong?

In the preliminary Sage extension, we do things like add a pre_run_code 
hook and run a startup file.  If we register a hook twice, I suppose it 
runs twice?

By the way, we use a decorator to implement the run-once functionality: 
https://gist.github.com/3804691

This decorator might be a good thing to add to the utility file if you 
still support the singleton model.

Thanks,

Jason



From python at elbonia.de  Sat Sep 29 14:39:39 2012
From: python at elbonia.de (Juergen Hasch)
Date: Sat, 29 Sep 2012 20:39:39 +0200
Subject: [IPython-dev] Hotkeys  notebook cell editing
In-Reply-To: <50671BC5.7060403@gmail.com>
References: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
	<50671BC5.7060403@gmail.com>
Message-ID: <5067406B.5050605@elbonia.de>

Hi,
when working with the HMTL notebook, I noticed that is is hard to 
quickly navigate when editing a cell, as Page up/down Pos1 and End keys 
only work inside the current cell. Jumping between cells requires the 
mouse. I find this especially annoying, because outside of code cells 
(no cell selected or a markdown cell), the keys work as is usual in the 
browser.

Am I missing something or how do you navigate when working with notebooks ?

I changed codecell.js and notebook.js and put them in my local 
profile/static/js directoy. Now Page up/down moves me one cell up or 
down and Ctrl+Pos1/End jumps to top/bottom, which I find quite pleasant 
to work with.

   Juergen

My changes can be found here:
https://github.com/juhasch/ipython/compare/cell-hotkeys




From takowl at gmail.com  Sat Sep 29 16:16:15 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 29 Sep 2012 21:16:15 +0100
Subject: [IPython-dev] Extension loading
In-Reply-To: <5067344E.1010405@creativetrax.com>
References: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
	<5067344E.1010405@creativetrax.com>
Message-ID: <CAOvn4qjFjqMvWkGEWz6Fq83av8ovtj=hrZZYCq+9=iPAOUna0Q@mail.gmail.com>

Thanks Jason, adding a hook is a good point.

On 29 September 2012 18:47, Jason Grout <jason-sage at creativetrax.com> wrote:
> By the way, we use a decorator to implement the run-once functionality:
> https://gist.github.com/3804691

Unfortunately that has the same problem as the global _loaded flag -
if wouldn't let you load the same extension in a second IPython shell
in the same process. The aim is to have a per-shell loaded flag, not a
per-process one.

Thomas


From takowl at gmail.com  Sat Sep 29 16:19:35 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 29 Sep 2012 21:19:35 +0100
Subject: [IPython-dev] Extension loading
In-Reply-To: <5067344E.1010405@creativetrax.com>
References: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
	<5067344E.1010405@creativetrax.com>
Message-ID: <CAOvn4qi_iRjzkEKvwmG1p8QRpqq7SSe-E15UNGHMru3OokRL6w@mail.gmail.com>

On 29 September 2012 18:47, Jason Grout <jason-sage at creativetrax.com> wrote:
> In the preliminary Sage extension, we do things like add a pre_run_code
> hook and run a startup file.  If we register a hook twice, I suppose it
> runs twice?

I should also add: if the extension does anything that shouldn't be
done twice, it should be reversed, if possible, by the
unload_ipython_extension function. We'll fix reload_extension up to
call unload_.. first.

Thomas


From jason-sage at creativetrax.com  Sat Sep 29 16:42:24 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 29 Sep 2012 15:42:24 -0500
Subject: [IPython-dev] Extension loading
In-Reply-To: <CAOvn4qjFjqMvWkGEWz6Fq83av8ovtj=hrZZYCq+9=iPAOUna0Q@mail.gmail.com>
References: <CAOvn4qiVjXipNNRVWqROKgsPDwwZS-eUCcCVN5vrV7C3ubXvuA@mail.gmail.com>
	<5067344E.1010405@creativetrax.com>
	<CAOvn4qjFjqMvWkGEWz6Fq83av8ovtj=hrZZYCq+9=iPAOUna0Q@mail.gmail.com>
Message-ID: <50675D30.9020704@creativetrax.com>

On 9/29/12 3:16 PM, Thomas Kluyver wrote:
> Thanks Jason, adding a hook is a good point.
>
> On 29 September 2012 18:47, Jason Grout <jason-sage at creativetrax.com> wrote:
>> By the way, we use a decorator to implement the run-once functionality:
>> https://gist.github.com/3804691
>
> Unfortunately that has the same problem as the global _loaded flag -
> if wouldn't let you load the same extension in a second IPython shell
> in the same process. The aim is to have a per-shell loaded flag, not a
> per-process one.

Yes, the decorator wasn't solving any problems.  It was just making the 
behavior a convenient decorator, rather than relying on someone getting 
the global variable logic right.

Thanks,

Jason




From asmeurer at gmail.com  Sat Sep 29 19:07:08 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sat, 29 Sep 2012 17:07:08 -0600
Subject: [IPython-dev] AST transformations - ready for review
In-Reply-To: <CAOvn4qja_L0CoMrdSO6uwzgJT+nHKkRFHi1-V+7Zjm3-gZFDLA@mail.gmail.com>
References: <CAOvn4qja_L0CoMrdSO6uwzgJT+nHKkRFHi1-V+7Zjm3-gZFDLA@mail.gmail.com>
Message-ID: <CAKgW=6K2wNisVRxZ961b0usZ04FsHJMT+WbH=8vdOGL+zNQMtA@mail.gmail.com>

I noted this on the pull request too, but in case anyone reads it
here, the code for that transformer is wrong.  It should be

class IntegerWrapper(ast.NodeTransformer):
    """Wraps all integers in a call to Integer()"""
    def visit_Num(self, node):
        if isinstance(node.n, int):
            return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()),
                            args=[node], keywords=[])
        return node


Otherwise, it will fail for expressions containing floating point numbers.

Aaron Meurer

On Sat, Sep 29, 2012 at 3:22 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Hi all,
>
> I think the AST transformation hook in PR #2301 is ready for review.
> https://github.com/ipython/ipython/pull/2301
>
> I've made sure that it works with %timeit, %time and macros. I've also
> added a test for the Integer() wrapper that Sage and Sympy both want
> to apply. The code for that is:
>
> class IntegerWrapper(ast.NodeTransformer):
>     """Wraps all integers in a call to Integer()"""
>     def visit_Num(self, node):
>         if isinstance(node.n, int):
>             return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()),
>                             args=[node], keywords=[])
>
>
> Thanks,
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev