From zachary.pincus at yale.edu Mon Apr 2 17:46:09 2012 From: zachary.pincus at yale.edu (Zachary Pincus) Date: Mon, 2 Apr 2012 17:46:09 -0400 Subject: [IPython-dev] greedy completer breaks useful completions - fixable? Message-ID: <6A0D9C2C-BC41-4F94-84FF-4860B0CEFDC7@yale.edu> Hi all, The normal ipython tab-completer is pretty useful in that one can do things like: a(b.[TAB] to get a list of b's attributes while writing out the function call. Similarly, open('./[TAB] will list the documents in the cwd. However, after enabling the "greedy completer", very useful in its own right, this behavior goes away. e.g. a(b.[TAB] gives nothing. This comes down to the difference between DELIMS and GREEDY_DELIMS in IPython/core/completer.py:183. Is there any way that the greedy completer could have a larger set of delimiters? Though, in some sense the current GREEDY_DELIMS is already too large a set, as it includes the space char, which causes the following problems: def a(b,c): return b+c a('a','b').[TAB] # works a('a', 'b').[TAB] # doesn't work -- the space breaks the completer. Is there any to make the greedy completer to be a bit smarter without it having to be a full python parser? Maybe some logic about what to do with balanced vs. unbalanced parens/braces might be appropriate? This sort of thing would fix both of the problems above. If people agree this is feasible, I could look into a patch, but I'm curious if anyone's tried/failed at this before... Thanks, Zach From benjaminrk at gmail.com Mon Apr 2 17:55:44 2012 From: benjaminrk at gmail.com (MinRK) Date: Mon, 2 Apr 2012 15:55:44 -0600 Subject: [IPython-dev] greedy completer breaks useful completions - fixable? In-Reply-To: <6A0D9C2C-BC41-4F94-84FF-4860B0CEFDC7@yale.edu> References: <6A0D9C2C-BC41-4F94-84FF-4860B0CEFDC7@yale.edu> Message-ID: On Mon, Apr 2, 2012 at 15:46, Zachary Pincus wrote: > Hi all, > > The normal ipython tab-completer is pretty useful in that one can do > things like: > a(b.[TAB] > to get a list of b's attributes while writing out the function call. > Similarly, > open('./[TAB] > will list the documents in the cwd. > > However, after enabling the "greedy completer", very useful in its own > right, this behavior goes away. e.g. > a(b.[TAB] > gives nothing. > > This comes down to the difference between DELIMS and GREEDY_DELIMS in > IPython/core/completer.py:183. > > Is there any way that the greedy completer could have a larger set of > delimiters? Though, in some sense the current GREEDY_DELIMS is already too > large a set, as it includes the space char, which causes the following > problems: > def a(b,c): return b+c > a('a','b').[TAB] # works > a('a', 'b').[TAB] # doesn't work -- the space breaks the completer. > > Is there any to make the greedy completer to be a bit smarter without it > having to be a full python parser? Maybe some logic about what to do with > balanced vs. unbalanced parens/braces might be appropriate? This sort of > thing would fix both of the problems above. > > If people agree this is feasible, I could look into a patch, but I'm > curious if anyone's tried/failed at this before... > This should absolutely be doable, and the current behavior should be considered a bug. I think greedy completion should produce a strict superset of the completions from the regular competer. The simplest way to guarantee that this happens might be to *not* replace the completer like we do, but call the original completer, then call a *different* one that only does the item-completion, and merge the results. -MinRK > > Thanks, > Zach > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Apr 3 07:47:05 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 3 Apr 2012 12:47:05 +0100 Subject: [IPython-dev] ShiningPanda update Message-ID: Some good and bad news for our continuous integration: The good news is that we can now pay for Windows tests on ShiningPanda, at $0.64 per hour. A test run on one version of Python takes about 3-6 minutes, so we're looking at about 5 cents per test, which could well be useful in the run up to a release. The not-so-good is that the free allowance for open source projects is being cut back. We now get 12.5 hours per month, which is 25 minutes per day, down from an hour. We're using approximately 25 minutes a day at present, so if we significantly expand the test suite, we'll need to keep track of it, and make some of the tests run less often. Or we can buy extra Linux build time at $0.48 per hour. http://blog.shiningpanda.com/2012/04/shiningpanda-new-plans-pro-accounts.html Thanks, Thomas From satra at mit.edu Tue Apr 3 11:09:33 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 3 Apr 2012 11:09:33 -0400 Subject: [IPython-dev] Fwd: [Nipy-devel] Travis CI automated testing In-Reply-To: References: Message-ID: hi thomas, in relation to shining panda, another CI server. cheers, satra ---------- Forwarded message ---------- From: Matthew Brett Date: Mon, Mar 26, 2012 at 9:38 PM Subject: [Nipy-devel] Travis CI automated testing To: NIPY Developer's List Hi, Just to say, I've been getting good success using Travis-CI: http://about.travis-ci.org/docs/user/getting-started/ See the .travis.yml file in nibabel: https://github.com/nipy/nibabel/blob/master/.travis.yml I turned on automated builds for my github clone of nibabel, and now it tests python 2.6 / 2.7 / 3.2 for each push I do; example here: http://travis-ci.org/#!/matthew-brett/nibabel/builds/953438 Happy hunting, Matthew _______________________________________________ Nipy-devel mailing list Nipy-devel at neuroimaging.scipy.org http://mail.scipy.org/mailman/listinfo/nipy-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Apr 3 11:55:19 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 3 Apr 2012 16:55:19 +0100 Subject: [IPython-dev] Fwd: [Nipy-devel] Travis CI automated testing In-Reply-To: References: Message-ID: On 3 April 2012 16:09, Satrajit Ghosh wrote: > in relation to shining panda, another CI server. [travis-ci] At a glance, that does look good, and they have an impressive list of sponsors. But I'd like to give it some more time, because they don't yet appear to have any business model, and I don't want to be hopping to each new offering only to move away when it has to balance the books. Shiningpanda is also working very well for us, and we've taken the time to set up advanced features like coverage metrics and structured output that lets us click through to individual failing tests. Thanks, Thomas From satra at mit.edu Tue Apr 3 23:31:51 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 3 Apr 2012 23:31:51 -0400 Subject: [IPython-dev] [Off-topic] julia In-Reply-To: References: Message-ID: came across this and looks interesting. http://julialang.org/ cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Apr 3 23:46:29 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 3 Apr 2012 20:46:29 -0700 Subject: [IPython-dev] [Off-topic] julia In-Reply-To: References: Message-ID: Actually the thing that I am most impressed by in the performance table is how fast Javascript is. It blows Python, Matlab, R and Octave away and is not far off Julia. On Tue, Apr 3, 2012 at 8:31 PM, Satrajit Ghosh wrote: > came across this and looks interesting. > > http://julialang.org/ > > cheers, > > satra > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From satra at mit.edu Tue Apr 3 23:50:54 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 3 Apr 2012 23:50:54 -0400 Subject: [IPython-dev] [Nipy-devel] [Off-topic] julia In-Reply-To: References: Message-ID: indeed and with webcl it could be even more powerful! cheers, satra On Tue, Apr 3, 2012 at 11:46 PM, Brian Granger wrote: > Actually the thing that I am most impressed by in the performance > table is how fast Javascript is. It blows Python, Matlab, R and > Octave away and is not far off Julia. > > On Tue, Apr 3, 2012 at 8:31 PM, Satrajit Ghosh wrote: > > came across this and looks interesting. > > > > http://julialang.org/ > > > > cheers, > > > > satra > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > Nipy-devel mailing list > Nipy-devel at neuroimaging.scipy.org > http://mail.scipy.org/mailman/listinfo/nipy-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Wed Apr 4 03:07:26 2012 From: asmeurer at gmail.com (Aaron Meurer) Date: Wed, 4 Apr 2012 01:07:26 -0600 Subject: [IPython-dev] [Nipy-devel] [Off-topic] julia In-Reply-To: References: Message-ID: Actually, it's not "Javascript", but "V8 Javascript". I wonder how SquirrelFish (Safari) or SpiderMonkey (Firefox) would perform. Of course, this isn't the main reason why we use Python. From the website: "Julia?s syntax is intended to be familiar to users of MATLAB?." No thank you. Aaron Meurer On Tue, Apr 3, 2012 at 9:50 PM, Satrajit Ghosh wrote: > indeed and with webcl it could be even more powerful! > > cheers, > > satra > > On Tue, Apr 3, 2012 at 11:46 PM, Brian Granger wrote: >> >> Actually the thing that I am most impressed by in the performance >> table is how fast Javascript is. ?It blows Python, Matlab, R and >> Octave away and is not far off Julia. >> >> On Tue, Apr 3, 2012 at 8:31 PM, Satrajit Ghosh wrote: >> > came across this and looks interesting. >> > >> > http://julialang.org/ >> > >> > cheers, >> > >> > satra >> > >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> Nipy-devel mailing list >> Nipy-devel at neuroimaging.scipy.org >> http://mail.scipy.org/mailman/listinfo/nipy-devel > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg at gmail.com Wed Apr 4 11:29:14 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 4 Apr 2012 08:29:14 -0700 Subject: [IPython-dev] [Nipy-devel] [Off-topic] julia In-Reply-To: References: Message-ID: On Wed, Apr 4, 2012 at 12:07 AM, Aaron Meurer wrote: > Actually, it's not "Javascript", but "V8 Javascript". ?I wonder how > SquirrelFish (Safari) or SpiderMonkey (Firefox) would perform. Right, but these days everyone has put lots of effort into optimizing their JS engines. > Of course, this isn't the main reason why we use Python. ?From the > website: "Julia?s syntax is intended to be familiar to users of > MATLAB?." ?No thank you. True. We do use Javascript though in the notebook and this gives me comfort that we are not going to outgrow its performance capabilities as we make more and more powerful interfaces. Cheers, Brian > Aaron Meurer > > On Tue, Apr 3, 2012 at 9:50 PM, Satrajit Ghosh wrote: >> indeed and with webcl it could be even more powerful! >> >> cheers, >> >> satra >> >> On Tue, Apr 3, 2012 at 11:46 PM, Brian Granger wrote: >>> >>> Actually the thing that I am most impressed by in the performance >>> table is how fast Javascript is. ?It blows Python, Matlab, R and >>> Octave away and is not far off Julia. >>> >>> On Tue, Apr 3, 2012 at 8:31 PM, Satrajit Ghosh wrote: >>> > came across this and looks interesting. >>> > >>> > http://julialang.org/ >>> > >>> > cheers, >>> > >>> > satra >>> > >>> > >>> > >>> > _______________________________________________ >>> > IPython-dev mailing list >>> > IPython-dev at scipy.org >>> > http://mail.scipy.org/mailman/listinfo/ipython-dev >>> > >>> >>> >>> >>> -- >>> Brian E. Granger >>> Cal Poly State University, San Luis Obispo >>> bgranger at calpoly.edu and ellisonbg at gmail.com >>> _______________________________________________ >>> Nipy-devel mailing list >>> Nipy-devel at neuroimaging.scipy.org >>> http://mail.scipy.org/mailman/listinfo/nipy-devel >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From damianavila at gmail.com Wed Apr 4 12:11:49 2012 From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=) Date: Wed, 04 Apr 2012 13:11:49 -0300 Subject: [IPython-dev] IPython notebook support for NINJA-IDE Message-ID: <4F7C72C5.8050803@gmail.com> Hi, guys... This is a preview of a little project I have been working around... IPython notebook support for NINJA-IDE!!! (yes, I like to embed IPython into other places...jeje!) Here you have a pic... I will post a link to a video later... and the plugin will be available soon... Regards. Dami?n. ** -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ninja-ide_&_ipython.png Type: image/png Size: 142117 bytes Desc: not available URL: From cschin at infoecho.net Wed Apr 4 12:38:59 2012 From: cschin at infoecho.net (Chen-Shan Chin) Date: Wed, 4 Apr 2012 09:38:59 -0700 Subject: [IPython-dev] output cell in html notebook with limited height and width with scroll bar Message-ID: <90715912-3E6E-49C1-B565-66CA15EF6C84@infoecho.net> Hi, All: Does it make sense to support output cell with some pre-specified sizes for the output cell? If the output is bigger than the size specified, using the scroll bar so a user can see the full results using finite space of the screen real estate. This will make navigation of the code easier if the output is long. Any thought about this? I tried to hack the CSS a little by putting overflow: auto or overflow: scroll and the max-height property for the output cell. It seems working well in Safari but not Firefox. (The 
 element in Firefox does not seem to respect the max-height property of its container.)

--Jason

From ellisonbg at gmail.com  Wed Apr  4 12:41:57 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 4 Apr 2012 09:41:57 -0700
Subject: [IPython-dev] output cell in html notebook with limited height
 and width with scroll bar
In-Reply-To: <90715912-3E6E-49C1-B565-66CA15EF6C84@infoecho.net>
References: <90715912-3E6E-49C1-B565-66CA15EF6C84@infoecho.net>
Message-ID: 

Yes, we need to think about the best way to size the output cells and
have them scroll.  Could you open a ticket for that?

On Wed, Apr 4, 2012 at 9:38 AM, Chen-Shan Chin  wrote:
> Hi, All:
>
> Does it make sense to support output cell with some pre-specified sizes for the output cell? If the output is bigger than the size specified, using the scroll bar so a user can see the full results using finite space of the screen real estate. ?This will make navigation of the code easier if the output is long. ?Any thought about this?
>
> I tried to hack the CSS a little by putting overflow: auto or overflow: scroll and the max-height property for the output cell. ?It seems working well in Safari but not Firefox. ?(The 
 element in Firefox does not seem to respect the max-height property of its container.)
>
> --Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From massimodisasha at gmail.com  Wed Apr  4 12:57:37 2012
From: massimodisasha at gmail.com (Massimo Di Stefano)
Date: Wed, 4 Apr 2012 12:57:37 -0400
Subject: [IPython-dev] output cell in html notebook with limited height
	and width with scroll bar
In-Reply-To: 
References: <90715912-3E6E-49C1-B565-66CA15EF6C84@infoecho.net>
	
Message-ID: 

I don't know if on topic,  but the pandas feature to print out the 'dataframe' in a html table
has already this capabilities, e.g.. if the output is long it add a scrollbar in the cell where the output is printed out.



Il giorno Apr 4, 2012, alle ore 12:41 PM, Brian Granger ha scritto:

> Yes, we need to think about the best way to size the output cells and
> have them scroll.  Could you open a ticket for that?
> 
> On Wed, Apr 4, 2012 at 9:38 AM, Chen-Shan Chin  wrote:
>> Hi, All:
>> 
>> Does it make sense to support output cell with some pre-specified sizes for the output cell? If the output is bigger than the size specified, using the scroll bar so a user can see the full results using finite space of the screen real estate.  This will make navigation of the code easier if the output is long.  Any thought about this?
>> 
>> I tried to hack the CSS a little by putting overflow: auto or overflow: scroll and the max-height property for the output cell.  It seems working well in Safari but not Firefox.  (The 
 element in Firefox does not seem to respect the max-height property of its container.)
>> 
>> --Jason
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> 
> -- 
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From warren.weckesser at enthought.com  Wed Apr  4 17:30:35 2012
From: warren.weckesser at enthought.com (Warren Weckesser)
Date: Wed, 4 Apr 2012 16:30:35 -0500
Subject: [IPython-dev] SciPy 2012 - The Eleventh Annual Conference on
 Scientific Computing with Python
Message-ID: 

SciPy 2012, the eleventh annual Conference on Scientific Computing with
Python, will be held July 16?21, 2012, in Austin, Texas.

At this conference, novel scientific applications and libraries related to
data acquisition, analysis, dissemination and visualization using Python
are presented. Attended by leading figures from both academia and industry,
it is an excellent opportunity to experience the cutting edge of scientific
software development.

The conference is preceded by two days of tutorials, during which community
experts provide training on several scientific Python packages.  Following
the main conference will be two days of coding sprints.

We invite you to give a talk or present a poster at SciPy 2012.

The list of topics that are appropriate for the conference includes (but is
not limited to):

   - new Python libraries for science and engineering;
   - applications of Python in solving scientific or computational problems;
   - high performance, parallel and GPU computing with Python;
   - use of Python in science education.



Specialized Tracks

Two specialized tracks run in parallel to the main conference:

   - High Performance Computing with Python
   Whether your algorithm is distributed, threaded, memory intensive or
   latency bound, Python is making headway into the problem.  We are looking
   for performance driven designs and applications in Python.  Candidates
   include the use of Python within a parallel application, new architectures,
   and ways of making traditional applications execute more efficiently.


   - Visualization
   They say a picture is worth a thousand words--we?re interested in both!
    Python provides numerous visualization tools that allow scientists to show
   off their work, and we want to know about any new tools and techniques out
   there.  Come show off your latest graphics, whether it?s an old library
   with a slick new feature, a new library out to challenge the status quo, or
   simply a beautiful result.



Domain-specific Mini-symposia

Mini-symposia on the following topics are also being organized:

   - Computational bioinformatics
   - Meteorology and climatology
   - Astronomy and astrophysics
   - Geophysics



Talks, papers and posters

We invite you to take part by submitting a talk or poster abstract.
 Instructions are on the conference website:

   
http://conference.scipy.org/scipy2012/talks.php
 
Selected talks are included as papers in the peer-reviewed conference
proceedings, to be published online.


Tutorials

Tutorials will be given July 16?17.  We invite instructors to submit
proposals for half-day tutorials on topics relevant to scientific computing
with Python.  See

  http://conference.scipy.org/scipy2012/tutorials.php
 
for information about submitting a tutorial proposal.  To encourage
tutorials of the highest quality, the instructor (or team of instructors)
is given a $1,000 stipend for each half day tutorial.


Student/Community Scholarships

We anticipate providing funding for students and for active members of the
SciPy community who otherwise might not be able to attend the conference.
 See

  http://conference.scipy.org/scipy2012/student.php
 
for scholarship application guidelines.


Be a Sponsor

The SciPy conference could not run without the generous support of the
institutions and corporations who share our enthusiasm for Python as a tool
for science.  Please consider sponsoring SciPy 2012.  For more information,
see

  http://conference.scipy.org/scipy2012/sponsor/index.php


Important dates:

   Monday, April 30: Talk abstracts and tutorial proposals due.
   Monday, May 7: Accepted tutorials announced.
   Monday, May 13: Accepted talks announced.

  Monday, June 18: Early registration ends. (Price increases after this
date.)
   Sunday, July 8: Online registration ends.

  Monday-Tuesday, July 16 - 17: Tutorials
   Wednesday-Thursday, July 18 - July 19: Conference
   Friday-Saturday, July 20 - July 21: Sprints

We look forward to seeing you all in Austin this year!

The SciPy 2012 Team
http://conference.scipy.org/scipy2012/organizers.php
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From matthewturk at gmail.com  Wed Apr  4 18:46:16 2012
From: matthewturk at gmail.com (Matthew Turk)
Date: Wed, 4 Apr 2012 18:46:16 -0400
Subject: [IPython-dev] Embedding the HTML notebook
Message-ID: 

Hi there,

After something of a protracted discussion on Fernando's Google+
stream about embedding the IPython notebook, I started digging into
the notebook manager code.  As something of an introduction, I'm
relatively new to this aspect of IPython, but I work on a project (
yt-project.org ) that currently has a web GUI with a
(execution-blocking, much less fancy, somewhat clunky) web notebook.
On top of this we've built a number of display widgets, which really
are what we spend the most time on and what we're most proud of, but
more importantly they're what we want to focus our development energy
on moving forward.

Anyway, this is a long way around of saying we're looking really hard
at ditching our current cell-REPL system and instead building in the
IPython notebook, since it's awesome.  I'm trying to come up to speed
with how the web notebook works, and I have a couple questions about
the broad feasibility.

To embed as is, it looks like (as shown with NINJA-ide) one can embed
an iframe to link to a running notebook server.  But rather than doing
something precisely like that, since we're looking at providing
additional items such as widgets (although Fernando mentioned that
interoperation with callbacks is still underway) is it possible to
extend the templates from the API?

Inside IPython/frontend/html/handlers.py, the render methods are code
with calls to specific template names, which then render out HTML
templates from a path that can be overridden with settings_override.
I *think* from reading the tornado docs that template_path has to be a
single string and not an iterable of strings, and the handlers
themselves are enumerated as classes in the NotebookWebApplication
handlers.  It was when I got this deep that I started to wonder if
perhaps there was an obvious, more straightforward way that I was
missing; if not, I'm definitely willing to do what I can to dive in
and contribute, if embedding is something of interest to the
community, too.

If we take as a given that an IPython engine is currently running,
what would the best route toward embedding a view -- that doesn't
fully duplicate the contents of static/ and templates/ -- into that
engine?  Would prepending an additional path to the base_project_url
work, so that any calls to/from the engine would require that
base_project_url?  And then, post-instantiation, appending new
handlers to the NotebookWebApplication, for the "embedding" app's
templates, which perhaps have to be rendered from strings?

Any ideas would be greatly appreciated, and again, thanks for all your
hard work on IPython.

-Matt


From klonuo at gmail.com  Wed Apr  4 19:04:21 2012
From: klonuo at gmail.com (klo uo)
Date: Thu, 5 Apr 2012 01:04:21 +0200
Subject: [IPython-dev] IPython notebook support for NINJA-IDE
In-Reply-To: <4F7C72C5.8050803@gmail.com>
References: <4F7C72C5.8050803@gmail.com>
Message-ID: 

Yeah, it's so magnificent... what's the problem with launching browser
outside your ide?

I tried your ide even when one ninja jump on this regular ipython mailing
list some time ago, but if you want audience you'll have to do more sane
work than making humor name for your ide or even jumping on ipython dev
mailing list
it's so un-ninja move

cheers


On Wed, Apr 4, 2012 at 6:11 PM, Dami?n Avila  wrote:

> **
> Hi, guys...
>
> This is a preview of a little project I have been working around...
> IPython notebook support for NINJA-IDE!!! (yes, I like to embed IPython
> into other places...jeje!)
> Here you have a pic... I will post a link to a video later... and the
> plugin will be available soon...
>
> Regards.
>
> Dami?n.
> **
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From damianavila at gmail.com  Wed Apr  4 22:34:47 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Wed, 04 Apr 2012 23:34:47 -0300
Subject: [IPython-dev] IPython notebook support for NINJA-IDE
In-Reply-To: 
References: <4F7C72C5.8050803@gmail.com>
	
Message-ID: <4F7D04C7.5040807@gmail.com>

El 04/04/12 20:04, klo uo escribi?:
> Yeah, it's so magnificent... what's the problem with launching browser 
> outside your ide?
>
> I tried your ide even when one ninja jump on this regular ipython 
> mailing list some time ago, but if you want audience you'll have to do 
> more sane work than making humor name for your ide or even jumping on 
> ipython dev mailing list
> it's so un-ninja move
>
> cheers
>
>
> On Wed, Apr 4, 2012 at 6:11 PM, Dami?n Avila  > wrote:
>
>     Hi, guys...
>
>     This is a preview of a little project I have been working around...
>     IPython notebook support for NINJA-IDE!!! (yes, I like to embed
>     IPython into other places...jeje!)
>     Here you have a pic... I will post a link to a video later... and
>     the plugin will be available soon...
>
>     Regards.
>
>     Dami?n.
>     **
>
>     _______________________________________________
>     IPython-dev mailing list
>     IPython-dev at scipy.org 
>     http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>

Dear Klo uo, take it easy... I think you have misinterpreted my 
intentions...
Some clarifications:
First, I am not a developer of NINJA-IDE, they offer the possibility to 
extend the NINJA features with plugins, so I am developing one... but I 
am not affiliated with them, I develop financial models with python, 
using IPython as my main tool.
Second, I have no problem with launching a browser outside the ide... 
but if we could develop some deeper integration between IPython and 
"any" IDE, it will be great (I know that a web-base IDE is the main 
idea, but we can explore another alternatives...). The idea of bring 
this plugin here is to start a discussion of how we could achieve this 
integration in a better way (maybe some people would be interested in 
this idea).
Third, today, in Google+, we have some informal discussion over the 
integration of IPython with other resources... a thread was opened today 
covering some of those ideas ([IPython-dev] Embedding the HTML 
notebook)... so because the problem of integration is better described 
there, i have no posted again in this thread... because the problems are 
common... so i want to avoid the overlapping...
Finally, there is no need to be sarcastic or rude, there are better ways 
to communicate with other people.

If you have to communicate me some personal thoughts, you can e-mail 
me... but if you want to discuss some issues regarding the integration 
between IPython and other platforms, we can follow here or in the other 
thread I have pointed out before.

Regards.

Dami?n.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From bussonniermatthias at gmail.com  Thu Apr  5 04:46:43 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Thu, 5 Apr 2012 10:46:43 +0200
Subject: [IPython-dev] ShiningPanda update
In-Reply-To: 
References: 
Message-ID: <350ADCEC-CFDD-4258-A9FA-175585637485@gmail.com>


Le 3 avr. 2012 ? 13:47, Thomas Kluyver a ?crit :

> Some good and bad news for our continuous integration:
> 
> The good news is that we can now pay for Windows tests on
> ShiningPanda, at $0.64 per hour. A test run on one version of Python
> takes about 3-6 minutes, so we're looking at about 5 cents per test,
> which could well be useful in the run up to a release.
> 
> The not-so-good is that the free allowance for open source projects is
> being cut back. We now get 12.5 hours per month, which is 25 minutes
> per day, down from an hour. We're using approximately 25 minutes a day
> at present, so if we significantly expand the test suite, we'll need
> to keep track of it, and make some of the tests run less often. Or we
> can buy extra Linux build time at $0.48 per hour.

Well... if it helps you I can try to run the same thing as I does on my iMac, but on Debian, i.e. run (most of) the test locally and upload the result somewhere.
That should decrease your computation time on shining panda, and should not took too much on my computation time if I don't reinstall the all virtualenv every time.

Let me know.
-- 
Matthias


> 
> http://blog.shiningpanda.com/2012/04/shiningpanda-new-plans-pro-accounts.html
> 
> Thanks,
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Thu Apr  5 05:54:52 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 5 Apr 2012 10:54:52 +0100
Subject: [IPython-dev] ShiningPanda update
In-Reply-To: <350ADCEC-CFDD-4258-A9FA-175585637485@gmail.com>
References: 
	<350ADCEC-CFDD-4258-A9FA-175585637485@gmail.com>
Message-ID: 

On 5 April 2012 09:46, Matthias BUSSONNIER  wrote:
> Well... if it helps you I can try to run the same thing as I does on my iMac, but on Debian, i.e. run (most of) the test locally and upload the result somewhere.

Thanks, Matthias. We'll stick with the current setup for now, but I'll
keep an eye on how much time we're using.

Thomas


From pivanov314 at gmail.com  Fri Apr  6 17:18:37 2012
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Fri, 6 Apr 2012 14:18:37 -0700
Subject: [IPython-dev] status of 0.12.1?
Message-ID: 

Hey everyone,

saw this branch got added to the git repo - is this for the purposes
of being updated in the new Ubuntu LTS release, as was discussed on in
an old thread?

just wanted to know what the status and purpose of this branch.

Thanks!

-- 
Paul Ivanov
314 address only used for lists, ?off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7


From benjaminrk at gmail.com  Fri Apr  6 17:39:46 2012
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 6 Apr 2012 14:39:46 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
Message-ID: 

On Fri, Apr 6, 2012 at 14:18, Paul Ivanov  wrote:

> Hey everyone,
>
> saw this branch got added to the git repo - is this for the purposes
> of being updated in the new Ubuntu LTS release, as was discussed on in
> an old thread?
>
> just wanted to know what the status and purpose of this branch.
>

Yes, that's the idea, since 0.13 wouldn't have a chance of making it.  It's
just bugfixes and a few minor enhancements since 0.12.  Were 12.04 not an
LTS, we wouldn't bother, and with the relatively near release of 0.13, it
is still of dubious value, but there are some serious issues in 0.12 that
we don't like the idea of surviving for 5 years.

I don't think it will be heavily advertised, as 0.13 should be pretty soon,
and is such a dramatic improvement.

-MinRK


>
> Thanks!
>
> --
> Paul Ivanov
> 314 address only used for lists,  off-list direct email at:
> http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jason-sage at creativetrax.com  Mon Apr  9 07:53:25 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Mon, 09 Apr 2012 06:53:25 -0500
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
Message-ID: <4F82CDB5.90703@creativetrax.com>

On 4/6/12 4:39 PM, MinRK wrote:
> I don't think it will be heavily advertised, as 0.13 should be pretty
> soon, and is such a dramatic improvement.

Do we have a rough timeline for 0.13?  In a few weeks I'll have time to 
work on moving Sage to the new IPython, and from your comments above, it 
sounds like it would be better to wait, rather than move to 0.12.

Is the git head fairly stable, or are things broken at this point (i.e., 
should we think about moving to git head?)

Thanks,

Jason



From emmanuel.bacry at polytechnique.fr  Mon Apr  9 14:43:23 2012
From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry)
Date: Mon, 9 Apr 2012 20:43:23 +0200
Subject: [IPython-dev] Fwd:  wxPython console
In-Reply-To: 
References: 
	
	<32771432.post@talk.nabble.com>
	
	<32772284.post@talk.nabble.com>
	
	
	
	
Message-ID: <8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>


I have been working a lot on the qtconsole, writing an efficient c++ library for math plots using wxwdigets.

I have one (simple) problem  : when running ipython in the qtconsole, is there a way to interrupt the execution of a command without restarting the whole kernel ??
A simple ctrl-c does that on ipython in a terminal window.
I am sure there is a good reason it is not working when running in the qt-console (locally on the same machine). I just want to understand what is going on.

Sorry for this "naive" question

Emmanuel


Le 9 nov. 2011 ? 07:47, Fernando Perez a ?crit :

> Forgot to cc the list...
> 
> ---------- Forwarded message ----------
> From: Fernando Perez 
> Date: Tue, Nov 8, 2011 at 5:26 PM
> Subject: Re: [IPython-dev] wxPython console
> To: Emmanuel Bacry 
> 
> 
> On Tue, Nov 8, 2011 at 1:37 PM, Emmanuel Bacry
>  wrote:
>> 2 processes are running right ? One with the wxmainloop and the kernel and one with the qtconsole which dialog with the kernel of the first process.
>> I don't quite understand how the wx mainloop and the kernel coexist in the same kernel. They are in the same thread right ? So when the kernel is running the main loop is not looping any more (idle events are not executed) right ? (it is not clear where this is in the code you pointed to)
>> Then how can you interrupt properly the execution of  a command run by the kernel ?
>> 
> 
> Yes, when the IPython kernel is *executing user code* the wx event
> loop is blocked.  But otherwise the kernel is just sitting there,
> waiting for events to arrive over the network.  Simply try running the
> example
> 
> ./ipkernel_wxapp.py
> 
> in the docs/examples/lib directory and play with it.  A little bit of
> interactive experimentation should clarify things.  Let us know if
> that's not enough.
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Mon Apr  9 14:54:23 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 9 Apr 2012 19:54:23 +0100
Subject: [IPython-dev] Fwd: wxPython console
In-Reply-To: <8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>
References: 
	
	<32771432.post@talk.nabble.com>
	
	<32772284.post@talk.nabble.com>
	
	
	
	
	<8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>
Message-ID: 

On 9 April 2012 19:43, Emmanuel Bacry  wrote:
> is there a way to interrupt the execution of a command without restarting the whole kernel ??
> A simple ctrl-c does that on ipython in a terminal window.

Ctrl-C should work the same way in the Qt console (at least for a
local kernel that the frontend 'owns'). I've just tested this on my
own machine. But it's possible that the integration with a GUI event
loop interferes with catching the interrupt.

Thomas


From emmanuel.bacry at polytechnique.fr  Mon Apr  9 15:29:13 2012
From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry)
Date: Mon, 9 Apr 2012 21:29:13 +0200
Subject: [IPython-dev] Fwd: wxPython console
In-Reply-To: 
References: 
	
	<32771432.post@talk.nabble.com>
	
	<32772284.post@talk.nabble.com>
	
	
	
	
	<8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>
	
Message-ID: 


No, I don't understand.

I just type

ipython qtconsole

and then in the console I execute any function that takes time and whenever I ctrl-C I have a dialog asking me whether I want to restart the kernel and i I say no, nothing happens ???

Emmanuel


> On 9 April 2012 19:43, Emmanuel Bacry  wrote:
>> is there a way to interrupt the execution of a command without restarting the whole kernel ??
>> A simple ctrl-c does that on ipython in a terminal window.
> 
> Ctrl-C should work the same way in the Qt console (at least for a
> local kernel that the frontend 'owns'). I've just tested this on my
> own machine. But it's possible that the integration with a GUI event
> loop interferes with catching the interrupt.
> 
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Mon Apr  9 15:40:40 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 9 Apr 2012 20:40:40 +0100
Subject: [IPython-dev] Fwd: wxPython console
In-Reply-To: 
References: 
	
	<32771432.post@talk.nabble.com>
	
	<32772284.post@talk.nabble.com>
	
	
	
	
	<8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>
	
	
Message-ID: 

On 9 April 2012 20:29, Emmanuel Bacry  wrote:
> and then in the console I execute any function that takes time and whenever I ctrl-C I have a dialog asking me whether I want to restart the kernel and i I say no, nothing happens ???

Can you replicate this with:

import time
time.sleep(10)
[Ctrl-C]

On my machine, that works as expected - I get a traceback from a
KeyboardInterrupt in the sleep call. If that works, then the problem
is in the functions you're calling - the interrupt must be causing
them to crash the kernel process. The dialog you mention appears when
the Qt console detects that its kernel has died.

Thomas


From ellisonbg at gmail.com  Mon Apr  9 15:54:24 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 9 Apr 2012 12:54:24 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: <4F82CDB5.90703@creativetrax.com>
References: 
	
	<4F82CDB5.90703@creativetrax.com>
Message-ID: 

Jason,

On Mon, Apr 9, 2012 at 4:53 AM, Jason Grout  wrote:
> On 4/6/12 4:39 PM, MinRK wrote:
>> I don't think it will be heavily advertised, as 0.13 should be pretty
>> soon, and is such a dramatic improvement.
>
> Do we have a rough timeline for 0.13? ?In a few weeks I'll have time to
> work on moving Sage to the new IPython, and from your comments above, it
> sounds like it would be better to wait, rather than move to 0.12.

We would like to get 0.13 out "soon" but we have a bunch of other
things on our plate.  I would say a realistic timescale for 0.13 would
be a month+.

> Is the git head fairly stable, or are things broken at this point (i.e.,
> should we think about moving to git head?)

We are trying to keep github master branch in production ready condition.

Cheers,

Brian

> Thanks,
>
> Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From emmanuel.bacry at polytechnique.fr  Mon Apr  9 15:55:39 2012
From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry)
Date: Mon, 9 Apr 2012 21:55:39 +0200
Subject: [IPython-dev] Fwd: wxPython console
In-Reply-To: 
References: 
	
	<32771432.post@talk.nabble.com>
	
	<32772284.post@talk.nabble.com>
	
	
	
	
	<8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>
	
	
	
Message-ID: 


Sorry,
You are right !
It has to do with the fact that I use pylab with 'wx' 
If I don't then it works....

More clearly : 

if in the config file of ipython I add 

 c.IPKernelApp.pylab = 'wx' 

then ctrl-C does not work properly, otherwise, it does.

Is there a way to avoid that ?

Emmanuel


> On 9 April 2012 20:29, Emmanuel Bacry  wrote:
>> and then in the console I execute any function that takes time and whenever I ctrl-C I have a dialog asking me whether I want to restart the kernel and i I say no, nothing happens ???
> 
> Can you replicate this with:
> 
> import time
> time.sleep(10)
> [Ctrl-C]
> 
> On my machine, that works as expected - I get a traceback from a
> KeyboardInterrupt in the sleep call. If that works, then the problem
> is in the functions you're calling - the interrupt must be causing
> them to crash the kernel process. The dialog you mention appears when
> the Qt console detects that its kernel has died.
> 
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From takowl at gmail.com  Mon Apr  9 16:01:21 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 9 Apr 2012 21:01:21 +0100
Subject: [IPython-dev] Fwd: wxPython console
In-Reply-To: 
References: 
	
	<32771432.post@talk.nabble.com>
	
	<32772284.post@talk.nabble.com>
	
	
	
	
	<8CC57498-4186-4940-BAEF-FD868A56FA06@polytechnique.fr>
	
	
	
	
Message-ID: 

On 9 April 2012 20:55, Emmanuel Bacry  wrote:
> if in the config file of ipython I add
>
> ?c.IPKernelApp.pylab = 'wx'
>
> then ctrl-C does not work properly, otherwise, it does.
>
> Is there a way to avoid that ?

That sounds like a bug - I know we'd been having some issues with Wx
integration. Check the issue tracker on github, and if it's not
already there, open a new issue for it.
https://github.com/ipython/ipython/issues

Thomas


From ellisonbg at gmail.com  Mon Apr  9 18:21:23 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 9 Apr 2012 15:21:23 -0700
Subject: [IPython-dev] Embedding the HTML notebook
In-Reply-To: 
References: 
Message-ID: 

Matthew,

On Wed, Apr 4, 2012 at 3:46 PM, Matthew Turk  wrote:
> Hi there,
>
> After something of a protracted discussion on Fernando's Google+
> stream about embedding the IPython notebook, I started digging into
> the notebook manager code. ?As something of an introduction, I'm
> relatively new to this aspect of IPython, but I work on a project (
> yt-project.org ) that currently has a web GUI with a
> (execution-blocking, much less fancy, somewhat clunky) web notebook.
> On top of this we've built a number of display widgets, which really
> are what we spend the most time on and what we're most proud of, but
> more importantly they're what we want to focus our development energy
> on moving forward.
>
> Anyway, this is a long way around of saying we're looking really hard
> at ditching our current cell-REPL system and instead building in the
> IPython notebook, since it's awesome. ?I'm trying to come up to speed
> with how the web notebook works, and I have a couple questions about
> the broad feasibility.
>
> To embed as is, it looks like (as shown with NINJA-ide) one can embed
> an iframe to link to a running notebook server. ?But rather than doing
> something precisely like that, since we're looking at providing
> additional items such as widgets (although Fernando mentioned that
> interoperation with callbacks is still underway) is it possible to
> extend the templates from the API?

Right now the answer is no.  We are currently using tornado as the
templating library and it doesn't support loading templates from
multiple directories.  However, we have plans to switch over the
jinja2, which supports this fine.  As part of that change we will also
be reorganizing the templates and javascript code to make it much
easier for people to customize things and reuse things as part of
their own web apps.  This is relatively high on my list of things
todo, but still might take a while to get to.  Part of the problem is
that this stuff is significant enough that it has to wait until after
the 0.13.

> Inside IPython/frontend/html/handlers.py, the render methods are code
> with calls to specific template names, which then render out HTML
> templates from a path that can be overridden with settings_override.
> I *think* from reading the tornado docs that template_path has to be a
> single string and not an iterable of strings,

Yes, and this is exactly the weakness of tornado's template renderer
and why we are moving away from it.

>and the handlers
> themselves are enumerated as classes in the NotebookWebApplication
> handlers. ?It was when I got this deep that I started to wonder if
> perhaps there was an obvious, more straightforward way that I was
> missing; if not, I'm definitely willing to do what I can to dive in
> and contribute, if embedding is something of interest to the
> community, too.
>
> If we take as a given that an IPython engine is currently running,
> what would the best route toward embedding a view -- that doesn't
> fully duplicate the contents of static/ and templates/ -- into that
> engine?

Today there is not easy way of doing this.  You would basically have
to duplicate everything.  After the refactoring, it should be much
easier.

> ?Would prepending an additional path to the base_project_url
> work, so that any calls to/from the engine would require that
> base_project_url? ?And then, post-instantiation, appending new
> handlers to the NotebookWebApplication, for the "embedding" app's
> templates, which perhaps have to be rendered from strings?
>
> Any ideas would be greatly appreciated, and again, thanks for all your
> hard work on IPython.

Unfortunately, for now I think the best answer is to wait.  We will
try our best to get to this ASAP though.

Cheers,

Brian

> -Matt
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From erik.tollerud at gmail.com  Mon Apr  9 21:39:38 2012
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Mon, 9 Apr 2012 18:39:38 -0700
Subject: [IPython-dev] excepthook-like behavior in ipython
Message-ID: 

Hello all,

I noticed some odd (although I now realize quite sensible) behavior in
IPython, and I'm curious if there's a workaround.  I have an
application log that I want to use to capture unhandled exceptions.
When not using IPython, it's straightforward to just override
sys.excepthook to first sent the exception to our logging system, and
then continue the normal excepthook behavior.  However, IPython seems
to always use only its own excepthook in interactive mode
(TerminalInteractiveShell.excepthook, at least in the standard
terminal).

So my question is: Is there any way to plug anything into the IPython
exception handling system, either with sys.excepthook-style syntax, or
some other way?  As I said, it doesn't have to actually catch the
exceptions - we just want entries placed in a log if an interactive
session hits an unhandled exception, and it could then pass normally
into the IPython formatting/pdb running system.

-- 
Erik Tollerud


From benjaminrk at gmail.com  Mon Apr  9 22:09:28 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 9 Apr 2012 19:09:28 -0700
Subject: [IPython-dev] excepthook-like behavior in ipython
In-Reply-To: 
References: 
Message-ID: 

On Mon, Apr 9, 2012 at 18:39, Erik Tollerud  wrote:

> Hello all,
>
> I noticed some odd (although I now realize quite sensible) behavior in
> IPython, and I'm curious if there's a workaround.  I have an
> application log that I want to use to capture unhandled exceptions.
> When not using IPython, it's straightforward to just override
> sys.excepthook to first sent the exception to our logging system, and
> then continue the normal excepthook behavior.  However, IPython seems
> to always use only its own excepthook in interactive mode
> (TerminalInteractiveShell.excepthook, at least in the standard
> terminal).
>
> So my question is: Is there any way to plug anything into the IPython
> exception handling system, either with sys.excepthook-style syntax, or
> some other way?  As I said, it doesn't have to actually catch the
> exceptions - we just want entries placed in a log if an interactive
> session hits an unhandled exception, and it could then pass normally
> into the IPython formatting/pdb running system.
>

You use shell.set_custom_excfor
this:

# define custom exception handler:

def custom_exc(shell, etype, evalue, tb, tb_offset=None):
    # do you own thing:
    print 'hi', etype
    # also do what IPython would have done, if you want:
    shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset)

# Tell IPython to use it for any/all Exceptions:
get_ipython().set_custom_exc((Exception,), custom_exc)


> --
> Erik Tollerud
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From matthewturk at gmail.com  Tue Apr 10 08:41:20 2012
From: matthewturk at gmail.com (Matthew Turk)
Date: Tue, 10 Apr 2012 08:41:20 -0400
Subject: [IPython-dev] Embedding the HTML notebook
In-Reply-To: 
References: 
	
Message-ID: 

Hi Brian,

Thanks for your reply.  Over the last couple days I've spent some time
investigating, and I was able to make this work with minimal changes
-- in advance of going any further, seeing as this is on your radar, I
wanted to reply with what I've found to see if it fits in with your
vision, or if I should scrap it.

On Mon, Apr 9, 2012 at 6:21 PM, Brian Granger  wrote:
> Matthew,
>
> On Wed, Apr 4, 2012 at 3:46 PM, Matthew Turk  wrote:
>> Hi there,
>>
>> After something of a protracted discussion on Fernando's Google+
>> stream about embedding the IPython notebook, I started digging into
>> the notebook manager code. ?As something of an introduction, I'm
>> relatively new to this aspect of IPython, but I work on a project (
>> yt-project.org ) that currently has a web GUI with a
>> (execution-blocking, much less fancy, somewhat clunky) web notebook.
>> On top of this we've built a number of display widgets, which really
>> are what we spend the most time on and what we're most proud of, but
>> more importantly they're what we want to focus our development energy
>> on moving forward.
>>
>> Anyway, this is a long way around of saying we're looking really hard
>> at ditching our current cell-REPL system and instead building in the
>> IPython notebook, since it's awesome. ?I'm trying to come up to speed
>> with how the web notebook works, and I have a couple questions about
>> the broad feasibility.
>>
>> To embed as is, it looks like (as shown with NINJA-ide) one can embed
>> an iframe to link to a running notebook server. ?But rather than doing
>> something precisely like that, since we're looking at providing
>> additional items such as widgets (although Fernando mentioned that
>> interoperation with callbacks is still underway) is it possible to
>> extend the templates from the API?
>
> Right now the answer is no. ?We are currently using tornado as the
> templating library and it doesn't support loading templates from
> multiple directories. ?However, we have plans to switch over the
> jinja2, which supports this fine. ?As part of that change we will also
> be reorganizing the templates and javascript code to make it much
> easier for people to customize things and reuse things as part of
> their own web apps. ?This is relatively high on my list of things
> todo, but still might take a while to get to. ?Part of the problem is
> that this stuff is significant enough that it has to wait until after
> the 0.13.

The change to Jinja2 for the multiple template directories makes sense
(although the additional C dependency, with possible pure python
fallback, is a tradeoff), although after looking into how
RequestHandler is implemented it seems that subclassing handles this
very nicely.

>
>> Inside IPython/frontend/html/handlers.py, the render methods are code
>> with calls to specific template names, which then render out HTML
>> templates from a path that can be overridden with settings_override.
>> I *think* from reading the tornado docs that template_path has to be a
>> single string and not an iterable of strings,
>
> Yes, and this is exactly the weakness of tornado's template renderer
> and why we are moving away from it.
>
>>and the handlers
>> themselves are enumerated as classes in the NotebookWebApplication
>> handlers. ?It was when I got this deep that I started to wonder if
>> perhaps there was an obvious, more straightforward way that I was
>> missing; if not, I'm definitely willing to do what I can to dive in
>> and contribute, if embedding is something of interest to the
>> community, too.
>>
>> If we take as a given that an IPython engine is currently running,
>> what would the best route toward embedding a view -- that doesn't
>> fully duplicate the contents of static/ and templates/ -- into that
>> engine?
>
> Today there is not easy way of doing this. ?You would basically have
> to duplicate everything. ?After the refactoring, it should be much
> easier.

What I was able to implement, without the additional dependency of
Jinja2, was a single-notebook embedded instance with default and
sub-default handlers, allowing for the two basically independent web
apps to be served by the same tornado instance.  However, it required
a handful of changes to the IPython source which I've put into a
(readily-destroyable) fork on github under my username MatthewTurk.  I
attempted to comply with the coding practices in the file and in the
IPython developer guide.

It accomplished this with a couple main changes to the IPython
codebase.  The main weakness remains that the HTML fragment that
specifically corresponds to the notebook components is not
customizable, but it also allows the embedding user to access the
IPython components, so setting up their own full HTML fragment should
also work.  This is accomplished through:

1) Adding EmbeddedNotebookApp and EmbeddedNotebookWebApplication
classes.  These subclass the appropriate classes in the existing
implementation.  Both expose a subset of functionality (for instance,
no notebook manager), but most importantly the
EmbeddedNotebookWebApplication accepts an additional_handlers
argument.  These additional handlers will not be prepended with the
base_project_url.
2) page.html has been modified to have {%%} blocks around every
component of the HTML.  This allows the initial and final HTML tags to
be stripped, along with body, script, etc.  embedded_notebook.html
subclasses notebook.html and overrides these tags, so that using
ExtJS's autoLoad functionality we can load it as a snippet (served by
Tornado) and insert it directly into an existing DOM.
3) To accommodate our existing widgets, I extended
IPython.Notebook.prototype with an external_handlers attribute.  Any
payloads that come through with the type "external_payload" will get
passed through to any callback that has been registered with the
content['payload_type'].

Setting up an embedded application is then just a matter of creating a
set of handlers which are fed in as additional_handlers to an instance
of EmbeddedNotebookApp.  Passing through the new notebook ID was
somewhat tricky, but disallowing access to the notebook manager and
redirecting from the base URL /notebook (not
base_project_url/notebook) to the correct notebook worked around that.

While these changes worked for me to get something going to prototype
and experiment with layout inside our app, I understand that they may
not fit into the broader vision for the HTML notebook.  If they are of
some use I would be more than happy to work on them until they are up
to the quality necessary for contribution.  In my conversations with
Fernando the interest of the IPython development community in
collaborating with other sub-communities in scientific python has been
pretty clear, and I would like to do what I can to help with this
effort.

>
>> ?Would prepending an additional path to the base_project_url
>> work, so that any calls to/from the engine would require that
>> base_project_url? ?And then, post-instantiation, appending new
>> handlers to the NotebookWebApplication, for the "embedding" app's
>> templates, which perhaps have to be rendered from strings?
>>
>> Any ideas would be greatly appreciated, and again, thanks for all your
>> hard work on IPython.
>
> Unfortunately, for now I think the best answer is to wait. ?We will
> try our best to get to this ASAP though.

Ah, I guess I missed that there's a refactor coming.  I will dig
further through the IPython-dev archives to try to find out more
information about this before we modify any of our code to comply with
IPython's current API; most of our users upgrade dependencies
relatively slowly, so perhaps it is premature for us to start looking
at interoperation with or utilization of the new 0MQ engine system and
the HTML notebook.

Again, thanks for your reply, and all of your amazing hard work on IPython.

-Matt

>
> Cheers,
>
> Brian
>
>> -Matt
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From cschin at infoecho.net  Tue Apr 10 10:47:56 2012
From: cschin at infoecho.net (Chen-Shan Chin)
Date: Tue, 10 Apr 2012 07:47:56 -0700
Subject: [IPython-dev] XTK
Message-ID: <59CF246B-D666-40DE-A871-B348A89DF3D8@infoecho.net>

see this today, https://github.com/xtk/X#readme

it looks something really useful for scientific visualization if it is integrated with ipython notebook seamlessly.

--Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ellisonbg at gmail.com  Tue Apr 10 11:49:06 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 10 Apr 2012 08:49:06 -0700
Subject: [IPython-dev] XTK
In-Reply-To: <59CF246B-D666-40DE-A871-B348A89DF3D8@infoecho.net>
References: <59CF246B-D666-40DE-A871-B348A89DF3D8@infoecho.net>
Message-ID: 

HI,

We definitely have our eye on this.  We have actually already written
a preproposal to integrate it into IPython.

Cheers,

Brian

On Tue, Apr 10, 2012 at 7:47 AM, Chen-Shan Chin  wrote:
> see this today,?https://github.com/xtk/X#readme
>
> it looks something really useful for scientific visualization if it is
> integrated with ipython notebook seamlessly.
>
> --Jason
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Apr 10 14:38:59 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 10 Apr 2012 11:38:59 -0700
Subject: [IPython-dev] Embedding the HTML notebook
In-Reply-To: 
References: 
	
	
Message-ID: 

On Tue, Apr 10, 2012 at 5:41 AM, Matthew Turk  wrote:
> Hi Brian,
>
> Thanks for your reply. ?Over the last couple days I've spent some time
> investigating, and I was able to make this work with minimal changes
> -- in advance of going any further, seeing as this is on your radar, I
> wanted to reply with what I've found to see if it fits in with your
> vision, or if I should scrap it.

OK great.

> On Mon, Apr 9, 2012 at 6:21 PM, Brian Granger  wrote:
>> Matthew,
>>
>> On Wed, Apr 4, 2012 at 3:46 PM, Matthew Turk  wrote:
>>> Hi there,
>>>
>>> After something of a protracted discussion on Fernando's Google+
>>> stream about embedding the IPython notebook, I started digging into
>>> the notebook manager code. ?As something of an introduction, I'm
>>> relatively new to this aspect of IPython, but I work on a project (
>>> yt-project.org ) that currently has a web GUI with a
>>> (execution-blocking, much less fancy, somewhat clunky) web notebook.
>>> On top of this we've built a number of display widgets, which really
>>> are what we spend the most time on and what we're most proud of, but
>>> more importantly they're what we want to focus our development energy
>>> on moving forward.
>>>
>>> Anyway, this is a long way around of saying we're looking really hard
>>> at ditching our current cell-REPL system and instead building in the
>>> IPython notebook, since it's awesome. ?I'm trying to come up to speed
>>> with how the web notebook works, and I have a couple questions about
>>> the broad feasibility.
>>>
>>> To embed as is, it looks like (as shown with NINJA-ide) one can embed
>>> an iframe to link to a running notebook server. ?But rather than doing
>>> something precisely like that, since we're looking at providing
>>> additional items such as widgets (although Fernando mentioned that
>>> interoperation with callbacks is still underway) is it possible to
>>> extend the templates from the API?
>>
>> Right now the answer is no. ?We are currently using tornado as the
>> templating library and it doesn't support loading templates from
>> multiple directories. ?However, we have plans to switch over the
>> jinja2, which supports this fine. ?As part of that change we will also
>> be reorganizing the templates and javascript code to make it much
>> easier for people to customize things and reuse things as part of
>> their own web apps. ?This is relatively high on my list of things
>> todo, but still might take a while to get to. ?Part of the problem is
>> that this stuff is significant enough that it has to wait until after
>> the 0.13.
>
> The change to Jinja2 for the multiple template directories makes sense
> (although the additional C dependency, with possible pure python
> fallback, is a tradeoff), although after looking into how
> RequestHandler is implemented it seems that subclassing handles this
> very nicely.

Yes, jinja2 is a non-trivial dep, but I think we need this power moving forward.

>>
>>> Inside IPython/frontend/html/handlers.py, the render methods are code
>>> with calls to specific template names, which then render out HTML
>>> templates from a path that can be overridden with settings_override.
>>> I *think* from reading the tornado docs that template_path has to be a
>>> single string and not an iterable of strings,
>>
>> Yes, and this is exactly the weakness of tornado's template renderer
>> and why we are moving away from it.
>>
>>>and the handlers
>>> themselves are enumerated as classes in the NotebookWebApplication
>>> handlers. ?It was when I got this deep that I started to wonder if
>>> perhaps there was an obvious, more straightforward way that I was
>>> missing; if not, I'm definitely willing to do what I can to dive in
>>> and contribute, if embedding is something of interest to the
>>> community, too.
>>>
>>> If we take as a given that an IPython engine is currently running,
>>> what would the best route toward embedding a view -- that doesn't
>>> fully duplicate the contents of static/ and templates/ -- into that
>>> engine?
>>
>> Today there is not easy way of doing this. ?You would basically have
>> to duplicate everything. ?After the refactoring, it should be much
>> easier.
>
> What I was able to implement, without the additional dependency of
> Jinja2, was a single-notebook embedded instance with default and
> sub-default handlers, allowing for the two basically independent web
> apps to be served by the same tornado instance. ?However, it required
> a handful of changes to the IPython source which I've put into a
> (readily-destroyable) fork on github under my username MatthewTurk. ?I
> attempted to comply with the coding practices in the file and in the
> IPython developer guide.
>
> It accomplished this with a couple main changes to the IPython
> codebase. ?The main weakness remains that the HTML fragment that
> specifically corresponds to the notebook components is not
> customizable, but it also allows the embedding user to access the
> IPython components, so setting up their own full HTML fragment should
> also work. ?This is accomplished through:
>
> 1) Adding EmbeddedNotebookApp and EmbeddedNotebookWebApplication
> classes. ?These subclass the appropriate classes in the existing
> implementation. ?Both expose a subset of functionality (for instance,
> no notebook manager), but most importantly the
> EmbeddedNotebookWebApplication accepts an additional_handlers
> argument. ?These additional handlers will not be prepended with the
> base_project_url.
> 2) page.html has been modified to have {%%} blocks around every
> component of the HTML. ?This allows the initial and final HTML tags to
> be stripped, along with body, script, etc. ?embedded_notebook.html
> subclasses notebook.html and overrides these tags, so that using
> ExtJS's autoLoad functionality we can load it as a snippet (served by
> Tornado) and insert it directly into an existing DOM.
> 3) To accommodate our existing widgets, I extended
> IPython.Notebook.prototype with an external_handlers attribute. ?Any
> payloads that come through with the type "external_payload" will get
> passed through to any callback that has been registered with the
> content['payload_type'].
>
> Setting up an embedded application is then just a matter of creating a
> set of handlers which are fed in as additional_handlers to an instance
> of EmbeddedNotebookApp. ?Passing through the new notebook ID was
> somewhat tricky, but disallowing access to the notebook manager and
> redirecting from the base URL /notebook (not
> base_project_url/notebook) to the correct notebook worked around that.

>From what I can tell, this approach sounds reasonable for your usage
case and the present condition of the IPython code base.  I don't
think these changes are appropriate to go into IPython properly,
especially given the fact that all of this code will be refactored in
a non-compatible manner.

I can say a bit more about our vision of what the code base will look
like after refactoring:

* Each page in the web application will have its own directory.
* Each web service will also have its own directory.
* Each directory will have its own template, javascript and handlers.
* Users who want to build a custom app will need to:
   - write their main program (the equivalent of notebookapp.py) that
pull in all the right handlers and templates (possibly their own
subclassed templates).
  - subclass any handlers
  - specify their own url scheme.

What I don't think is reasonable is for the main IPython notebook app
to be a super customizable meta app that covers everyones usage cases.
 I am strongly -1 on that.  Our philosophy is to give developers well
isolated components they can use to build whatever they want.  The
range of options developers will want is simply too broad to try and
cover them all from a single main program.  Even in IPython, it is
likely we will have different versions of the main program that
configure the notebook in different ways.  If we build the components
well, it will be relatively straightforward to build custom apps using
them.

> While these changes worked for me to get something going to prototype
> and experiment with layout inside our app, I understand that they may
> not fit into the broader vision for the HTML notebook. ?If they are of
> some use I would be more than happy to work on them until they are up
> to the quality necessary for contribution. ?In my conversations with
> Fernando the interest of the IPython development community in
> collaborating with other sub-communities in scientific python has been
> pretty clear, and I would like to do what I can to help with this
> effort.

We absolutely want to build collaborations with other communities.  In
this case, I think we need to just get the refactoring done and have
you try it out as we do the work so we make sure it will work well for
your usage cases.

The tricky bit is that 1) we have to wait until post 0.13 and 2) the
changes to the codebase will be so significant that all other
development on the notebook will have to be frozen.  Because the
notebook is going to be the foundation of so many things, it is really
important that we develop a solid architecture from the beginning, and
that takes time.  But, in our experience, it is well worth it in the
long run.  It just means things move more slowly at times.  Hope you
can be patient :)


>>
>>> ?Would prepending an additional path to the base_project_url
>>> work, so that any calls to/from the engine would require that
>>> base_project_url? ?And then, post-instantiation, appending new
>>> handlers to the NotebookWebApplication, for the "embedding" app's
>>> templates, which perhaps have to be rendered from strings?
>>>
>>> Any ideas would be greatly appreciated, and again, thanks for all your
>>> hard work on IPython.
>>
>> Unfortunately, for now I think the best answer is to wait. ?We will
>> try our best to get to this ASAP though.
>
> Ah, I guess I missed that there's a refactor coming. ?I will dig
> further through the IPython-dev archives to try to find out more
> information about this before we modify any of our code to comply with
> IPython's current API; most of our users upgrade dependencies
> relatively slowly, so perhaps it is premature for us to start looking
> at interoperation with or utilization of the new 0MQ engine system and
> the HTML notebook.
>
> Again, thanks for your reply, and all of your amazing hard work on IPython.

Thanks, let's keep in touch.

Cheers,

Brian

> -Matt
>
>>
>> Cheers,
>>
>> Brian
>>
>>> -Matt
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Tue Apr 10 16:02:13 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 10 Apr 2012 15:02:13 -0500
Subject: [IPython-dev] Security
Message-ID: <4F8491C5.1030705@creativetrax.com>

Hi everyone,

I just want to confirm that I'm analyzing things correctly with regards 
to security in the IPython (and zmq) infrastructure.  I've read 
http://ipython.org/ipython-doc/dev/parallel/parallel_security.html 
(which notes that is not thorough and pyzmq needs a security audit) and 
http://ipython.org/ipython-doc/dev/parallel/parallel_intro.html#security. 
My aim is to have a client on one computer and a hub and engine on 
another computer, with the messages flowing between them as securely as 
if I had the client ssh into the hub/engine computer, run python, and 
captured the stdout.  In particular, I want to prevent other users on 
either the client or the hub/engine machine from seeing messages or 
inserting their own messages, at least to the level of protection that 
ssh or filesystem permission controls provide (I trust those).  And I'm 
remembering the old adage that if I'm hand-implementing the security, 
I'm doing it wrong.

As I understand it now (am I assessing the situation correctly?), I can 
use ssh to tunnel the messages from the client computer to the engine 
computer.  From there, the messages are transmitted over tcp via the 
loopback to the hub, and then again over tcp via the loopback to the 
engine.  Well, in reality, the message may not actually go through the 
entire tcp stack since the loopback interface can be optimized on the OS 
level.  IPython has implemented some controls for rejecting messages 
that are not signed with specific authentication credentials, so if I 
securely transmit and store the credentials (using ssh and filesystem 
permissions), things *should* be okay as long as the loopback interface 
is not compromised.  However, the loopback interface still allows 
connections from anyone on the local computer to any port, and the hub 
and engines still take up ports on the server.  So if an untrusted user 
exploited a bug in the signing mechanism, or wanted to do some sort of 
DNS attack, that would be possible.

Is it possible to instead have the messages go over unix domain sockets 
(via the zmq ipc transport), which could then be protected using 
filesystem permissions?  In that scenario, the client could have an ssh 
tunnel to the hub/engine machine, then connect via IPC to the hub.  The 
unix domain socket address would be protected by necessary filesystem 
controls so that untrusted users on the hub/engine machine couldn't even 
connect to the hub.  In fact, if the client, hub, and engine were all 
running on the same computer as the same user, the socket could be set 
to have no permissions for other users, providing a secure message 
pathway.  Am I thinking of things correctly?  Using these sockets seems 
to take one potential worry out (any user being able to connect to the 
hub or engine).

Thanks,

Jason


From jrocher at enthought.com  Tue Apr 10 16:39:31 2012
From: jrocher at enthought.com (Jonathan Rocher)
Date: Tue, 10 Apr 2012 15:39:31 -0500
Subject: [IPython-dev] Embedding the HTML notebook
In-Reply-To: 
References: 
	
	
	
Message-ID: 

Hi Brian,

thanks to you and Matthew for kicking off this discussion. It is something
I am interested in as well. I have looked into the codebase as well,
thinking about customizing it, and ran into the same conclusions.

Brian, do you have any (tentative, approximate) timeline for those efforts?

Thanks to all for building these awesome tools.
Jonathan

On Tue, Apr 10, 2012 at 1:38 PM, Brian Granger  wrote:

> On Tue, Apr 10, 2012 at 5:41 AM, Matthew Turk 
> wrote:
> > Hi Brian,
> >
> > Thanks for your reply.  Over the last couple days I've spent some time
> > investigating, and I was able to make this work with minimal changes
> > -- in advance of going any further, seeing as this is on your radar, I
> > wanted to reply with what I've found to see if it fits in with your
> > vision, or if I should scrap it.
>
> OK great.
>
> > On Mon, Apr 9, 2012 at 6:21 PM, Brian Granger 
> wrote:
> >> Matthew,
> >>
> >> On Wed, Apr 4, 2012 at 3:46 PM, Matthew Turk 
> wrote:
> >>> Hi there,
> >>>
> >>> After something of a protracted discussion on Fernando's Google+
> >>> stream about embedding the IPython notebook, I started digging into
> >>> the notebook manager code.  As something of an introduction, I'm
> >>> relatively new to this aspect of IPython, but I work on a project (
> >>> yt-project.org ) that currently has a web GUI with a
> >>> (execution-blocking, much less fancy, somewhat clunky) web notebook.
> >>> On top of this we've built a number of display widgets, which really
> >>> are what we spend the most time on and what we're most proud of, but
> >>> more importantly they're what we want to focus our development energy
> >>> on moving forward.
> >>>
> >>> Anyway, this is a long way around of saying we're looking really hard
> >>> at ditching our current cell-REPL system and instead building in the
> >>> IPython notebook, since it's awesome.  I'm trying to come up to speed
> >>> with how the web notebook works, and I have a couple questions about
> >>> the broad feasibility.
> >>>
> >>> To embed as is, it looks like (as shown with NINJA-ide) one can embed
> >>> an iframe to link to a running notebook server.  But rather than doing
> >>> something precisely like that, since we're looking at providing
> >>> additional items such as widgets (although Fernando mentioned that
> >>> interoperation with callbacks is still underway) is it possible to
> >>> extend the templates from the API?
> >>
> >> Right now the answer is no.  We are currently using tornado as the
> >> templating library and it doesn't support loading templates from
> >> multiple directories.  However, we have plans to switch over the
> >> jinja2, which supports this fine.  As part of that change we will also
> >> be reorganizing the templates and javascript code to make it much
> >> easier for people to customize things and reuse things as part of
> >> their own web apps.  This is relatively high on my list of things
> >> todo, but still might take a while to get to.  Part of the problem is
> >> that this stuff is significant enough that it has to wait until after
> >> the 0.13.
> >
> > The change to Jinja2 for the multiple template directories makes sense
> > (although the additional C dependency, with possible pure python
> > fallback, is a tradeoff), although after looking into how
> > RequestHandler is implemented it seems that subclassing handles this
> > very nicely.
>
> Yes, jinja2 is a non-trivial dep, but I think we need this power moving
> forward.
>
> >>
> >>> Inside IPython/frontend/html/handlers.py, the render methods are code
> >>> with calls to specific template names, which then render out HTML
> >>> templates from a path that can be overridden with settings_override.
> >>> I *think* from reading the tornado docs that template_path has to be a
> >>> single string and not an iterable of strings,
> >>
> >> Yes, and this is exactly the weakness of tornado's template renderer
> >> and why we are moving away from it.
> >>
> >>>and the handlers
> >>> themselves are enumerated as classes in the NotebookWebApplication
> >>> handlers.  It was when I got this deep that I started to wonder if
> >>> perhaps there was an obvious, more straightforward way that I was
> >>> missing; if not, I'm definitely willing to do what I can to dive in
> >>> and contribute, if embedding is something of interest to the
> >>> community, too.
> >>>
> >>> If we take as a given that an IPython engine is currently running,
> >>> what would the best route toward embedding a view -- that doesn't
> >>> fully duplicate the contents of static/ and templates/ -- into that
> >>> engine?
> >>
> >> Today there is not easy way of doing this.  You would basically have
> >> to duplicate everything.  After the refactoring, it should be much
> >> easier.
> >
> > What I was able to implement, without the additional dependency of
> > Jinja2, was a single-notebook embedded instance with default and
> > sub-default handlers, allowing for the two basically independent web
> > apps to be served by the same tornado instance.  However, it required
> > a handful of changes to the IPython source which I've put into a
> > (readily-destroyable) fork on github under my username MatthewTurk.  I
> > attempted to comply with the coding practices in the file and in the
> > IPython developer guide.
> >
> > It accomplished this with a couple main changes to the IPython
> > codebase.  The main weakness remains that the HTML fragment that
> > specifically corresponds to the notebook components is not
> > customizable, but it also allows the embedding user to access the
> > IPython components, so setting up their own full HTML fragment should
> > also work.  This is accomplished through:
> >
> > 1) Adding EmbeddedNotebookApp and EmbeddedNotebookWebApplication
> > classes.  These subclass the appropriate classes in the existing
> > implementation.  Both expose a subset of functionality (for instance,
> > no notebook manager), but most importantly the
> > EmbeddedNotebookWebApplication accepts an additional_handlers
> > argument.  These additional handlers will not be prepended with the
> > base_project_url.
> > 2) page.html has been modified to have {%%} blocks around every
> > component of the HTML.  This allows the initial and final HTML tags to
> > be stripped, along with body, script, etc.  embedded_notebook.html
> > subclasses notebook.html and overrides these tags, so that using
> > ExtJS's autoLoad functionality we can load it as a snippet (served by
> > Tornado) and insert it directly into an existing DOM.
> > 3) To accommodate our existing widgets, I extended
> > IPython.Notebook.prototype with an external_handlers attribute.  Any
> > payloads that come through with the type "external_payload" will get
> > passed through to any callback that has been registered with the
> > content['payload_type'].
> >
> > Setting up an embedded application is then just a matter of creating a
> > set of handlers which are fed in as additional_handlers to an instance
> > of EmbeddedNotebookApp.  Passing through the new notebook ID was
> > somewhat tricky, but disallowing access to the notebook manager and
> > redirecting from the base URL /notebook (not
> > base_project_url/notebook) to the correct notebook worked around that.
>
> >From what I can tell, this approach sounds reasonable for your usage
> case and the present condition of the IPython code base.  I don't
> think these changes are appropriate to go into IPython properly,
> especially given the fact that all of this code will be refactored in
> a non-compatible manner.
>
> I can say a bit more about our vision of what the code base will look
> like after refactoring:
>
> * Each page in the web application will have its own directory.
> * Each web service will also have its own directory.
> * Each directory will have its own template, javascript and handlers.
> * Users who want to build a custom app will need to:
>   - write their main program (the equivalent of notebookapp.py) that
> pull in all the right handlers and templates (possibly their own
> subclassed templates).
>  - subclass any handlers
>  - specify their own url scheme.
>
> What I don't think is reasonable is for the main IPython notebook app
> to be a super customizable meta app that covers everyones usage cases.
>  I am strongly -1 on that.  Our philosophy is to give developers well
> isolated components they can use to build whatever they want.  The
> range of options developers will want is simply too broad to try and
> cover them all from a single main program.  Even in IPython, it is
> likely we will have different versions of the main program that
> configure the notebook in different ways.  If we build the components
> well, it will be relatively straightforward to build custom apps using
> them.
>
> > While these changes worked for me to get something going to prototype
> > and experiment with layout inside our app, I understand that they may
> > not fit into the broader vision for the HTML notebook.  If they are of
> > some use I would be more than happy to work on them until they are up
> > to the quality necessary for contribution.  In my conversations with
> > Fernando the interest of the IPython development community in
> > collaborating with other sub-communities in scientific python has been
> > pretty clear, and I would like to do what I can to help with this
> > effort.
>
> We absolutely want to build collaborations with other communities.  In
> this case, I think we need to just get the refactoring done and have
> you try it out as we do the work so we make sure it will work well for
> your usage cases.
>
> The tricky bit is that 1) we have to wait until post 0.13 and 2) the
> changes to the codebase will be so significant that all other
> development on the notebook will have to be frozen.  Because the
> notebook is going to be the foundation of so many things, it is really
> important that we develop a solid architecture from the beginning, and
> that takes time.  But, in our experience, it is well worth it in the
> long run.  It just means things move more slowly at times.  Hope you
> can be patient :)
>
>
> >>
> >>>  Would prepending an additional path to the base_project_url
> >>> work, so that any calls to/from the engine would require that
> >>> base_project_url?  And then, post-instantiation, appending new
> >>> handlers to the NotebookWebApplication, for the "embedding" app's
> >>> templates, which perhaps have to be rendered from strings?
> >>>
> >>> Any ideas would be greatly appreciated, and again, thanks for all your
> >>> hard work on IPython.
> >>
> >> Unfortunately, for now I think the best answer is to wait.  We will
> >> try our best to get to this ASAP though.
> >
> > Ah, I guess I missed that there's a refactor coming.  I will dig
> > further through the IPython-dev archives to try to find out more
> > information about this before we modify any of our code to comply with
> > IPython's current API; most of our users upgrade dependencies
> > relatively slowly, so perhaps it is premature for us to start looking
> > at interoperation with or utilization of the new 0MQ engine system and
> > the HTML notebook.
> >
> > Again, thanks for your reply, and all of your amazing hard work on
> IPython.
>
> Thanks, let's keep in touch.
>
> Cheers,
>
> Brian
>
> > -Matt
> >
> >>
> >> Cheers,
> >>
> >> Brian
> >>
> >>> -Matt
> >>> _______________________________________________
> >>> IPython-dev mailing list
> >>> IPython-dev at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
> >>
> >>
> >> --
> >> Brian E. Granger
> >> Cal Poly State University, San Luis Obispo
> >> bgranger at calpoly.edu and ellisonbg at gmail.com
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Jonathan Rocher, PhD
Scientific software developer
Enthought, Inc.
jrocher at enthought.com
1-512-536-1057
http://www.enthought.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Tue Apr 10 16:53:52 2012
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 10 Apr 2012 13:53:52 -0700
Subject: [IPython-dev] Security
In-Reply-To: <4F8491C5.1030705@creativetrax.com>
References: <4F8491C5.1030705@creativetrax.com>
Message-ID: 

On Tue, Apr 10, 2012 at 13:02, Jason Grout wrote:

> Hi everyone,
>
> I just want to confirm that I'm analyzing things correctly with regards
> to security in the IPython (and zmq) infrastructure.  I've read
> http://ipython.org/ipython-doc/dev/parallel/parallel_security.html
> (which notes that is not thorough and pyzmq needs a security audit) and
> http://ipython.org/ipython-doc/dev/parallel/parallel_intro.html#security.
> My aim is to have a client on one computer and a hub and engine on
> another computer, with the messages flowing between them as securely as
> if I had the client ssh into the hub/engine computer, run python, and
> captured the stdout.  In particular, I want to prevent other users on
> either the client or the hub/engine machine from seeing messages or
> inserting their own messages, at least to the level of protection that
> ssh or filesystem permission controls provide (I trust those).  And I'm
> remembering the old adage that if I'm hand-implementing the security,
> I'm doing it wrong.
>
> As I understand it now (am I assessing the situation correctly?), I can
> use ssh to tunnel the messages from the client computer to the engine
> computer.  From there, the messages are transmitted over tcp via the
> loopback to the hub, and then again over tcp via the loopback to the
> engine.  Well, in reality, the message may not actually go through the
> entire tcp stack since the loopback interface can be optimized on the OS
> level.  IPython has implemented some controls for rejecting messages
> that are not signed with specific authentication credentials, so if I
> securely transmit and store the credentials (using ssh and filesystem
> permissions), things *should* be okay as long as the loopback interface
> is not compromised.  However, the loopback interface still allows
> connections from anyone on the local computer to any port, and the hub
> and engines still take up ports on the server.  So if an untrusted user
> exploited a bug in the signing mechanism, or wanted to do some sort of
> DNS attack, that would be possible.
>
> Is it possible to instead have the messages go over unix domain sockets
> (via the zmq ipc transport), which could then be protected using
> filesystem permissions?  In that scenario, the client could have an ssh
> tunnel to the hub/engine machine, then connect via IPC to the hub.  The
> unix domain socket address would be protected by necessary filesystem
> controls so that untrusted users on the hub/engine machine couldn't even
> connect to the hub.  In fact, if the client, hub, and engine were all
> running on the same computer as the same user, the socket could be set
> to have no permissions for other users, providing a secure message
> pathway.  Am I thinking of things correctly?  Using these sockets seems
> to take one potential worry out (any user being able to connect to the
> hub or engine).
>


I just tried IPC on my latptop, and it definitely does work, though the
config is a bit weird, due to some assumptions that TCP is what people
actually use:

$> ipcontroller --transport=ipc --ip=$HOME/ipcluster/socket

will result in a bunch of files called `$HOME/ipcluster/socket:12345`.
 This sets up IPC communication for the cluster.

The only thing I don't know about is forwarding UDS connections from the
client to the Hub, but if you know how to do that, you might be set (some
code would need to change in the ssh forwarding utils, but that shouldn't
be much).

-MinRK


> Thanks,
>
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From ellisonbg at gmail.com  Tue Apr 10 17:45:50 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 10 Apr 2012 14:45:50 -0700
Subject: [IPython-dev] Embedding the HTML notebook
In-Reply-To: 
References: 
	
	
	
	
Message-ID: 

On Tue, Apr 10, 2012 at 1:39 PM, Jonathan Rocher  wrote:
> Hi Brian,
>
> thanks to you and Matthew for kicking off this discussion. It is something I
> am interested in as well. I have looked into the codebase as well, thinking
> about customizing it, and ran into the same conclusions.
>
> Brian, do you have any (tentative, approximate) timeline for those efforts?

We won't even start this until post 0.13, which will be at least 4-6
weeks.  It might take a month after that.  So maybe 3 months total,
being somewhat conservative.

Cheers,

Brian

> Thanks to all for building these awesome tools.
> Jonathan
>
>
> On Tue, Apr 10, 2012 at 1:38 PM, Brian Granger  wrote:
>>
>> On Tue, Apr 10, 2012 at 5:41 AM, Matthew Turk 
>> wrote:
>> > Hi Brian,
>> >
>> > Thanks for your reply. ?Over the last couple days I've spent some time
>> > investigating, and I was able to make this work with minimal changes
>> > -- in advance of going any further, seeing as this is on your radar, I
>> > wanted to reply with what I've found to see if it fits in with your
>> > vision, or if I should scrap it.
>>
>> OK great.
>>
>> > On Mon, Apr 9, 2012 at 6:21 PM, Brian Granger 
>> > wrote:
>> >> Matthew,
>> >>
>> >> On Wed, Apr 4, 2012 at 3:46 PM, Matthew Turk 
>> >> wrote:
>> >>> Hi there,
>> >>>
>> >>> After something of a protracted discussion on Fernando's Google+
>> >>> stream about embedding the IPython notebook, I started digging into
>> >>> the notebook manager code. ?As something of an introduction, I'm
>> >>> relatively new to this aspect of IPython, but I work on a project (
>> >>> yt-project.org ) that currently has a web GUI with a
>> >>> (execution-blocking, much less fancy, somewhat clunky) web notebook.
>> >>> On top of this we've built a number of display widgets, which really
>> >>> are what we spend the most time on and what we're most proud of, but
>> >>> more importantly they're what we want to focus our development energy
>> >>> on moving forward.
>> >>>
>> >>> Anyway, this is a long way around of saying we're looking really hard
>> >>> at ditching our current cell-REPL system and instead building in the
>> >>> IPython notebook, since it's awesome. ?I'm trying to come up to speed
>> >>> with how the web notebook works, and I have a couple questions about
>> >>> the broad feasibility.
>> >>>
>> >>> To embed as is, it looks like (as shown with NINJA-ide) one can embed
>> >>> an iframe to link to a running notebook server. ?But rather than doing
>> >>> something precisely like that, since we're looking at providing
>> >>> additional items such as widgets (although Fernando mentioned that
>> >>> interoperation with callbacks is still underway) is it possible to
>> >>> extend the templates from the API?
>> >>
>> >> Right now the answer is no. ?We are currently using tornado as the
>> >> templating library and it doesn't support loading templates from
>> >> multiple directories. ?However, we have plans to switch over the
>> >> jinja2, which supports this fine. ?As part of that change we will also
>> >> be reorganizing the templates and javascript code to make it much
>> >> easier for people to customize things and reuse things as part of
>> >> their own web apps. ?This is relatively high on my list of things
>> >> todo, but still might take a while to get to. ?Part of the problem is
>> >> that this stuff is significant enough that it has to wait until after
>> >> the 0.13.
>> >
>> > The change to Jinja2 for the multiple template directories makes sense
>> > (although the additional C dependency, with possible pure python
>> > fallback, is a tradeoff), although after looking into how
>> > RequestHandler is implemented it seems that subclassing handles this
>> > very nicely.
>>
>> Yes, jinja2 is a non-trivial dep, but I think we need this power moving
>> forward.
>>
>> >>
>> >>> Inside IPython/frontend/html/handlers.py, the render methods are code
>> >>> with calls to specific template names, which then render out HTML
>> >>> templates from a path that can be overridden with settings_override.
>> >>> I *think* from reading the tornado docs that template_path has to be a
>> >>> single string and not an iterable of strings,
>> >>
>> >> Yes, and this is exactly the weakness of tornado's template renderer
>> >> and why we are moving away from it.
>> >>
>> >>>and the handlers
>> >>> themselves are enumerated as classes in the NotebookWebApplication
>> >>> handlers. ?It was when I got this deep that I started to wonder if
>> >>> perhaps there was an obvious, more straightforward way that I was
>> >>> missing; if not, I'm definitely willing to do what I can to dive in
>> >>> and contribute, if embedding is something of interest to the
>> >>> community, too.
>> >>>
>> >>> If we take as a given that an IPython engine is currently running,
>> >>> what would the best route toward embedding a view -- that doesn't
>> >>> fully duplicate the contents of static/ and templates/ -- into that
>> >>> engine?
>> >>
>> >> Today there is not easy way of doing this. ?You would basically have
>> >> to duplicate everything. ?After the refactoring, it should be much
>> >> easier.
>> >
>> > What I was able to implement, without the additional dependency of
>> > Jinja2, was a single-notebook embedded instance with default and
>> > sub-default handlers, allowing for the two basically independent web
>> > apps to be served by the same tornado instance. ?However, it required
>> > a handful of changes to the IPython source which I've put into a
>> > (readily-destroyable) fork on github under my username MatthewTurk. ?I
>> > attempted to comply with the coding practices in the file and in the
>> > IPython developer guide.
>> >
>> > It accomplished this with a couple main changes to the IPython
>> > codebase. ?The main weakness remains that the HTML fragment that
>> > specifically corresponds to the notebook components is not
>> > customizable, but it also allows the embedding user to access the
>> > IPython components, so setting up their own full HTML fragment should
>> > also work. ?This is accomplished through:
>> >
>> > 1) Adding EmbeddedNotebookApp and EmbeddedNotebookWebApplication
>> > classes. ?These subclass the appropriate classes in the existing
>> > implementation. ?Both expose a subset of functionality (for instance,
>> > no notebook manager), but most importantly the
>> > EmbeddedNotebookWebApplication accepts an additional_handlers
>> > argument. ?These additional handlers will not be prepended with the
>> > base_project_url.
>> > 2) page.html has been modified to have {%%} blocks around every
>> > component of the HTML. ?This allows the initial and final HTML tags to
>> > be stripped, along with body, script, etc. ?embedded_notebook.html
>> > subclasses notebook.html and overrides these tags, so that using
>> > ExtJS's autoLoad functionality we can load it as a snippet (served by
>> > Tornado) and insert it directly into an existing DOM.
>> > 3) To accommodate our existing widgets, I extended
>> > IPython.Notebook.prototype with an external_handlers attribute. ?Any
>> > payloads that come through with the type "external_payload" will get
>> > passed through to any callback that has been registered with the
>> > content['payload_type'].
>> >
>> > Setting up an embedded application is then just a matter of creating a
>> > set of handlers which are fed in as additional_handlers to an instance
>> > of EmbeddedNotebookApp. ?Passing through the new notebook ID was
>> > somewhat tricky, but disallowing access to the notebook manager and
>> > redirecting from the base URL /notebook (not
>> > base_project_url/notebook) to the correct notebook worked around that.
>>
>> >From what I can tell, this approach sounds reasonable for your usage
>> case and the present condition of the IPython code base. ?I don't
>> think these changes are appropriate to go into IPython properly,
>> especially given the fact that all of this code will be refactored in
>> a non-compatible manner.
>>
>> I can say a bit more about our vision of what the code base will look
>> like after refactoring:
>>
>> * Each page in the web application will have its own directory.
>> * Each web service will also have its own directory.
>> * Each directory will have its own template, javascript and handlers.
>> * Users who want to build a custom app will need to:
>> ? - write their main program (the equivalent of notebookapp.py) that
>> pull in all the right handlers and templates (possibly their own
>> subclassed templates).
>> ?- subclass any handlers
>> ?- specify their own url scheme.
>>
>> What I don't think is reasonable is for the main IPython notebook app
>> to be a super customizable meta app that covers everyones usage cases.
>> ?I am strongly -1 on that. ?Our philosophy is to give developers well
>> isolated components they can use to build whatever they want. ?The
>> range of options developers will want is simply too broad to try and
>> cover them all from a single main program. ?Even in IPython, it is
>> likely we will have different versions of the main program that
>> configure the notebook in different ways. ?If we build the components
>> well, it will be relatively straightforward to build custom apps using
>> them.
>>
>> > While these changes worked for me to get something going to prototype
>> > and experiment with layout inside our app, I understand that they may
>> > not fit into the broader vision for the HTML notebook. ?If they are of
>> > some use I would be more than happy to work on them until they are up
>> > to the quality necessary for contribution. ?In my conversations with
>> > Fernando the interest of the IPython development community in
>> > collaborating with other sub-communities in scientific python has been
>> > pretty clear, and I would like to do what I can to help with this
>> > effort.
>>
>> We absolutely want to build collaborations with other communities. ?In
>> this case, I think we need to just get the refactoring done and have
>> you try it out as we do the work so we make sure it will work well for
>> your usage cases.
>>
>> The tricky bit is that 1) we have to wait until post 0.13 and 2) the
>> changes to the codebase will be so significant that all other
>> development on the notebook will have to be frozen. ?Because the
>> notebook is going to be the foundation of so many things, it is really
>> important that we develop a solid architecture from the beginning, and
>> that takes time. ?But, in our experience, it is well worth it in the
>> long run. ?It just means things move more slowly at times. ?Hope you
>> can be patient :)
>>
>>
>> >>
>> >>> ?Would prepending an additional path to the base_project_url
>> >>> work, so that any calls to/from the engine would require that
>> >>> base_project_url? ?And then, post-instantiation, appending new
>> >>> handlers to the NotebookWebApplication, for the "embedding" app's
>> >>> templates, which perhaps have to be rendered from strings?
>> >>>
>> >>> Any ideas would be greatly appreciated, and again, thanks for all your
>> >>> hard work on IPython.
>> >>
>> >> Unfortunately, for now I think the best answer is to wait. ?We will
>> >> try our best to get to this ASAP though.
>> >
>> > Ah, I guess I missed that there's a refactor coming. ?I will dig
>> > further through the IPython-dev archives to try to find out more
>> > information about this before we modify any of our code to comply with
>> > IPython's current API; most of our users upgrade dependencies
>> > relatively slowly, so perhaps it is premature for us to start looking
>> > at interoperation with or utilization of the new 0MQ engine system and
>> > the HTML notebook.
>> >
>> > Again, thanks for your reply, and all of your amazing hard work on
>> > IPython.
>>
>> Thanks, let's keep in touch.
>>
>> Cheers,
>>
>> Brian
>>
>> > -Matt
>> >
>> >>
>> >> Cheers,
>> >>
>> >> Brian
>> >>
>> >>> -Matt
>> >>> _______________________________________________
>> >>> IPython-dev mailing list
>> >>> IPython-dev at scipy.org
>> >>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >>
>> >>
>> >>
>> >> --
>> >> Brian E. Granger
>> >> Cal Poly State University, San Luis Obispo
>> >> bgranger at calpoly.edu and ellisonbg at gmail.com
>> >> _______________________________________________
>> >> IPython-dev mailing list
>> >> IPython-dev at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Jonathan Rocher, PhD
> Scientific software developer
> Enthought, Inc.
> jrocher at enthought.com
> 1-512-536-1057
> http://www.enthought.com
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From jason-sage at creativetrax.com  Wed Apr 11 08:32:46 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 11 Apr 2012 07:32:46 -0500
Subject: [IPython-dev] Security
In-Reply-To: 
References: <4F8491C5.1030705@creativetrax.com>
	
Message-ID: <4F8579EE.6030109@creativetrax.com>

On 4/10/12 3:53 PM, MinRK wrote:
> I just tried IPC on my latptop, and it definitely does work, though the
> config is a bit weird, due to some assumptions that TCP is what people
> actually use:
>
> $> ipcontroller --transport=ipc --ip=$HOME/ipcluster/socket
>
> will result in a bunch of files called `$HOME/ipcluster/socket:12345`.

Thanks; I missed the --transport option.  When you say it is a bit 
weird, are you just talking about the --ip option?  What if we make --ip 
a synonym for --address?  --address seems to be more general (though it 
still doesn't seem to roll off the tongue as well as, say, --basename 
and using filenames that were more descriptive when using ipc).

>   This sets up IPC communication for the cluster.
>
> The only thing I don't know about is forwarding UDS connections from the
> client to the Hub, but if you know how to do that, you might be set
> (some code would need to change in the ssh forwarding utils, but that
> shouldn't be much).

I was thinking that if you had forwarding set up, then you were sshing 
into the hub server and then running the client there.  Hmmm...it does 
seem like an interesting problem to tunnel a UDS connection from the 
client machine to the hub machine.  If you do tunnel, then you're 
opening up a tcp port, so the advantages of not having a port open are 
out the window, I guess.  So maybe this is just a solution where the 
client and hub are all on one computer.  Or, like in our planned case, 
we have Google App Engine messages coming in over a App Engine "channel" 
to the hub computer, then sent to the hub.

Jason


From jason-sage at creativetrax.com  Wed Apr 11 09:38:59 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 11 Apr 2012 08:38:59 -0500
Subject: [IPython-dev] Security
In-Reply-To: 
References: <4F8491C5.1030705@creativetrax.com>
	
Message-ID: <4F858973.1060406@creativetrax.com>

On 4/10/12 3:53 PM, MinRK wrote:
> The only thing I don't know about is forwarding UDS connections from the
> client to the Hub, but if you know how to do that, you might be set
> (some code would need to change in the ssh forwarding utils, but that
> shouldn't be much).

Just for further reference, poking around on the internet revealed 
several things related to this:

* forwarding can be done with the socat utility: 
http://www.debian-administration.org/users/dkg/weblog/68

* William Ahern has a patch to openSSH that allows forwarding UDS: 
http://www.25thandclement.com/~william/projects/streamlocal.html

* apparently forwarding UDS is a debian bug report: 
http://www.25thandclement.com/~william/projects/streamlocal.html

* here is another effort to modify ssh to support UDS forwarding: 
http://www.ll.mit.edu/mission/communications/ist/corpora/README.sshhop

Thanks,

Jason



From takowl at gmail.com  Wed Apr 11 10:48:41 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 11 Apr 2012 15:48:41 +0100
Subject: [IPython-dev] Where should __future__ statements affect the
	interactive environment.
Message-ID: 

Pierre Haessig has spotted inconsistencies in the effect of a
statement like 'from __future__ import division'. If it's in a startup
file with a .py extension, it's local to that file; if the same file
has a .ipy extension, it affects the interactive environment. It
occurs to me that we haven't given this proper consideration. There
are a lot of places that we execute external code:

* At startup: `exec_lines`, `exec_files`, startup files. There are .py
and .ipy files, and they should definitely behave the same way with
future statements.
* `%run` (and `%run -i`), `%rerun`, `%edit`, macros, `%loadpy`
* Extensions
* Code loaded by the autoreload extension

My gut feeling is that of those, `exec_lines`, `%loadpy`, macros and
maybe `%run -i` should 'leak' future statements to the interactive
environment. Everything else should keep future statements to itself
(except by using `run_cell()` to run them through IPython). But I'm
asking here to get other opinions.

Technical details: future statements act via Python's bytecode
compiler, not the virtual machine. We use the rather niche 'codeop'
module from the standard library, which offers a compiler that
remembers future statements from one compilation to the next. So code
that is passed to ip.compile() leaks __future__ statements, while code
passed to the builtin compile() or exec() doesn't.

This is a continuation of my discussion with Pierre on PR #950:
https://github.com/ipython/ipython/pull/950

Thomas


From mdroe at stsci.edu  Wed Apr 11 12:43:17 2012
From: mdroe at stsci.edu (Michael Droettboom)
Date: Wed, 11 Apr 2012 12:43:17 -0400
Subject: [IPython-dev] clear_output() of only last line?
Message-ID: <4F85B4A5.4030001@stsci.edu>

I'm working on getting our animated progressbars in astropy to work in 
the IPython notebook.  I found the recipe of using "clear_output()" in 
this bug report:

https://github.com/ipython/ipython/issues/1527

That gets things closer, but unlike "\r" at a standard terminal, 
clear_output() clears the entire output chunk, not just the last line.  
So if we have multiple progress bars, or progress bars interspersed with 
logging messages, all but the last progress bar gets deleted.

For example:

from IPython.core.display import clear_output

print "Working"
for i in range(80):
     clear_output()
     print "\r" + ("=" * i)

Any suggestions?

Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jason-sage at creativetrax.com  Thu Apr 12 10:22:29 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 12 Apr 2012 09:22:29 -0500
Subject: [IPython-dev] zmq when using ipython
Message-ID: <4F86E525.1090801@creativetrax.com>

When I run the plain ipython command (just 'ipython'), are there tcp zmq 
connections opened to localhost ports, or is it using something like 
inproc zmq connections?  I can't seem to find any references to open 
ports in the .config/ipython directories or any mention of open ports 
when I do ipython --debug.  I'm just trying to understand the security 
implications of using just the ipython command in a single-user situation.

Thanks,

Jason


From takowl at gmail.com  Thu Apr 12 10:29:42 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 12 Apr 2012 15:29:42 +0100
Subject: [IPython-dev] zmq when using ipython
In-Reply-To: <4F86E525.1090801@creativetrax.com>
References: <4F86E525.1090801@creativetrax.com>
Message-ID: 

On 12 April 2012 15:22, Jason Grout  wrote:
> When I run the plain ipython command (just 'ipython'), are there tcp zmq
> connections opened to localhost ports, or is it using something like
> inproc zmq connections?

Plain 'ipython' doesn't use any ZMQ connections - the terminal
frontend runs in a single process, the old fashioned way. It shouldn't
even import zmq.

Thomas


From vbraun.name at gmail.com  Thu Apr 12 10:43:38 2012
From: vbraun.name at gmail.com (Volker Braun)
Date: Thu, 12 Apr 2012 07:43:38 -0700 (PDT)
Subject: [IPython-dev] IPython+zmq and fork()
Message-ID: <1334241818735-4855890.post@n6.nabble.com>

Sage is getting ready to upgrade to the new IPython, excuses for any email
deluge on this list. One question that I have is whether you thought about
fork() to quickly spawn children (this is crucial for Sage since starting up
a new Sage process is quite slow). According to the zeromq mailinglist, you
shouldn't fork after creating a zmq context. This is not just a theoretical
problem, I wrote a clustering tool for my own purposes and found out the
hard way that bad things can happen if you do. 

Will IPython always be usable without zmq?  You are not planning to
eventually deprecate the non-zmq console client? Or maybe you have some
awesome other solution?





--
View this message in context: http://python.6.n6.nabble.com/IPython-zmq-and-fork-tp4855890p4855890.html
Sent from the IPython - Development mailing list archive at Nabble.com.


From jason-sage at creativetrax.com  Thu Apr 12 10:47:22 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 12 Apr 2012 09:47:22 -0500
Subject: [IPython-dev] zmq when using ipython
In-Reply-To: 
References: <4F86E525.1090801@creativetrax.com>
	
Message-ID: <4F86EAFA.9010103@creativetrax.com>

On 4/12/12 9:29 AM, Thomas Kluyver wrote:
> On 12 April 2012 15:22, Jason Grout  wrote:
>> When I run the plain ipython command (just 'ipython'), are there tcp zmq
>> connections opened to localhost ports, or is it using something like
>> inproc zmq connections?
>
> Plain 'ipython' doesn't use any ZMQ connections - the terminal
> frontend runs in a single process, the old fashioned way. It shouldn't
> even import zmq.

Thanks, this corrects a big misunderstanding I had (and maybe is also an 
answer to Volker's post).  I thought *everything* in the new ipython 
depended on the messaging protocol and connections with zmq.

Thanks,

Jason




From jason-sage at creativetrax.com  Thu Apr 12 11:01:25 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 12 Apr 2012 10:01:25 -0500
Subject: [IPython-dev] IPython+zmq and fork()
In-Reply-To: <1334241818735-4855890.post@n6.nabble.com>
References: <1334241818735-4855890.post@n6.nabble.com>
Message-ID: <4F86EE45.2040909@creativetrax.com>

On 4/12/12 9:43 AM, Volker Braun wrote:
> Sage is getting ready to upgrade to the new IPython, excuses for any email
> deluge on this list. One question that I have is whether you thought about
> fork() to quickly spawn children (this is crucial for Sage since starting up
> a new Sage process is quite slow). According to the zeromq mailinglist, you
> shouldn't fork after creating a zmq context. This is not just a theoretical
> problem, I wrote a clustering tool for my own purposes and found out the
> hard way that bad things can happen if you do.
>
> Will IPython always be usable without zmq?  You are not planning to
> eventually deprecate the non-zmq console client? Or maybe you have some
> awesome other solution?

Just as a point of reference, this documentation page (which seems to be 
outdated, maybe?) indicates that a single-user command line session will 
eventually use a 2-process model with zmq to communicate between the 
processes:

http://ipython.org/ipython-doc/dev/development/ipythonzmq.html

Thanks,

Jason


From benjaminrk at gmail.com  Thu Apr 12 12:42:01 2012
From: benjaminrk at gmail.com (Min RK)
Date: Thu, 12 Apr 2012 09:42:01 -0700
Subject: [IPython-dev] IPython+zmq and fork()
In-Reply-To: <4F86EE45.2040909@creativetrax.com>
References: <1334241818735-4855890.post@n6.nabble.com>
	<4F86EE45.2040909@creativetrax.com>
Message-ID: <4B84E215-41E7-4981-980E-EFCAAC7AF871@gmail.com>

zmq will never be a dependency for regular IPython.  If we do make basic IPython use the two-process model, it will need to be able to run with stdlib only (there would be no kernel sharing or anything, just frontend surviving segfaults, allowing restart, etc.)

-MinRK

On Apr 12, 2012, at 8:01, Jason Grout  wrote:

> On 4/12/12 9:43 AM, Volker Braun wrote:
>> Sage is getting ready to upgrade to the new IPython, excuses for any email
>> deluge on this list. One question that I have is whether you thought about
>> fork() to quickly spawn children (this is crucial for Sage since starting up
>> a new Sage process is quite slow). According to the zeromq mailinglist, you
>> shouldn't fork after creating a zmq context. This is not just a theoretical
>> problem, I wrote a clustering tool for my own purposes and found out the
>> hard way that bad things can happen if you do.
>> 
>> Will IPython always be usable without zmq?  You are not planning to
>> eventually deprecate the non-zmq console client? Or maybe you have some
>> awesome other solution?
> 
> Just as a point of reference, this documentation page (which seems to be 
> outdated, maybe?) indicates that a single-user command line session will 
> eventually use a 2-process model with zmq to communicate between the 
> processes:
> 
> http://ipython.org/ipython-doc/dev/development/ipythonzmq.html
> 
> Thanks,
> 
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From ROsborn at anl.gov  Thu Apr 12 12:44:15 2012
From: ROsborn at anl.gov (Ray Osborn)
Date: Thu, 12 Apr 2012 11:44:15 -0500
Subject: [IPython-dev] Embedding IPython+zmq
References: <4F86EE45.2040909@creativetrax.com>
Message-ID: <808F6AA8-5CD4-452C-937C-D907276710C8@anl.gov>

I didn't want to hijack the other thread so I thought I would create a new one. I just read the page that Jason Grout references about making the two-process iPython standard. I have, I think successfully, adapted Robert Kern's method of creating a kernel that fakes zmq messages within a single process, so that I can embed the iPython shell in a Qt GUI. I need this because I want to display items in the iPython shell namespace in a separate widget and need to trigger Qt signals when certain items in the namespace are changed. As Robert admits, this requires hacking into the zmq code and is probably not as elegant as it could be. 

Are there plans to refactor the two-process code to enable a single-process mode easily? Or perhaps zmq messaging could be used to facilitate communication to external GUI widgets, such as providing a way of mapping messages to signals? This is not my area of expertise, so let me know if I have not explained the problem clearly enough. 

Thanks,
Ray

Begin forwarded message:

> From: Jason Grout 
> Subject: Re: [IPython-dev] IPython+zmq and fork()
> Date: April 12, 2012 10:01:25 AM CDT
> To: ipython-dev at scipy.org
> Reply-To: IPython developers list 
> 
> On 4/12/12 9:43 AM, Volker Braun wrote:
>> Sage is getting ready to upgrade to the new IPython, excuses for any email
>> deluge on this list. One question that I have is whether you thought about
>> fork() to quickly spawn children (this is crucial for Sage since starting up
>> a new Sage process is quite slow). According to the zeromq mailinglist, you
>> shouldn't fork after creating a zmq context. This is not just a theoretical
>> problem, I wrote a clustering tool for my own purposes and found out the
>> hard way that bad things can happen if you do.
>> 
>> Will IPython always be usable without zmq?  You are not planning to
>> eventually deprecate the non-zmq console client? Or maybe you have some
>> awesome other solution?
> 
> Just as a point of reference, this documentation page (which seems to be 
> outdated, maybe?) indicates that a single-user command line session will 
> eventually use a 2-process model with zmq to communicate between the 
> processes:
> 
> http://ipython.org/ipython-doc/dev/development/ipythonzmq.html
> 
> Thanks,
> 
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-- 
Ray Osborn
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov





From fperez.net at gmail.com  Thu Apr 12 23:42:37 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 12 Apr 2012 20:42:37 -0700
Subject: [IPython-dev] IPython+zmq and fork()
In-Reply-To: <4F86EE45.2040909@creativetrax.com>
References: <1334241818735-4855890.post@n6.nabble.com>
	<4F86EE45.2040909@creativetrax.com>
Message-ID: 

On Thu, Apr 12, 2012 at 8:01 AM, Jason Grout
 wrote:
> Just as a point of reference, this documentation page (which seems to be
> outdated, maybe?) indicates that a single-user command line session will
> eventually use a 2-process model with zmq to communicate between the
> processes:
>
> http://ipython.org/ipython-doc/dev/development/ipythonzmq.html

That already exists, and you can start it with

ipython console

which is capable of connecting to an already running kernel with the
`--existing` flag.

But to clarify, both for the benefit of Sage and any other project
using IPython: we have a policy of keeping the 'classic'
single-process IPython *always* installable with nothing beyond the
standard library (plus pyreadline and setuptools on Windows).  This
hasn't changed now, and there's no sensible reason to change that
ever.  There is enormous value in IPython being able to run just on a
'naked' python installation as a better interactive shell, and many
projects and use cases benefit from this, so we'll never change that
(if we ever accidentally break this, it should be reported as a bug).

Cheers,

f


From fperez.net at gmail.com  Thu Apr 12 23:48:01 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 12 Apr 2012 20:48:01 -0700
Subject: [IPython-dev] zmq when using ipython
In-Reply-To: <4F86EAFA.9010103@creativetrax.com>
References: <4F86E525.1090801@creativetrax.com>
	
	<4F86EAFA.9010103@creativetrax.com>
Message-ID: 

On Thu, Apr 12, 2012 at 7:47 AM, Jason Grout
 wrote:
> Thanks, this corrects a big misunderstanding I had (and maybe is also an
> answer to Volker's post). ?I thought *everything* in the new ipython
> depended on the messaging protocol and connections with zmq.

The biggest change for Sage is going to be that customizing IPython
now uses a richer but also more complex configuration system:

http://ipython.org/ipython-doc/dev/config/index.html

It appears a bit daunting at first, and it does have a few rough
edges, but by and large we've found that it works very well in letting
us configure all the many moving parts IPython now has while using the
full power of the Python language (since the config files are regular
.py).

It's *possible* that we've removed/forgotten something that Sage used
in its command-line client.  If that's the case let us know, as we
didn't intend to remove any of the features that made the kind of
aggressive customization Sage does impossible.  Such a change would be
just an accidental oversight, so just report it and we'll do our best
to restore any missing functionality.

Cheers,

f


From fperez.net at gmail.com  Fri Apr 13 00:17:13 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 12 Apr 2012 21:17:13 -0700
Subject: [IPython-dev] greedy completer breaks useful completions -
	fixable?
In-Reply-To: 
References: <6A0D9C2C-BC41-4F94-84FF-4860B0CEFDC7@yale.edu>
	
Message-ID: 

On Mon, Apr 2, 2012 at 2:55 PM, MinRK  wrote:
> This should absolutely be doable, and the current behavior should be
> considered a bug. ?I think greedy completion should produce a strict
> superset of the completions from the regular competer. ?The simplest way to
> guarantee that this happens might be to *not* replace the completer like we
> do, but call the original completer, then call a *different* one that only
> does the item-completion, and merge the results.

Agreed.

Cheers,

f


From matthew.brett at gmail.com  Fri Apr 13 01:16:39 2012
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 12 Apr 2012 22:16:39 -0700
Subject: [IPython-dev] Notebook command line - parameters checked?
Message-ID: 

Hi,

I just noticed that these command lines:

$ ipython notebook unlikely
$ ipython notebook unlikely=impossible
$ ipython notebook --pylab=impossible

all pass without remark, but this:

$ ipython notebook --unlikely=impossible

gives:

WARNING: Unrecognized alias: 'unlikely', it will probably have no
effect.[NotebookApp] Using existing profile dir:
u'/Users/mb312/.ipython/profile_default'

Is this intended (the no-comments I mean)?

See you,

Matthew


From jason-sage at creativetrax.com  Fri Apr 13 01:43:40 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 13 Apr 2012 00:43:40 -0500
Subject: [IPython-dev] zmq when using ipython
In-Reply-To: 
References: <4F86E525.1090801@creativetrax.com>
	
	<4F86EAFA.9010103@creativetrax.com>
	
Message-ID: <4F87BD0C.3010901@creativetrax.com>

On 4/12/12 10:48 PM, Fernando Perez wrote:
> On Thu, Apr 12, 2012 at 7:47 AM, Jason Grout
>   wrote:
>> Thanks, this corrects a big misunderstanding I had (and maybe is also an
>> answer to Volker's post).  I thought *everything* in the new ipython
>> depended on the messaging protocol and connections with zmq.
>
> The biggest change for Sage is going to be that customizing IPython
> now uses a richer but also more complex configuration system:
>
> http://ipython.org/ipython-doc/dev/config/index.html
>
> It appears a bit daunting at first, and it does have a few rough
> edges, but by and large we've found that it works very well in letting
> us configure all the many moving parts IPython now has while using the
> full power of the Python language (since the config files are regular
> .py).
>
> It's *possible* that we've removed/forgotten something that Sage used
> in its command-line client.  If that's the case let us know, as we
> didn't intend to remove any of the features that made the kind of
> aggressive customization Sage does impossible.  Such a change would be
> just an accidental oversight, so just report it and we'll do our best
> to restore any missing functionality.
>

Mike Hansen posted a working patch for the latest Sage build: 
http://trac.sagemath.org/sage_trac/ticket/12719

I've been using it for about a day now, and haven't noticed any real 
problems.

I'm also interested in the forking issue.  With the Sage Cell Server 
[1], we end up forking a running Sage instance to create a new worker 
process (i.e., kernel).  We're still thinking through the design and how 
we can best take advantage of the new IPython functionality, so I'll 
probably have more coherent questions later.  However, one feature we'd 
love to see is allowing user code to send messages in the same channel 
as normal output messages.  For example, I'd like to send the following 
message, which our custom IPython client already interprets to make a 
Sage interact with javascript controls:

{"msg_type": "interact_prepare",
  "content": {"interact_id": "2096051932116277084",
              "layout": {"top_center": ["n"]},
              "update": {"n": ["n"]},
              "controls": {"n": {"control_type": "slider", "raw": true, 
"default": 0.0, "step": 0.004, "label": null, "subtype": "continuous", 
"range": [0.0, 1.0], "display_value": true}}
             }
}

Last year, we talked about wrapping such user messages so that the 
message the client would actually see this:

{"header": {...},
"parent_header": {...},
"msg_type": "extension",
"content": user_message, # message above
}

Is there a way to send such messages from userspace code?

Thanks,

Jason


From JDM at MarchRay.net  Fri Apr 13 10:28:36 2012
From: JDM at MarchRay.net (Jonathan March)
Date: Fri, 13 Apr 2012 09:28:36 -0500
Subject: [IPython-dev] QT-console bugfix commits and shepherding
Message-ID: 

Hi core folk,

There are two new qtconsole bugfix PRs
https://github.com/ipython/ipython/pull/1569
https://github.com/ipython/ipython/pull/1573

I with colleagues at Enthought have reviewed them both, and I would like to
ask permission to merge them.

I have commit rights, which* *I have not hitherto used for non-trivial
commits.

Since Enthought is using qtconsole regularly, while you are deeply engaged
in notebook development, and @epatters is busy with school, I would also
offer to take on the shepherding and reviewing of qtconsole PRs, if that's
agreeable to you. I am by no means QT-wise, but can draw on the
considerable expertise of colleagues.

Thanks,
Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Fri Apr 13 21:36:05 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 13 Apr 2012 18:36:05 -0700
Subject: [IPython-dev] QT-console bugfix commits and shepherding
In-Reply-To: 
References: 
Message-ID: 

Hi Jonathan,

On Fri, Apr 13, 2012 at 7:28 AM, Jonathan March  wrote:
> Hi core folk,
>
> There are two new qtconsole bugfix PRs
> https://github.com/ipython/ipython/pull/1569

There are some minor nits to be resolved here, and then it looks good to go.

> https://github.com/ipython/ipython/pull/1573

Yup, I see you merged it after my feedback, thanks!  BTW, as a
guideline for future merges, in the 'blue box' merge widget that
github offers, try to always add at the end a line that says

Closes #NN...

so that any relevant issues get auto-closed upon merge.  In this case,
I closed manually #1574, but it's a good idea to keep the merge
messages as a summary of the issues the entire merge addresses (esp.
since a large merge may fix multiple issues).

> I with colleagues at Enthought have reviewed them both, and I would like to
> ask permission to merge them.
>
> I have commit rights, which I have not hitherto used for non-trivial
> commits.
>
> Since Enthought is using qtconsole regularly, while you are deeply engaged
> in notebook development, and @epatters is busy with school, I would also
> offer to take on the shepherding and reviewing of qtconsole PRs, if that's
> agreeable to you. I am by no means QT-wise, but can draw on the considerable
> expertise of colleagues.

That would be great.  You should still allow time for other devs to
also have a chance to have a look; e.g. for #1573 I had some small
suggestions for improvements before merging.  But if you guys at
Enthought give it a solid round of testing and review, that
drastically reduces the burden on other reviewers, and the chances of
a fast merge go way up.

So as a guideline, I suggest: you (collective Enthought you)
test/review these Qt-focused PRs (and you're welcome to do the same in
other parts of the codebase, of course), and leave that feedback in
the PR discussion so everyone else can see it.  If in a few days once
it's been reviewed you haven't gotten any other eyes on that
particular PR from anyone else, I suggest you give us one last ping
on-list, and if nobody responds within a reasonable delay (say a day
or two in non-crazy-holiday times), go ahead and merge.  This should
give everyone a chance to have a look at the code, while not
bottlenecking you.

It's fantastic to see that you guys can take some of the effort on
this code, because as you correctly point out, right now we don't have
the bandwidth for it.

How does the above sound?

Cheers,

f

ps - as a quick reminder, when doing merges, even single-commit ones,
let's always use the github merge button, and put in a reasonable
message that describes the overall PR (especially useful for a long,
multi-commit PR).  A good starting point for that message is often the
introductory message of the PR itself, though I often manually tweak
things for clarity and conciseness.


From fperez.net at gmail.com  Sat Apr 14 05:17:46 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 14 Apr 2012 02:17:46 -0700
Subject: [IPython-dev] Heads-up (esp. Thomas) - eyes needed on 74c10b3c
Message-ID: 

Howdy,

after an insane 5 weeks (lots of good news for IPython, I'll try to
post a summary soon), I'm trying to catch up on the backlog, and
yesterday merged several PRs.  In particular,
https://github.com/ipython/ipython/pull/1538, which unfortunately I
failed to notice broke the py3 builds.

Fortunately thanks to Thomas' setup of shining panda, I got a warning
very quickly (it will be great when we can trigger these builds *pre*
merging with something like the sympy bot/github integration).  So
I've just pushed a quick emergency commit which (I did test) fixes
installation and the full test suite on python3:

https://github.com/ipython/ipython/commit/74c10b3c03e377612b54eac8eb19fe0c95930683

but I'd like in particular Thomas to have a look in case there's
anything specific to our py3 machinery I should/could have done
differently.  I think it's OK, but since I don't use py3 regularly, a
bit of review would be good (and  in any case, for these kinds of
unreviewed emergency commits, post-hoc review is always a good
practice).

Cheers,

f


From fperez.net at gmail.com  Sat Apr 14 05:56:27 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 14 Apr 2012 02:56:27 -0700
Subject: [IPython-dev] PR reviews needed, big backlog...
Message-ID: 

Hi all,

we currently have 25 PRs open; I've tried to close a bunch of Min's so
he would have his hands free to review other people's work, but it
would be great if anyone with a bit of time can pitch in and give us a
hand testing/reviewing these.  Even if the core devs end up doing the
merges, it's always great to see a PR on which there has already been
good discussion and fine-tuning.  And reviewing other people's PRs is
a great way to learn about parts of IPython you might not be familiar
with!

We still have some real work to do before we can push 0.13 out, but
obviously 25 outstanding PRs is a backlog we need to clear before that
can happen, so any help anyone can provide will be greatly
appreciated!

Cheers,

f


From takowl at gmail.com  Sat Apr 14 13:21:03 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 14 Apr 2012 18:21:03 +0100
Subject: [IPython-dev] Heads-up (esp. Thomas) - eyes needed on 74c10b3c
In-Reply-To: 
References: 
Message-ID: 

On 14 April 2012 10:17, Fernando Perez  wrote:
> https://github.com/ipython/ipython/commit/74c10b3c03e377612b54eac8eb19fe0c95930683
>
> but I'd like in particular Thomas to have a look in case there's
> anything specific to our py3 machinery I should/could have done
> differently. ?I think it's OK, but since I don't use py3 regularly, a
> bit of review would be good (and ?in any case, for these kinds of
> unreviewed emergency commits, post-hoc review is always a good
> practice).

We're interpolating a bytes string into a regular string, which
doesn't quite work as expected:

commit = "b'816e3fa'"

We'll need to explicitly decode it first.

Apart from that, since the file should always be pure ASCII, it should
be possible to use the built in open() function, rather than
io.open(), and avoid having to do map(unicode, ...).

I'll prepare a small PR to add to our stack ;-)

Thomas


From benjaminrk at gmail.com  Sat Apr 14 14:38:04 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 14 Apr 2012 11:38:04 -0700
Subject: [IPython-dev] Heads-up (esp. Thomas) - eyes needed on 74c10b3c
In-Reply-To: 
References: 
	
Message-ID: 

On Sat, Apr 14, 2012 at 10:21, Thomas Kluyver  wrote:

> On 14 April 2012 10:17, Fernando Perez  wrote:
> >
> https://github.com/ipython/ipython/commit/74c10b3c03e377612b54eac8eb19fe0c95930683
> >
> > but I'd like in particular Thomas to have a look in case there's
> > anything specific to our py3 machinery I should/could have done
> > differently.  I think it's OK, but since I don't use py3 regularly, a
> > bit of review would be good (and  in any case, for these kinds of
> > unreviewed emergency commits, post-hoc review is always a good
> > practice).
>
> We're interpolating a bytes string into a regular string, which
> doesn't quite work as expected:
>
> commit = "b'816e3fa'"
>
> We'll need to explicitly decode it first.
>
> Apart from that, since the file should always be pure ASCII, it should
> be possible to use the built in open() function, rather than
> io.open(), and avoid having to do map(unicode, ...).
>
> I'll prepare a small PR to add to our stack ;-)
>

Yes, I imagine just using builtin open and str literals is the right way to
go. No need to deal with unicode for two lines of ascii text.  I just got
carried away with our recently discovery that we should use io.open
everywhere (clearly that excludes setup).


>
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From takowl at gmail.com  Sat Apr 14 14:50:03 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 14 Apr 2012 19:50:03 +0100
Subject: [IPython-dev] Heads-up (esp. Thomas) - eyes needed on 74c10b3c
In-Reply-To: 
References: 
	
	
Message-ID: 

On 14 April 2012 19:38, MinRK  wrote:
> I just got carried away with our recently discovery that we should use
> io.open everywhere (clearly that excludes setup).

I think it's a sensible approach, even in setup, if we had to handle
non-ascii characters - but thankfully we don't.

Thomas


From benjaminrk at gmail.com  Sat Apr 14 15:03:13 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 14 Apr 2012 12:03:13 -0700
Subject: [IPython-dev] Heads-up (esp. Thomas) - eyes needed on 74c10b3c
In-Reply-To: 
References: 
	
	
	
Message-ID: 

On Sat, Apr 14, 2012 at 11:50, Thomas Kluyver  wrote:

> On 14 April 2012 19:38, MinRK  wrote:
> > I just got carried away with our recently discovery that we should use
> > io.open everywhere (clearly that excludes setup).
>
> I think it's a sensible approach, even in setup, if we had to handle
> non-ascii characters - but thankfully we don't.
>

Sure - if we were doing real arbitrary text handling, but two lines of
ascii text is not that, and your PR is the right approach.  No need to
solve problems we don't have.


>
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 02:17:42 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 14 Apr 2012 23:17:42 -0700
Subject: [IPython-dev] Security
In-Reply-To: <4F8579EE.6030109@creativetrax.com>
References: <4F8491C5.1030705@creativetrax.com>
	
	<4F8579EE.6030109@creativetrax.com>
Message-ID: 

On Wed, Apr 11, 2012 at 05:32, Jason Grout wrote:

> On 4/10/12 3:53 PM, MinRK wrote:
> > I just tried IPC on my latptop, and it definitely does work, though the
> > config is a bit weird, due to some assumptions that TCP is what people
> > actually use:
> >
> > $> ipcontroller --transport=ipc --ip=$HOME/ipcluster/socket
> >
> > will result in a bunch of files called `$HOME/ipcluster/socket:12345`.
>
> Thanks; I missed the --transport option.  When you say it is a bit
> weird, are you just talking about the --ip option?  What if we make --ip
> a synonym for --address?  --address seems to be more general (though it
> still doesn't seem to roll off the tongue as well as, say, --basename
> and using filenames that were more descriptive when using ipc).
>

There are just some TCP assumptions - for instance, it is weird to specify
`ip` as an arg that is actually a file, and it's also a bit weird that it
creates a series of files with 'socket:12345' filenames, due to the ip/port
assumptions in the code.  These are really cosmetic issues, of course.


>
> >   This sets up IPC communication for the cluster.
> >
> > The only thing I don't know about is forwarding UDS connections from the
> > client to the Hub, but if you know how to do that, you might be set
> > (some code would need to change in the ssh forwarding utils, but that
> > shouldn't be much).
>
> I was thinking that if you had forwarding set up, then you were sshing
> into the hub server and then running the client there.


If you are just going to ssh to the Hub machine and start the client there,
then there is nothing to tunnel, and this will just work as it is right
now.  I was thinking of using the tunneling we have now to start a Client
on a machine remote to the Hub, which requires setting up tunnels.



>  Hmmm...it does
> seem like an interesting problem to tunnel a UDS connection from the
> client machine to the hub machine.  If you do tunnel, then you're
> opening up a tcp port, so the advantages of not having a port open are
> out the window, I guess.  So maybe this is just a solution where the
> client and hub are all on one computer.  Or, like in our planned case,
> we have Google App Engine messages coming in over a App Engine "channel"
> to the hub computer, then sent to the hub.
>
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 02:43:14 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sat, 14 Apr 2012 23:43:14 -0700
Subject: [IPython-dev] plaintext cells now called 'raw'
Message-ID: 

Heads-up for users collaborating on notebooks with git master,

notebook format v3 in 0.13 will introduce a new 'raw' cell, for entering
content
that should be passed untransformed during export to other formats (e.g.
for typing raw LaTeX or reST etc.).

This was called 'plaintext' until today, when this
PRwas merged, replacing
'plaintext' with 'raw', which makes a good deal more sense.  If you were
using this `plaintext` cell,
they will be renamed to 'raw' automatically, but you should be careful if
you are collaborating
with users also on master, to ensure that you update IPython together, as
master older than today will
not recognize the 'raw' cell-type.

We do try to be conservative with this sort of thing, but the name seemed
problematic, and with a relatively
low-profile feature only in master for a little while, it seemed better to
fix the name, than have some convoluted code
like a notebook version increment.

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Sun Apr 15 03:20:35 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 15 Apr 2012 00:20:35 -0700
Subject: [IPython-dev] Strange messages in IPython.parallel tests, Min,
	any idea?
Message-ID: 

Hey guys, esp. Min,

I just noticed on a test run these messages from the parallel part of the suite:

blanca[~]> iptest IPython.parallel
.................................................................[Application]
keys mismatch
[Application] Table _ exists and doesn't match db format, trying __1
.[Application] keys mismatch
[Application] Table _ exists and doesn't match db format, trying __1
.[Application] keys mismatch
[Application] Table _ exists and doesn't match db format, trying __1
.[Application] keys mismatch
etc...

They repeat for a while, though they don't cause any actual errors. I
hadn't seen these before.  I haven't started bisecting it yet, though
a look at the source shows it comes from here:

blanca[IPython]> grin "match db format"
./parallel/controller/sqlitedb.py:
  235 :                 "Table %s exists and doesn't match db format,
trying %s"%


Min, do you have any idea why these are happening?  I don't really
know that part of the code too much, so before digging in, I figured
I'd ask you first.

Cheers,

f


From fperez.net at gmail.com  Sun Apr 15 03:24:07 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 15 Apr 2012 00:24:07 -0700
Subject: [IPython-dev] Shining panda reports Mac/OSX test failure
Message-ID: 

Thomas, thanks again for setting up the SP machinery!!!  We're really
seeing the benefits of it now, esp. with the multi-platform bots that
are running (and kudos to Mathias for contributing the Mac OSX bot)!

I just saw this report:

https://jenkins.shiningpanda.com/ipython/job/ipython-mac/76/testReport/junit/IPython.core.tests/test_magic/test_notebook_reformat_json/

It may have been something I've merged (since yesterday we've gone
from 32 to 16 open PRs, so a ton of new code has landed), but
unfortunately I wouldn't have seen this as it works on my box:

blanca[~]> iptest -vv IPython.core.tests.test_magic:test_notebook_reformat_json
IPython.core.tests.test_magic.test_notebook_reformat_json ... ok

----------------------------------------------------------------------
Ran 1 test in 0.007s

OK


Can anyone on a Mac reproduce/fix this one?

Cheers,

f


From fperez.net at gmail.com  Sun Apr 15 04:16:14 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 15 Apr 2012 01:16:14 -0700
Subject: [IPython-dev] Strange test failure on Shining Panda with parallel
	tests...
Message-ID: 

Hey Min,

this one looks odd to me, b/c the tests do pass just fine on my
system, but for some reason the ar_sugar merge is causing trouble on
one of the linux build bots:

https://jenkins.shiningpanda.com/ipython/job/ipython-multi/PLATFORM=debian6,PYTHON=CPython-2.6/67/console

Any idea what's going on there?  Those attributes were all added in
that PR, so I'm not sure why they're not being found...

Cheers,

f


From takowl at gmail.com  Sun Apr 15 07:16:36 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 15 Apr 2012 12:16:36 +0100
Subject: [IPython-dev] Shining panda reports Mac/OSX test failure
In-Reply-To: 
References: 
Message-ID: 

On 15 April 2012 08:24, Fernando Perez  wrote:
> It may have been something I've merged (since yesterday we've gone
> from 32 to 16 open PRs, so a ton of new code has landed), but
> unfortunately I wouldn't have seen this as it works on my box:

I think I've got it: https://github.com/ipython/ipython/pull/1600

It's a regular annoyance that io.open takes its default encoding from
the locale, rather than just using utf-8. Of course, for all of us on
Linux or Mac, our locale specifies UTF-8 anyway, so we don't notice. I
guess Matthias' cron job runs with an ascii locale, which reveals the
problem.

Thomas


From bussonniermatthias at gmail.com  Sun Apr 15 07:28:12 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Sun, 15 Apr 2012 13:28:12 +0200
Subject: [IPython-dev] Shining panda reports Mac/OSX test failure
In-Reply-To: 
References: 
	
Message-ID: <4B34D2C2-06BD-4359-B7A8-C4C4171C04B1@gmail.com>


Le 15 avr. 2012 ? 13:16, Thomas Kluyver a ?crit :

> On 15 April 2012 08:24, Fernando Perez  wrote:
>> It may have been something I've merged (since yesterday we've gone
>> from 32 to 16 open PRs, so a ton of new code has landed), but
>> unfortunately I wouldn't have seen this as it works on my box:
> 
> I think I've got it: https://github.com/ipython/ipython/pull/1600
> 
> It's a regular annoyance that io.open takes its default encoding from
> the locale, rather than just using utf-8. Of course, for all of us on
> Linux or Mac, our locale specifies UTF-8 anyway, so we don't notice. I
> guess Matthias' cron job runs with an ascii locale, which reveals the
> problem.
> 

hum...

me at mycomputer$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

Virtualenv  maybe ? 

I'll try to look at it monday or tuesday.

On a side note, do anyone plan on attending EuroSciPy in August ? 
-- 
Matthias




From takowl at gmail.com  Sun Apr 15 07:54:46 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 15 Apr 2012 12:54:46 +0100
Subject: [IPython-dev] Shining panda reports Mac/OSX test failure
In-Reply-To: <4B34D2C2-06BD-4359-B7A8-C4C4171C04B1@gmail.com>
References: 
	
	<4B34D2C2-06BD-4359-B7A8-C4C4171C04B1@gmail.com>
Message-ID: 

On 15 April 2012 12:28, Matthias BUSSONNIER
 wrote:
> me at mycomputer$ locale

That's not necessarily the same for a cron job, though:
http://www.logikdev.com/2010/02/02/locale-settings-for-your-cron-job/

In any case, it's a useful test, because for other systems (Windows),
the default encoding really won't be UTF-8. Hopefully my fix will
work.

Thomas


From benjaminrk at gmail.com  Sun Apr 15 13:51:48 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 Apr 2012 10:51:48 -0700
Subject: [IPython-dev] Strange messages in IPython.parallel tests, Min,
 any idea?
In-Reply-To: 
References: 
Message-ID: 

On Sun, Apr 15, 2012 at 00:20, Fernando Perez  wrote:

> Hey guys, esp. Min,
>
> I just noticed on a test run these messages from the parallel part of the
> suite:
>
> blanca[~]> iptest IPython.parallel
>
> .................................................................[Application]
> keys mismatch
> [Application] Table _ exists and doesn't match db format, trying __1
> .[Application] keys mismatch
> [Application] Table _ exists and doesn't match db format, trying __1
> .[Application] keys mismatch
> [Application] Table _ exists and doesn't match db format, trying __1
> .[Application] keys mismatch
> etc...
>
> They repeat for a while, though they don't cause any actual errors. I
> hadn't seen these before.  I haven't started bisecting it yet, though
> a look at the source shows it comes from here:
>
> blanca[IPython]> grin "match db format"
> ./parallel/controller/sqlitedb.py:
>  235 :                 "Table %s exists and doesn't match db format,
> trying %s"%
>
>
> Min, do you have any idea why these are happening?  I don't really
> know that part of the code too much, so before digging in, I figured
> I'd ask you first.
>

It's the added `received` column in the sqlitedb, which changes the DB
format.  If you scrub the `iptest` profile, it should stop happening.  I
will look into trying to make sure it's clean.


>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 14:15:04 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 Apr 2012 11:15:04 -0700
Subject: [IPython-dev] Strange messages in IPython.parallel tests, Min,
 any idea?
In-Reply-To: 
References: 
	
Message-ID: 

On Sun, Apr 15, 2012 at 10:51, MinRK  wrote:

>
>
> On Sun, Apr 15, 2012 at 00:20, Fernando Perez wrote:
>
>> Hey guys, esp. Min,
>>
>> I just noticed on a test run these messages from the parallel part of the
>> suite:
>>
>> blanca[~]> iptest IPython.parallel
>>
>> .................................................................[Application]
>> keys mismatch
>> [Application] Table _ exists and doesn't match db format, trying __1
>> .[Application] keys mismatch
>> [Application] Table _ exists and doesn't match db format, trying __1
>> .[Application] keys mismatch
>> [Application] Table _ exists and doesn't match db format, trying __1
>> .[Application] keys mismatch
>> etc...
>>
>> They repeat for a while, though they don't cause any actual errors. I
>> hadn't seen these before.  I haven't started bisecting it yet, though
>> a look at the source shows it comes from here:
>>
>> blanca[IPython]> grin "match db format"
>> ./parallel/controller/sqlitedb.py:
>>  235 :                 "Table %s exists and doesn't match db format,
>> trying %s"%
>>
>>
>> Min, do you have any idea why these are happening?  I don't really
>> know that part of the code too much, so before digging in, I figured
>> I'd ask you first.
>>
>
> It's the added `received` column in the sqlitedb, which changes the DB
> format.  If you scrub the `iptest` profile, it should stop happening.  I
> will look into trying to make sure it's clean.
>

Actually, it's not even in the profile dir, it's making a temporary db file
(/tmp/tasks.db on Linux, probably), but doesn't clean it up.  I'll push
with the fix shortly.


>
>
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 14:22:18 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 Apr 2012 11:22:18 -0700
Subject: [IPython-dev] Strange messages in IPython.parallel tests, Min,
 any idea?
In-Reply-To: 
References: 
	
	
Message-ID: 

PR issued 

On Sun, Apr 15, 2012 at 11:15, MinRK  wrote:

>
>
> On Sun, Apr 15, 2012 at 10:51, MinRK  wrote:
>
>>
>>
>> On Sun, Apr 15, 2012 at 00:20, Fernando Perez wrote:
>>
>>> Hey guys, esp. Min,
>>>
>>> I just noticed on a test run these messages from the parallel part of
>>> the suite:
>>>
>>> blanca[~]> iptest IPython.parallel
>>>
>>> .................................................................[Application]
>>> keys mismatch
>>> [Application] Table _ exists and doesn't match db format, trying __1
>>> .[Application] keys mismatch
>>> [Application] Table _ exists and doesn't match db format, trying __1
>>> .[Application] keys mismatch
>>> [Application] Table _ exists and doesn't match db format, trying __1
>>> .[Application] keys mismatch
>>> etc...
>>>
>>> They repeat for a while, though they don't cause any actual errors. I
>>> hadn't seen these before.  I haven't started bisecting it yet, though
>>> a look at the source shows it comes from here:
>>>
>>> blanca[IPython]> grin "match db format"
>>> ./parallel/controller/sqlitedb.py:
>>>  235 :                 "Table %s exists and doesn't match db format,
>>> trying %s"%
>>>
>>>
>>> Min, do you have any idea why these are happening?  I don't really
>>> know that part of the code too much, so before digging in, I figured
>>> I'd ask you first.
>>>
>>
>> It's the added `received` column in the sqlitedb, which changes the DB
>> format.  If you scrub the `iptest` profile, it should stop happening.  I
>> will look into trying to make sure it's clean.
>>
>
> Actually, it's not even in the profile dir, it's making a temporary db
> file (/tmp/tasks.db on Linux, probably), but doesn't clean it up.  I'll
> push with the fix shortly.
>
>
>>
>>
>>>
>>> Cheers,
>>>
>>> f
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 14:42:20 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 Apr 2012 11:42:20 -0700
Subject: [IPython-dev] Strange test failure on Shining Panda with
	parallel tests...
In-Reply-To: 
References: 
Message-ID: 

On Sun, Apr 15, 2012 at 01:16, Fernando Perez  wrote:

> Hey Min,
>
> this one looks odd to me, b/c the tests do pass just fine on my
> system, but for some reason the ar_sugar merge is causing trouble on
> one of the linux build bots:
>
>
> https://jenkins.shiningpanda.com/ipython/job/ipython-multi/PLATFORM=debian6,PYTHON=CPython-2.6/67/console
>
> Any idea what's going on there?  Those attributes were all added in
> that PR, so I'm not sure why they're not being found...
>

They are properties, and I override `__getattr__()` for metadata access, so
for some reason Python 2.6 handles this differently than Python ? 2.7.
 I'll try to nail this one down.


>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 14:46:44 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 Apr 2012 11:46:44 -0700
Subject: [IPython-dev] Strange test failure on Shining Panda with
	parallel tests...
In-Reply-To: 
References: 
	
Message-ID: 

On Sun, Apr 15, 2012 at 11:42, MinRK  wrote:

>
>
> On Sun, Apr 15, 2012 at 01:16, Fernando Perez wrote:
>
>> Hey Min,
>>
>> this one looks odd to me, b/c the tests do pass just fine on my
>> system, but for some reason the ar_sugar merge is causing trouble on
>> one of the linux build bots:
>>
>>
>> https://jenkins.shiningpanda.com/ipython/job/ipython-multi/PLATFORM=debian6,PYTHON=CPython-2.6/67/console
>>
>> Any idea what's going on there?  Those attributes were all added in
>> that PR, so I'm not sure why they're not being found...
>>
>
> They are properties, and I override `__getattr__()` for metadata access,
> so for some reason Python 2.6 handles this differently than Python ? 2.7.
>  I'll try to nail this one down.
>

Aha!  timedelta.total_seconds() was added in Python 2.7.  I'll add a
wrapper, so that I don't rely on the method.  The error messages are just
super weird, due to the bizarre code path followed when there are
properties and getattr override and errors.  It seems to be doing:

try:
    ar.elapsed_property()
except:
    ar.__getattr__('elapsed')

So the error in the property is actually resulting in the error in getattr
(which should never have been called)


>
>
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Sun Apr 15 14:58:57 2012
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 15 Apr 2012 11:58:57 -0700
Subject: [IPython-dev] Strange test failure on Shining Panda with
	parallel tests...
In-Reply-To: 
References: 
	
	
Message-ID: 

Should be fixed 

On Sun, Apr 15, 2012 at 11:46, MinRK  wrote:

>
>
> On Sun, Apr 15, 2012 at 11:42, MinRK  wrote:
>
>>
>>
>> On Sun, Apr 15, 2012 at 01:16, Fernando Perez wrote:
>>
>>> Hey Min,
>>>
>>> this one looks odd to me, b/c the tests do pass just fine on my
>>> system, but for some reason the ar_sugar merge is causing trouble on
>>> one of the linux build bots:
>>>
>>>
>>> https://jenkins.shiningpanda.com/ipython/job/ipython-multi/PLATFORM=debian6,PYTHON=CPython-2.6/67/console
>>>
>>> Any idea what's going on there?  Those attributes were all added in
>>> that PR, so I'm not sure why they're not being found...
>>>
>>
>> They are properties, and I override `__getattr__()` for metadata access,
>> so for some reason Python 2.6 handles this differently than Python ? 2.7.
>>  I'll try to nail this one down.
>>
>
> Aha!  timedelta.total_seconds() was added in Python 2.7.  I'll add a
> wrapper, so that I don't rely on the method.  The error messages are just
> super weird, due to the bizarre code path followed when there are
> properties and getattr override and errors.  It seems to be doing:
>
> try:
>     ar.elapsed_property()
> except:
>     ar.__getattr__('elapsed')
>
> So the error in the property is actually resulting in the error in getattr
> (which should never have been called)
>
>
>>
>>
>>>
>>> Cheers,
>>>
>>> f
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Sun Apr 15 17:22:43 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 15 Apr 2012 14:22:43 -0700
Subject: [IPython-dev] Strange test failure on Shining Panda with
	parallel tests...
In-Reply-To: 
References: 
	
	
	
Message-ID: 

On Sun, Apr 15, 2012 at 11:58 AM, MinRK  wrote:
> Should be fixed

Great, thanks!  Will have a look next...


From fperez.net at gmail.com  Mon Apr 16 01:11:50 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 15 Apr 2012 22:11:50 -0700
Subject: [IPython-dev] IPEP 1: Cleanup and extension of the Magic system in
	IPython
Message-ID: 

Hi folks,

I've finally written up a plan for cell level magics and a full
cleanup of the magic system:

https://github.com/ipython/ipython/issues/1611

Please beat up on it, and once we reach some agreement, I'll start
implementing things...

I propose people submit feedback on the github issue itself, so it all
stays focused in one place and we don't swamp the list with this
discussion.

Cheers,

f


From jtaylor.debian at googlemail.com  Mon Apr 16 19:23:26 2012
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Tue, 17 Apr 2012 01:23:26 +0200
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
Message-ID: <4F8CA9EE.5080807@googlemail.com>

On 04/06/2012 11:39 PM, MinRK wrote:
> 
> 
> On Fri, Apr 6, 2012 at 14:18, Paul Ivanov  > wrote:
> 
>     Hey everyone,
> 
>     saw this branch got added to the git repo - is this for the purposes
>     of being updated in the new Ubuntu LTS release, as was discussed on in
>     an old thread?
> 
>     just wanted to know what the status and purpose of this branch.
> 
> 
> Yes, that's the idea, since 0.13 wouldn't have a chance of making it.
>  It's just bugfixes and a few minor enhancements since 0.12.  Were 12.04
> not an LTS, we wouldn't bother, and with the relatively near release of
> 0.13, it is still of dubious value, but there are some serious issues in
> 0.12 that we don't like the idea of surviving for 5 years.
> 

the timing is unfortunate but there is no good alternative. As the
recent notebook format change in master showed, a git snapshot of 0.13
would be even worse at this point.
I'll provide an official backport of 0.13 for 12.04 when its ready.

I'd like to remind you that the 0.12.1 release would be needed by the
end of this week latest. The archive freezes on the 24. and I need some
time for packaging too.
Preferably with unminified javascript sources or I'll have to repackage it.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From fperez.net at gmail.com  Mon Apr 16 19:30:18 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 16 Apr 2012 16:30:18 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: <4F8CA9EE.5080807@googlemail.com>
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
Message-ID: 

On Mon, Apr 16, 2012 at 4:23 PM, Julian Taylor
 wrote:
>
> I'd like to remind you that the 0.12.1 release would be needed by the
> end of this week latest. The archive freezes on the 24. and I need some
> time for packaging too.
> Preferably with unminified javascript sources or I'll have to repackage it.

Thanks for the reminder, Julian.  Min, what's the status of this
branch?  Should we more or less package it now as-is?  The release
notes for it can be *minimal*, as it's just a collection of bug fixes
only.

Cheers,

f


From benjaminrk at gmail.com  Mon Apr 16 19:53:21 2012
From: benjaminrk at gmail.com (Min RK)
Date: Mon, 16 Apr 2012 16:53:21 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
Message-ID: <489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>

I'd be fine with cutting it exactly as-is after some deployment testing. It already has basic generated Issues/PR lists.

Are there any of the issues closed in the last few days that should go in?

Candidates related to release/packaging: don't build sphinx docs, and unhide git info.

-MinRK

On Apr 16, 2012, at 16:30, Fernando Perez  wrote:

> On Mon, Apr 16, 2012 at 4:23 PM, Julian Taylor
>  wrote:
>> 
>> I'd like to remind you that the 0.12.1 release would be needed by the
>> end of this week latest. The archive freezes on the 24. and I need some
>> time for packaging too.
>> Preferably with unminified javascript sources or I'll have to repackage it.
> 
> Thanks for the reminder, Julian.  Min, what's the status of this
> branch?  Should we more or less package it now as-is?  The release
> notes for it can be *minimal*, as it's just a collection of bug fixes
> only.
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From fperez.net at gmail.com  Mon Apr 16 19:57:12 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 16 Apr 2012 16:57:12 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: <489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
Message-ID: 

On Mon, Apr 16, 2012 at 4:53 PM, Min RK  wrote:
> Are there any of the issues closed in the last few days that should go in?
>
> Candidates related to release/packaging: don't build sphinx docs, and unhide git info.

+1 for those two


From benjaminrk at gmail.com  Mon Apr 16 22:38:33 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 16 Apr 2012 19:38:33 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
Message-ID: 

On Mon, Apr 16, 2012 at 16:57, Fernando Perez  wrote:

> On Mon, Apr 16, 2012 at 4:53 PM, Min RK  wrote:
> > Are there any of the issues closed in the last few days that should go
> in?
> >
> > Candidates related to release/packaging: don't build sphinx docs, and
> unhide git info.
>
> +1 for those two
>

Okay, both are in now.  If we want to just draw the line here and start
testing, I can make a tarball (much less expensive, now that we don't
include docs!).


> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Mon Apr 16 22:43:38 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 16 Apr 2012 19:43:38 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
Message-ID: 

On Mon, Apr 16, 2012 at 7:38 PM, MinRK  wrote:
> Okay, both are in now. ?If we want to just draw the line here and start
> testing, I can make a tarball (much less expensive, now that we don't
> include docs!).

+1, thanks!


From benjaminrk at gmail.com  Mon Apr 16 23:35:08 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 16 Apr 2012 20:35:08 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
Message-ID: 

On Mon, Apr 16, 2012 at 19:43, Fernando Perez  wrote:

> On Mon, Apr 16, 2012 at 7:38 PM, MinRK  wrote:
> > Okay, both are in now.  If we want to just draw the line here and start
> > testing, I can make a tarball (much less expensive, now that we don't
> > include docs!).
>
> +1, thanks!
>

Okay, I'll get started.

As for unminified sources, jQuery and prettify are easy and pretty small,
and I would be fine adding them to the release.  jQuery-UI is the only
other minified source, and it's pretty large (~1MB and 10s of files after
stripping out support files), increasing the total install size by about 20%

Julian - what can I do to help you?

prettify unminified:
http://google-code-prettify.googlecode.com/files/prettify-1-Jun-2011.tar.bz2
jQuery-1.6.2 unminified: http://code.jquery.com/jquery-1.6.2.js
jQueryUI-1.8.14 unminified:
https://github.com/jquery/jquery-ui/tarball/1.8.14

If it's easy for you to add these in the packaging process that would be my
preference, but if not then we can consider bringing them in ourselves.

I've posted a first 0.12.1.rc1 on
archive.ipython.org

-MinRK


> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From Fernando.Perez at berkeley.edu  Tue Apr 17 21:46:10 2012
From: Fernando.Perez at berkeley.edu (Fernando Perez)
Date: Tue, 17 Apr 2012 18:46:10 -0700
Subject: [IPython-dev] msg spec
In-Reply-To: 
References: 
Message-ID: 

Hey,

[ Putting this on-list so we have a record of the conversation for the
archives, I also know that Jason Grout has been using the spec in some
of the new sage work, so hey may have a stake here]

On Tue, Apr 17, 2012 at 15:41, MinRK  wrote:
> I've started to write tests for the message spec, via the KernelManager, and
> I'm finding that we don't actually comply very well with the spec in our doc
> (first simple execute test has two failures).
>
> execute_reply:
>
> 1. payload - this is not a dict, it is a *list* of payload dicts

Yes, the docs need updating here: since payloads can be created by
user code at arbitrary times during execution, there's no way to put
them into a single dict.  We probably just hadn't realized that when
we first wrote the spec, and then forgot to update the docs as we
started having more than one payload come back.

> 2. transformed_input - this key is never in an execute_reply, it only
> appears in a specific auto_rewrite *payload*.

Mmh, we may have taken the cheap way out here: conceptually, since
code was sent out for execution to the kernel, so it makes sense that
the transformed code (if it was transformed) should come back as a
top-level entity.  However, implementation-wise that's probably tricky
to do, b/c inside the IPython object, the rewriting happens deep in
the bowels of the execution codepath.

Back in the days of terminal-only client that was no big deal: a
properly positioned 'print' statement was all one needed to see what
had happened.

But now we'd like that transformed code to propagate *back* all the
way to the caller, and that would require that this was also available
in the kernel instead of being dumped to stdout somewhere along the
way.

So we most likely punted, and replaced the 'print' with a payload
generator so at least the transformed output would make its way back
to the calling client, but didn't really honor the spec.

So the question is what to do now:

1. we can say that any rewrite of the input can happen at arbitrary
points and that therefore it makes more sense to let it be delivered
as a payload instead of a top-level return value.

2. or we stick to the spec, and then we have to fix the code.  The
hard way would require messing with the entire execution pipeline to
do it 'right', the easy one would be to put a check right before the
message is sent, pull any 'rewrite' payload we find out of the
payloads list and stick it into the 'transformed_code' field of the
message.  Then the message could be sent according to the current
spec.

I was initially thinking that 2 was really the way to do it, but now
I'm not so sure: in principle different parts of the execution
pipeline could apply different transformations of code (imagine our
base rewrite plus user-driven prefiltering).  So I'm now leaning
towards 1 as a sensible option (which has the advantage of being
trivial to implement, as it only requires updating the doc to mention
'rewrite' as a possible payload).

> I've been using traitlets to do the message validation, and I'm running into
> the fact that we are unclear about what keys can be None and what cannot.
>  Should I start with the assumption that *anything* can be None, or not?

I think that's a reasonable starting point. Once the tests are in
solid shape we can revisit this and go over the spec/implementation
more carefully.

Thanks for digging into this, it's one of the big missing pieces we have!

Cheers,

f


From jason-sage at creativetrax.com  Tue Apr 17 21:57:23 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 17 Apr 2012 20:57:23 -0500
Subject: [IPython-dev] msg spec
In-Reply-To: 
References: 
	
Message-ID: <4F8E1F83.9060409@creativetrax.com>

On 4/17/12 8:46 PM, Fernando Perez wrote:
> Hey,
>
> [ Putting this on-list so we have a record of the conversation for the
> archives, I also know that Jason Grout has been using the spec in some
> of the new sage work, so hey may have a stake here]

Thanks.  Up to this point, we've used the spec for a point of reference, 
but we've developed both the sender and receivers, so we've not been 
affected by non-compliant implementations.  However, we hope to start 
using the new ipython in our work this summer (moving aleph.sagemath.org 
to Google App Engine), so we are starting to look hard again at the spec 
and how to use ipython's native implementation.


So please, keep conversations about the messaging spec on the list.

>
> On Tue, Apr 17, 2012 at 15:41, MinRK  wrote:
>> I've started to write tests for the message spec, via the KernelManager, and
>> I'm finding that we don't actually comply very well with the spec in our doc
>> (first simple execute test has two failures).
>>
>> execute_reply:
>>
>> 1. payload - this is not a dict, it is a *list* of payload dicts
>
> Yes, the docs need updating here: since payloads can be created by
> user code at arbitrary times during execution, there's no way to put
> them into a single dict.  We probably just hadn't realized that when
> we first wrote the spec, and then forgot to update the docs as we
> started having more than one payload come back.
>
>> 2. transformed_input - this key is never in an execute_reply, it only
>> appears in a specific auto_rewrite *payload*.
>
> Mmh, we may have taken the cheap way out here: conceptually, since
> code was sent out for execution to the kernel, so it makes sense that
> the transformed code (if it was transformed) should come back as a
> top-level entity.  However, implementation-wise that's probably tricky
> to do, b/c inside the IPython object, the rewriting happens deep in
> the bowels of the execution codepath.
>
> Back in the days of terminal-only client that was no big deal: a
> properly positioned 'print' statement was all one needed to see what
> had happened.
>
> But now we'd like that transformed code to propagate *back* all the
> way to the caller, and that would require that this was also available
> in the kernel instead of being dumped to stdout somewhere along the
> way.
>
> So we most likely punted, and replaced the 'print' with a payload
> generator so at least the transformed output would make its way back
> to the calling client, but didn't really honor the spec.
>
> So the question is what to do now:
>
> 1. we can say that any rewrite of the input can happen at arbitrary
> points and that therefore it makes more sense to let it be delivered
> as a payload instead of a top-level return value.
>
> 2. or we stick to the spec, and then we have to fix the code.  The
> hard way would require messing with the entire execution pipeline to
> do it 'right', the easy one would be to put a check right before the
> message is sent, pull any 'rewrite' payload we find out of the
> payloads list and stick it into the 'transformed_code' field of the
> message.  Then the message could be sent according to the current
> spec.
>
> I was initially thinking that 2 was really the way to do it, but now
> I'm not so sure: in principle different parts of the execution
> pipeline could apply different transformations of code (imagine our
> base rewrite plus user-driven prefiltering).

Like the Sage preparser, right?


> So I'm now leaning
> towards 1 as a sensible option (which has the advantage of being
> trivial to implement, as it only requires updating the doc to mention
> 'rewrite' as a possible payload).
>
>> I've been using traitlets to do the message validation, and I'm running into
>> the fact that we are unclear about what keys can be None and what cannot.
>>   Should I start with the assumption that *anything* can be None, or not?
>
> I think that's a reasonable starting point. Once the tests are in
> solid shape we can revisit this and go over the spec/implementation
> more carefully.
>
> Thanks for digging into this, it's one of the big missing pieces we have!

Thanks also from us in the Sage aleph project for tightening loose ends 
up here!

Jason



From fperez.net at gmail.com  Tue Apr 17 21:58:27 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 17 Apr 2012 18:58:27 -0700
Subject: [IPython-dev] msg spec
In-Reply-To: <4F8E1F83.9060409@creativetrax.com>
References: 
	
	<4F8E1F83.9060409@creativetrax.com>
Message-ID: 

On Tue, Apr 17, 2012 at 6:57 PM, Jason Grout
 wrote:
> Like the Sage preparser, right?

Yes, that's precisely what I had in mind...


From satra at mit.edu  Tue Apr 17 22:27:04 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Tue, 17 Apr 2012 22:27:04 -0400
Subject: [IPython-dev] traitlets question [from Re:  msg spec]
Message-ID: 

> > I've been using traitlets to do the message validation, and I'm running
> into
> > the fact that we are unclear about what keys can be None and what cannot.
> >  Should I start with the assumption that *anything* can be None, or not?
>
> I think that's a reasonable starting point. Once the tests are in
> solid shape we can revisit this and go over the spec/implementation
> more carefully.
>

question out of not following the traitlets code. a long while back it was
deemed that traitlets needed to have every element defined. is that no
longer necessary? i.e could a trait be undefined (or set to None).

and to follow up, does traitlets have event delegation?

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Wed Apr 18 01:19:55 2012
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 17 Apr 2012 22:19:55 -0700
Subject: [IPython-dev] msg spec
In-Reply-To: 
References: 
	
	<4F8E1F83.9060409@creativetrax.com>
	
Message-ID: 

On Tue, Apr 17, 2012 at 18:58, Fernando Perez  wrote:

> On Tue, Apr 17, 2012 at 6:57 PM, Jason Grout
>  wrote:
> > Like the Sage preparser, right?
>
> Yes, that's precisely what I had in mind...
>

I'm fine leaving it in the payload system.  Having it in the actual reply
does make some sense.  Isn't it fairly clear the point immediately prior to
execution?  I don't know where precisely this is

Another msg spec failure, along my way:

We do not actually support the detail_level key in the object_info messages
(on either end), so presumably we are only doing detail_level=0.  Also the
spec doc suggests the only options are 1 and 2, but the default is in fact
0, and this is the only one we currently expose.

I presume just adding the arg to KM.object_info, and passing it to
inspector.info in Kernel.object_info_request is all that we need to do.


> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Wed Apr 18 01:53:23 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 17 Apr 2012 22:53:23 -0700
Subject: [IPython-dev] msg spec
In-Reply-To: 
References: 
	
	<4F8E1F83.9060409@creativetrax.com>
	
	
Message-ID: 

On Tue, Apr 17, 2012 at 10:19 PM, MinRK  wrote:
> I'm fine leaving it in the payload system. ?Having it in the actual reply
> does make some sense. ?Isn't it fairly clear the point immediately prior to
> execution? ?I don't know where precisely this is

Sorry, I can't quite understand if in the above you're voting for the
payload system or for 'fixing' it as it was originally intended...

> Another msg spec failure, along my way:
>
> We do not actually support the detail_level key in the object_info messages
> (on either end), so presumably we are only doing detail_level=0. ?Also the
> spec doc suggests the only options are 1 and 2, but the default is in fact
> 0, and this is the only one we currently expose.
>
> I presume just adding the arg to KM.object_info, and passing it to
> inspector.info in Kernel.object_info_request is all that we need to do.

Yes, that's right.  So far we've only used the info requests with
basic info, b/c 'x??' executes via a second codepath.  But if we
wanted for example to improve the calltips with a '+' button that
would fill in with the equivalent of '??', we'd need to support the
full value range.  So yes, let's fix this one...

Cheers,

f


From fperez.net at gmail.com  Wed Apr 18 21:14:42 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 18 Apr 2012 18:14:42 -0700
Subject: [IPython-dev] msg spec
In-Reply-To: <4F8E1F83.9060409@creativetrax.com>
References: 
	
	<4F8E1F83.9060409@creativetrax.com>
Message-ID: 

On Tue, Apr 17, 2012 at 6:57 PM, Jason Grout
 wrote:
> Thanks also from us in the Sage aleph project for tightening loose ends
> up here!

Jason, just to let you know that we've merged this PR:

https://github.com/ipython/ipython/pull/1627

So now at least the docs and spec are reasonably matched, and there's
a test suite that will catch deviations in the future.  There's still
room for improvement, obviously, but this is an awesome step forward,
thanks again to Min for pushing it.

Cheers,

f


From jason-sage at creativetrax.com  Wed Apr 18 22:24:08 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Wed, 18 Apr 2012 21:24:08 -0500
Subject: [IPython-dev] msg spec
In-Reply-To: 
References: 
	
	<4F8E1F83.9060409@creativetrax.com>
	
Message-ID: <4F8F7748.3080807@creativetrax.com>

On 4/18/12 8:14 PM, Fernando Perez wrote:
> On Tue, Apr 17, 2012 at 6:57 PM, Jason Grout
>   wrote:
>> Thanks also from us in the Sage aleph project for tightening loose ends
>> up here!
>
> Jason, just to let you know that we've merged this PR:
>
> https://github.com/ipython/ipython/pull/1627
>
> So now at least the docs and spec are reasonably matched, and there's
> a test suite that will catch deviations in the future.  There's still
> room for improvement, obviously, but this is an awesome step forward,
> thanks again to Min for pushing it.

Cool!  Thanks!

Jason


From benjaminrk at gmail.com  Thu Apr 19 18:31:58 2012
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 19 Apr 2012 15:31:58 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
Message-ID: 

On Mon, Apr 16, 2012 at 20:35, MinRK  wrote:

>
>
> On Mon, Apr 16, 2012 at 19:43, Fernando Perez wrote:
>
>> On Mon, Apr 16, 2012 at 7:38 PM, MinRK  wrote:
>> > Okay, both are in now.  If we want to just draw the line here and start
>> > testing, I can make a tarball (much less expensive, now that we don't
>> > include docs!).
>>
>> +1, thanks!
>>
>
> Okay, I'll get started.
>
> As for unminified sources, jQuery and prettify are easy and pretty small,
> and I would be fine adding them to the release.  jQuery-UI is the only
> other minified source, and it's pretty large (~1MB and 10s of files after
> stripping out support files), increasing the total install size by about 20%
>
> Julian - what can I do to help you?
>
> prettify unminified:
> http://google-code-prettify.googlecode.com/files/prettify-1-Jun-2011.tar.bz2
> jQuery-1.6.2 unminified: http://code.jquery.com/jquery-1.6.2.js
> jQueryUI-1.8.14 unminified:
> https://github.com/jquery/jquery-ui/tarball/1.8.14
>
> If it's easy for you to add these in the packaging process that would be
> my preference, but if not then we can consider bringing them in ourselves.
>
> I've posted a first 0.12.1.rc1 on archive.ipython.org
>


0.12.1.rc2 made,
with minor tweaks:

* unminified prettify.js included for packagers
* don't gzip manpages in sdist
* use raw-string band-aid for #1635 to get Windows test passing

I've got all tests passing on my machines, so I expect this will be 0.12.1
unless anyone reports issues.

-MinRK



>
> -MinRK
>
>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Thu Apr 19 20:15:18 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 19 Apr 2012 17:15:18 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
	
Message-ID: 

On Thu, Apr 19, 2012 at 3:31 PM, MinRK  wrote:
>
> I've got all tests passing on my machines, so I expect this will be 0.12.1
> unless anyone reports issues.

Awesome, thanks a ton Min!  Julian, let us know if this is OK for you
and we can officially tag things.

Cheers,

f


From matthew.brett at gmail.com  Fri Apr 20 13:59:21 2012
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 20 Apr 2012 10:59:21 -0700
Subject: [IPython-dev] Notebook command line - parameters checked?
In-Reply-To: 
References: 
Message-ID: 

Hi,

On Thu, Apr 12, 2012 at 10:16 PM, Matthew Brett  wrote:
> Hi,
>
> I just noticed that these command lines:
>
> $ ipython notebook unlikely
> $ ipython notebook unlikely=impossible
> $ ipython notebook --pylab=impossible
>
> all pass without remark, but this:
>
> $ ipython notebook --unlikely=impossible
>
> gives:
>
> WARNING: Unrecognized alias: 'unlikely', it will probably have no
> effect.[NotebookApp] Using existing profile dir:
> u'/Users/mb312/.ipython/profile_default'
>
> Is this intended (the no-comments I mean)?

Should I enter an issue for this?

Best,

Matthew


From damianavila at gmail.com  Fri Apr 20 14:31:49 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Fri, 20 Apr 2012 15:31:49 -0300
Subject: [IPython-dev] Export notebook to image >>> not editable print view
Message-ID: <4F91AB95.8090808@gmail.com>

Working in a ipython notebook plugin for an ide, I realize that the 
print view of the notebook is functional notebook, and hence, it is 
editable.
For a print view, it would be a better options to get an non-editable 
notebook. There was a little discussion in github around this topic 
(https://github.com/ipython/ipython/issues/1292). 
Brian propose to generate a static html for the print view. Then, I 
proposed another way, so I bring the discussion here:

We can capture the notebook into a png (and other format) images using 
qwebkit inside PyQt (we have the last one as dependency for Qt console). 
In this way, first, we can support another ways of export the notebook, 
such as images...
Then, we can import this notebook images inside PyQt objects and 
displayed into the javascript enviroment...
I wrote a little script to capture a complete image of the notebook and 
show the image inside a web browser and it is displaying great, like the 
notebook itself.

Another way to have a non editable print view would be an export to a 
raw html (https://github.com/ipython/ipython/issues/860), but we have to 
deal with MathJax, CodeMirror and Markdown.

What do you think about the "image" way to generate a non-editable view 
of the notebook? I think it would be easy to implement.

Regards.

Dami?n.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Fri Apr 20 14:42:19 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 11:42:19 -0700
Subject: [IPython-dev] Notebook command line - parameters checked?
In-Reply-To: 
References: 
	
Message-ID: 

Hey Matthew,

sorry, this one fell through the cracks..

> On Thu, Apr 12, 2012 at 10:16 PM, Matthew Brett  wrote:
>> Hi,
>>
>> I just noticed that these command lines:
>>
>> $ ipython notebook unlikely
>> $ ipython notebook unlikely=impossible
>> $ ipython notebook --pylab=impossible
>>
>> all pass without remark, but this:
>>
>> $ ipython notebook --unlikely=impossible
>>
>> gives:
>>
>> WARNING: Unrecognized alias: 'unlikely', it will probably have no
>> effect.[NotebookApp] Using existing profile dir:
>> u'/Users/mb312/.ipython/profile_default'
>>
>> Is this intended (the no-comments I mean)?
>
> Should I enter an issue for this?

Yes, I think so, though I'm not sure how fixable it is, Min will have
a better idea: the problem is that the command-line handling captures
flags and then has to pass that information 'downstream' to all
components that make up an application, but it doesn't know in advance
what all the valid flags are from 'below'.  But definitely file it so
we have a focused place to document/discuss this...

Cheers,

f


From matthew.brett at gmail.com  Fri Apr 20 14:48:23 2012
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 20 Apr 2012 11:48:23 -0700
Subject: [IPython-dev] Notebook command line - parameters checked?
In-Reply-To: 
References: 
	
	
Message-ID: 

Hi,

On Fri, Apr 20, 2012 at 11:42 AM, Fernando Perez  wrote:
> Hey Matthew,
>
> sorry, this one fell through the cracks..
>
>> On Thu, Apr 12, 2012 at 10:16 PM, Matthew Brett  wrote:
>>> Hi,
>>>
>>> I just noticed that these command lines:
>>>
>>> $ ipython notebook unlikely
>>> $ ipython notebook unlikely=impossible
>>> $ ipython notebook --pylab=impossible
>>>
>>> all pass without remark, but this:
>>>
>>> $ ipython notebook --unlikely=impossible
>>>
>>> gives:
>>>
>>> WARNING: Unrecognized alias: 'unlikely', it will probably have no
>>> effect.[NotebookApp] Using existing profile dir:
>>> u'/Users/mb312/.ipython/profile_default'
>>>
>>> Is this intended (the no-comments I mean)?
>>
>> Should I enter an issue for this?
>
> Yes, I think so, though I'm not sure how fixable it is, Min will have
> a better idea: the problem is that the command-line handling captures
> flags and then has to pass that information 'downstream' to all
> components that make up an application, but it doesn't know in advance
> what all the valid flags are from 'below'. ?But definitely file it so
> we have a focused place to document/discuss this...

Thanks:

https://github.com/ipython/ipython/issues/1639

See you,

Matthew


From takowl at gmail.com  Fri Apr 20 18:42:34 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 20 Apr 2012 23:42:34 +0100
Subject: [IPython-dev] Export notebook to image >>> not editable print
	view
In-Reply-To: <4F91AB95.8090808@gmail.com>
References: <4F91AB95.8090808@gmail.com>
Message-ID: 

Immediate thought: not as an image, please. Documents saved as images
are inefficient and unhelpful (want to copy some text? hard luck!). If
we're going to use the browser in a way like that, I'm pretty sure
it's possible to export a PDF, which at least keeps text as text. See
e.g. http://code.google.com/p/wkhtmltopdf/ , which uses QWebkit for a
similar purpose.

Also, I don't think this is an alternative to a real static HTML
export. The notebook shouldn't depend on having PyQt/PySide installed
on either the server or the client. It could be useful as a temporary
measure while we work out the details of producing static HTML,
though.

Thanks,
Thomas

On 20 April 2012 19:31, Dami?n Avila  wrote:
> Working in a ipython notebook plugin for an ide, I realize that the print
> view of the notebook is functional notebook, and hence, it is editable.
> For a print view, it would be a better options to get an non-editable
> notebook. There was a little discussion in github around this topic
> (https://github.com/ipython/ipython/issues/1292). Brian propose to generate
> a static html for the print view. Then, I proposed another way, so I bring
> the discussion here:
>
> We can capture the notebook into a png (and other format) images using
> qwebkit inside PyQt (we have the last one as dependency for Qt console). In
> this way, first, we can support another ways of export the notebook, such as
> images...
> Then, we can import this notebook images inside PyQt objects and displayed
> into the javascript enviroment...
> I wrote a little script to capture a complete image of the notebook and show
> the image inside a web browser and it is displaying great, like the notebook
> itself.
>
> Another way to have a non editable print view would be an export to a raw
> html (https://github.com/ipython/ipython/issues/860), but we have to deal
> with MathJax, CodeMirror and Markdown.
>
> What do you think about the "image" way to generate a non-editable view of
> the notebook? I think it would be easy to implement.
>
> Regards.
>
> Dami?n.
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From damianavila at gmail.com  Fri Apr 20 19:12:09 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Fri, 20 Apr 2012 20:12:09 -0300
Subject: [IPython-dev] Export notebook to image >>> not editable print
 view
In-Reply-To: 
References: <4F91AB95.8090808@gmail.com>
	
Message-ID: <4F91ED49.2010304@gmail.com>

El 20/04/12 19:42, Thomas Kluyver escribi?:
> Immediate thought: not as an image, please. Documents saved as images
> are inefficient and unhelpful (want to copy some text? hard luck!). If
> we're going to use the browser in a way like that, I'm pretty sure
> it's possible to export a PDF, which at least keeps text as text. See
> e.g. http://code.google.com/p/wkhtmltopdf/ , which uses QWebkit for a
> similar purpose.
>
> Also, I don't think this is an alternative to a real static HTML
> export. The notebook shouldn't depend on having PyQt/PySide installed
> on either the server or the client. It could be useful as a temporary
> measure while we work out the details of producing static HTML,
> though.
>
> Thanks,
> Thomas
>
> On 20 April 2012 19:31, Dami?n Avila  wrote:
>> Working in a ipython notebook plugin for an ide, I realize that the print
>> view of the notebook is functional notebook, and hence, it is editable.
>> For a print view, it would be a better options to get an non-editable
>> notebook. There was a little discussion in github around this topic
>> (https://github.com/ipython/ipython/issues/1292). Brian propose to generate
>> a static html for the print view. Then, I proposed another way, so I bring
>> the discussion here:
>>
>> We can capture the notebook into a png (and other format) images using
>> qwebkit inside PyQt (we have the last one as dependency for Qt console). In
>> this way, first, we can support another ways of export the notebook, such as
>> images...
>> Then, we can import this notebook images inside PyQt objects and displayed
>> into the javascript enviroment...
>> I wrote a little script to capture a complete image of the notebook and show
>> the image inside a web browser and it is displaying great, like the notebook
>> itself.
>>
>> Another way to have a non editable print view would be an export to a raw
>> html (https://github.com/ipython/ipython/issues/860), but we have to deal
>> with MathJax, CodeMirror and Markdown.
>>
>> What do you think about the "image" way to generate a non-editable view of
>> the notebook? I think it would be easy to implement.
>>
>> Regards.
>>
>> Dami?n.
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


Thanks for your answer Thomas...

Regarding what you have pointed out, I have also captured the notebook 
as a pdf (with qwebkit)... now I am writing the script to insert it into 
a Qwebkit object to show it properly in the browser...

I am agree with the idea that, finally, the static html is the way to do 
it... this would be a temporary solution (quick and easy to implement), 
and maybe another way to get an virtually non-modifiable (read-only) 
view of the notebook...

The PyQt dependency is not a problem for the user that install the 
complete IPython "suite", but it is a great drawback, like you said, in 
others client/servers relationships (and I presume that the notebook - 
and the project - is going to this way).

Thanks,

Dami?n.



From fperez.net at gmail.com  Fri Apr 20 19:56:25 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 16:56:25 -0700
Subject: [IPython-dev] Export notebook to image >>> not editable print
	view
In-Reply-To: <4F91ED49.2010304@gmail.com>
References: <4F91AB95.8090808@gmail.com>
	
	<4F91ED49.2010304@gmail.com>
Message-ID: 

Hi Dami?n,

On Fri, Apr 20, 2012 at 4:12 PM, Dami?n Avila  wrote:
>
> I am agree with the idea that, finally, the static html is the way to do
> it...

sorry, I meant to reply earlier but I've been very busy.  It would be
*awesome* if you, or any other dev, could help out with the 'real'
conversion code:

https://github.com/ipython/nbconvert

That's really the place where we need to solve this; once the code is
solid we can merge it into IPython proper.  I started it in a separate
repo because it began as a pycon sprint, and I didn't want the new
sprint contributors to worry about having to manage the whole IPython
repo in their first lightweight sprinting effort.

Helping with notebook conversion is right now:

- super high impact (many of us really, really need this, but I don't
have the time to finish it right now).

- easy: it can be developed in isolation and only uses a tiny fraction
of IPython, so you don't need to understand everything in IPython to
make progress here.

- incrementally rewarding: once basic export to rst/html is working,
it will be super useful, even if that export isn't perfect.  This can
be polished for a long time with lots of fanciness (as any
publication-oriented project worth its salt, it will never be 'done').
 But even basic functionality will be immensely valuable.

So if you, or anyone else, is looking to help out IPython and would
like to code on a low barrier of entry but high reward project, please
jump in!!

And incidentally, Dami?n, doing it this way completely bypasses all
the problems with Qt dependencies or image capture (on which I agree
completely with Thomas).

Cheers,

f


From damianavila at gmail.com  Fri Apr 20 22:22:36 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Fri, 20 Apr 2012 23:22:36 -0300
Subject: [IPython-dev] Export notebook to image >>> not editable print
 view
In-Reply-To: 
References: <4F91AB95.8090808@gmail.com>		<4F91ED49.2010304@gmail.com>
	
Message-ID: <4F9219EC.4070606@gmail.com>

El 20/04/12 20:56, Fernando Perez escribi?:
> Hi Dami?n,
>
> On Fri, Apr 20, 2012 at 4:12 PM, Dami?n Avila  wrote:
>> I am agree with the idea that, finally, the static html is the way to do
>> it...
> sorry, I meant to reply earlier but I've been very busy.  It would be
> *awesome* if you, or any other dev, could help out with the 'real'
> conversion code:
>
> https://github.com/ipython/nbconvert
>
> That's really the place where we need to solve this; once the code is
> solid we can merge it into IPython proper.  I started it in a separate
> repo because it began as a pycon sprint, and I didn't want the new
> sprint contributors to worry about having to manage the whole IPython
> repo in their first lightweight sprinting effort.
>
> Helping with notebook conversion is right now:
>
> - super high impact (many of us really, really need this, but I don't
> have the time to finish it right now).
>
> - easy: it can be developed in isolation and only uses a tiny fraction
> of IPython, so you don't need to understand everything in IPython to
> make progress here.
>
> - incrementally rewarding: once basic export to rst/html is working,
> it will be super useful, even if that export isn't perfect.  This can
> be polished for a long time with lots of fanciness (as any
> publication-oriented project worth its salt, it will never be 'done').
>   But even basic functionality will be immensely valuable.
>
> So if you, or anyone else, is looking to help out IPython and would
> like to code on a low barrier of entry but high reward project, please
> jump in!!
>
> And incidentally, Dami?n, doing it this way completely bypasses all
> the problems with Qt dependencies or image capture (on which I agree
> completely with Thomas).
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Thanks for your words, Fernando. .
I just throw this idea into the list because I wanted a "non-editable 
view" of the notebook for my IPython plugin, and into the plugin, the 
solution was easy implemented like I described before (doing by static 
html is a bigger project that the plugin itself, jeje!). However, I 
understand that the idea I proposed is not suitable for the IPython 
project and his needs.
But, with the discussion, always some new things arises, such as the 
repo you pointed us before (I read the first version of nbconvert from 
gist, but I did not have enough time yet to discover it into this repo, 
so thanks!
I hope I can contribute into this and/or other areas of IPython.

Cheers,

Dami?n.




From jason-sage at creativetrax.com  Fri Apr 20 22:52:52 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Fri, 20 Apr 2012 21:52:52 -0500
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the Interactive
 Editor for Python
In-Reply-To: 
References: 
Message-ID: <4F922104.90009@creativetrax.com>

Some of you probably saw this message on the scipy list.  I thought that 
the "Yoton" library they are developing sounded intriguing for the 
IPython crowd.  It seems that there are several libraries being 
developed these days to allow python-controlled remote execution kernels 
(2 such things in Sage---sage cell server and william's sage workspace 
code [1], this yoton project, and of course, your client/kernel code). 
We should at least know about each other.

http://code.google.com/p/yoton/

Thanks,

Jason

[1] https://github.com/williamstein/sagews

-------- Original Message --------
Subject: 	[SciPy-User] ANN: IEP 3.0.beta - the Interactive Editor for Python
Date: 	Fri, 20 Apr 2012 22:41:14 +0200
From: 	Almar Klein 
Reply-To: 	SciPy Users List 
To: 	SciPy Users List 



Dear all,

On behalf of Rob, Ludo and myself,? I'm pleased to finally announce
version 3.0.beta of the Interactive Editor for Python.? Give it at try
and let us know what you think!

IEP? is a cross-platform Python IDE focused on interactivity and
introspection, which makes it very suitable for scientific computing.
Its practical design is aimed at simplicity and efficiency.? IEP is
written in Python 3 and Qt. Binaries are available for Windows, Linux,
and Mac.?

Website:? http://code.google.com/p/iep/
Discussion group:? http://groups.google.com/group/iep_
Release notes:? http://code.google.com/p/iep/wiki/Release

Regards,
?  Almar?

--
Almar Klein, PhD
Science Applied
phone: +31 6 19268652 
e-mail: a.klein at science-applied.nl 
-------------- next part --------------
_______________________________________________
SciPy-User mailing list
SciPy-User at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user


From benjaminrk at gmail.com  Fri Apr 20 23:41:04 2012
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 20 Apr 2012 20:41:04 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
	
	
Message-ID: 

On Thu, Apr 19, 2012 at 17:15, Fernando Perez  wrote:

> On Thu, Apr 19, 2012 at 3:31 PM, MinRK  wrote:
> >
> > I've got all tests passing on my machines, so I expect this will be
> 0.12.1
> > unless anyone reports issues.
>
> Awesome, thanks a ton Min!  Julian, let us know if this is OK for you
> and we can officially tag things.
>

It seems to be working well for me in various situations.  Should we call
it done?

-MinRK


>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Fri Apr 20 23:46:18 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 20:46:18 -0700
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: <4F922104.90009@creativetrax.com>
References: 
	<4F922104.90009@creativetrax.com>
Message-ID: 

Hey Jason,

On Fri, Apr 20, 2012 at 7:52 PM, Jason Grout
 wrote:
> Some of you probably saw this message on the scipy list. ?I thought that the
> "Yoton" library they are developing sounded intriguing for the IPython
> crowd. ?It seems that there are several libraries being developed these days
> to allow python-controlled remote execution kernels (2 such things in
> Sage---sage cell server and william's sage workspace code [1], this yoton
> project, and of course, your client/kernel code). We should at least know
> about each other.
>
> http://code.google.com/p/yoton/

Thanks for the pointer, I saw the announcement of IEP, but not the
yoton one.  It looks extremely similar to zmq (down to the project
motto "Easiest. Messaging. Ever", for zeromq a while back I think it
read "Fastest. Messaging. Ever").  But I don't quite see the point of
reimplementing pyzmq: zeromq is already very good, has fantastic
python bindings, and there's a very good reason to want your low-level
networking machinery to be implemented in C++ instead of Python: the
GIL.

Most of ipython's communications code continues to function regardless
of the load on the python process and whatever may be happening to the
GIL, and that's a pretty darn important point (there are obviously
parts that need the gil, but a good amount happens independent of it).
 Believe, we know: we used Twisted for years, and we saw the impact
this could have for use cases like engines meant to do heavy-duty
computing.

My take on this is: IPython offers very nice abstractions for certain
workloads *on top* of zeromq, and for many problems is the right tool
to use.  For many others, pyzmq itself may be a better fit, and not
IPython, and that's fine.  But rewriting zeromq itself just so it can
have worse performance and be bound to the constraints of the GIL?  I
don't get it.

And you're right that their design brief is more or less identical to
IPython's design:

http://yoton.readthedocs.org/en/latest/examples.html#ide-kernel

This was started in September 2010, time by which we already had
pyzmq, a documented messaging protocol, a public lightweight
prototype, and we were almost finished with the production version of
the full kernel and client libraries as well as the Qt console.  I
kind of wish they'd join forces instead of starting from scratch on a
reimplementation of exactly the same thing we'd just done...
(including rewriting layers below ours, since we didn't have to write
zmq itself!).

But hey, as long as they have fun working on it, that's all that
matters.  I'm not being sarcastic here, I mean this: people should
feel free to play even with projects that seem redundant/pointless to
others, and I'm not criticizing them at all.  *I* don't see the point
for myself, but that doesn't matter.  And who knows, the ease of
development of Python may take them in interesting new directions that
zmq isn't exploring, so in the long run it may prove to be a very
useful project.

But thanks again for the pointer, I'd missed this and it's always good
to know what's out there on similar problems.

I know you keep close tabs on what we're doing (and I try my best to
know what's happening with Sage), so I trust that on the ipython/sage
front, we can make better use of resources by sharing work instead of
duplicating it ;)

Cheers,

f


From fperez.net at gmail.com  Fri Apr 20 23:47:10 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 20:47:10 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
	
	
	
Message-ID: 

On Fri, Apr 20, 2012 at 8:41 PM, MinRK  wrote:
> It seems to be working well for me in various situations. ?Should we call it
> done?

Works for me; do you want to do the tagging/release or do you want me
to do it?  I'm happy if you do :)

Cheers,
f


From fperez.net at gmail.com  Fri Apr 20 23:49:32 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 20:49:32 -0700
Subject: [IPython-dev] Export notebook to image >>> not editable print
	view
In-Reply-To: <4F9219EC.4070606@gmail.com>
References: <4F91AB95.8090808@gmail.com>
	
	<4F91ED49.2010304@gmail.com>
	
	<4F9219EC.4070606@gmail.com>
Message-ID: 

On Fri, Apr 20, 2012 at 7:22 PM, Dami?n Avila  wrote:
> But, with the discussion, always some new things arises, such as the

Indeed, and I'm very happy that you came to the list to discuss this!
Even if we end up suggesting a different route, it's precisely this
kind of exchange that gets things in motion.  Just this morning a
colleague had asked me about the status of nbconvert, and I'd promised
to him that I'd write to the list about it, hoping to gather some
energy.  But I still hadn't done anything, and your message is what
prompted me to finally move :)

> repo you pointed us before (I read the first version of nbconvert from
> gist, but I did not have enough time yet to discover it into this repo,
> so thanks!
> I hope I can contribute into this and/or other areas of IPython.

That would be fantastic.  As I said, nbconvert is high-value
low-hanging fruit, ripe for a good contributor to really help out.

Cheers,

f


From benjaminrk at gmail.com  Fri Apr 20 23:56:49 2012
From: benjaminrk at gmail.com (Min RK)
Date: Fri, 20 Apr 2012 20:56:49 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
	
	
	
	
Message-ID: <0B29AA11-B793-4998-A321-BA2DCDFD6C96@gmail.com>

If you want to take it, just so there's different set of eyes on the Final than the RCs, I might be a bit more comfortable.

-MinRK

On Apr 20, 2012, at 20:47, Fernando Perez  wrote:

> On Fri, Apr 20, 2012 at 8:41 PM, MinRK  wrote:
>> It seems to be working well for me in various situations.  Should we call it
>> done?
> 
> Works for me; do you want to do the tagging/release or do you want me
> to do it?  I'm happy if you do :)
> 
> Cheers,
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From fperez.net at gmail.com  Sat Apr 21 00:08:57 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 21:08:57 -0700
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: <0B29AA11-B793-4998-A321-BA2DCDFD6C96@gmail.com>
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
	
	
	
	
	<0B29AA11-B793-4998-A321-BA2DCDFD6C96@gmail.com>
Message-ID: 

On Fri, Apr 20, 2012 at 8:56 PM, Min RK  wrote:
> If you want to take it, just so there's different set of eyes on the Final than the RCs, I might be a bit more comfortable

No problem, I'll start now and may finish tomorrow (gotta go soon).

Many, many thanks for doing *all* the git legwork to make this
possible.  While it entailed no new code, I know you had to spend a
ton of time managing the merges, and I really appreciate it.  It will
be great to have a cleaner, more solid 0.12.1 out for users and
especially for Ubuntu LTS that at least carries the notebook out.

Cheers,

f


From fperez.net at gmail.com  Sat Apr 21 01:37:49 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 22:37:49 -0700
Subject: [IPython-dev] Test failure in master
Message-ID: 

Is anyone else seeing this?

blanca[junk]> iptest IPython.lib
...............EE....................
======================================================================
ERROR: Verify that plot is not available when pylab_import_all = False
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
line 228, in skipper_func
    return f(*args, **kwargs)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
line 225, in skipper_func
    if skip_val():
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/lib/tests/test_irunner_pylab_magic.py",
line 21, in pylab_not_importable
    import pylab
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/pylab.py",
line 1, in 
    from matplotlib.pylab import *
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/pylab.py",
line 222, in 
    from matplotlib import mpl  # pulls in most modules
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/mpl.py",
line 1, in 
    from matplotlib import artist
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/artist.py",
line 6, in 
    from transforms import Bbox, IdentityTransform, TransformedBbox,
TransformedPath
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/transforms.py",
line 47, in 
    from path import Path
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
line 19, in 
    class Path(object):
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
line 398, in Path
    _unit_regular_polygons = WeakValueDictionary()
  File "/usr/lib/python2.7/weakref.py", line 53, in __init__
    UserDict.UserDict.__init__(self, *args, **kw)
TypeError: unbound method __init__() must be called with UserDict
instance as first argument (got WeakValueDictionary instance instead)

======================================================================
ERROR: Verify that plot is available when pylab_import_all = True
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
line 228, in skipper_func
    return f(*args, **kwargs)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
line 225, in skipper_func
    if skip_val():
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/lib/tests/test_irunner_pylab_magic.py",
line 21, in pylab_not_importable
    import pylab
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/pylab.py",
line 1, in 
    from matplotlib.pylab import *
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/pylab.py",
line 222, in 
    from matplotlib import mpl  # pulls in most modules
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/mpl.py",
line 1, in 
    from matplotlib import artist
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/artist.py",
line 6, in 
    from transforms import Bbox, IdentityTransform, TransformedBbox,
TransformedPath
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/transforms.py",
line 47, in 
    from path import Path
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
line 19, in 
    class Path(object):
  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
line 398, in Path
    _unit_regular_polygons = WeakValueDictionary()
  File "/usr/lib/python2.7/weakref.py", line 53, in __init__
    UserDict.UserDict.__init__(self, *args, **kw)
TypeError: unbound method __init__() must be called with UserDict
instance as first argument (got WeakValueDictionary instance instead)

----------------------------------------------------------------------
Ran 38 tests in 2.755s


f


From jason-sage at creativetrax.com  Sat Apr 21 02:30:03 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 21 Apr 2012 01:30:03 -0500
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: 
References: 
	<4F922104.90009@creativetrax.com>
	
Message-ID: <4F9253EB.1030705@creativetrax.com>

On 4/20/12 10:46 PM, Fernando Perez wrote:
> I know you keep close tabs on what we're doing (and I try my best to
> know what's happening with Sage), so I trust that on the ipython/sage
> front, we can make better use of resources by sharing work instead of
> duplicating it ;)

In about 3 weeks (after school finishes), several students and I are 
going to investigate a lot deeper into using the new IPython 
client/kernel machinery to power the sage cell server.  I've been 
thinking about it (and several students are exploring it as well), but 
we haven't made much progress during the end-of-semester business. 
William has also been implementing his code to do remote kernel 
execution via an http channel (if I understand his commits correctly). 
It would be great if we could have a video conference between the three 
projects (IPython, William's Sage Workspace project, and our Sage Cell 
Server) to exchange ideas.  You're much further along than either of us, 
so what I see is (a) we learn from you problems you faced and overcame 
and (b) you learn from us possibly ways the IPython protocol/machinery 
could be made more general to handle our use-cases (like a "custom" 
message type with user-defined arbitrary content).

(ccing William: William, for context, the full message I'm replying to 
is here: http://mail.scipy.org/pipermail/ipython-dev/2012-April/009021.html)

Thanks,

Jason


From benjaminrk at gmail.com  Sat Apr 21 02:34:08 2012
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 20 Apr 2012 23:34:08 -0700
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
Message-ID: 

Yes, there's even an Issue
for it.  It's
deepreload's fault somehow.  Perhaps deep reload tests should
be handled in a subprocess, so they are isolated.


On Fri, Apr 20, 2012 at 22:37, Fernando Perez  wrote:

> Is anyone else seeing this?
>
> blanca[junk]> iptest IPython.lib
> ...............EE....................
> ======================================================================
> ERROR: Verify that plot is not available when pylab_import_all = False
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>  File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
> line 228, in skipper_func
>    return f(*args, **kwargs)
>  File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
> line 225, in skipper_func
>    if skip_val():
>  File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/lib/tests/test_irunner_pylab_magic.py",
> line 21, in pylab_not_importable
>    import pylab
>  File "/home/fperez/usr/opt/lib/python2.7/site-packages/pylab.py",
> line 1, in 
>    from matplotlib.pylab import *
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/pylab.py",
> line 222, in 
>    from matplotlib import mpl  # pulls in most modules
>  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/mpl.py",
> line 1, in 
>    from matplotlib import artist
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/artist.py",
> line 6, in 
>    from transforms import Bbox, IdentityTransform, TransformedBbox,
> TransformedPath
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/transforms.py",
> line 47, in 
>    from path import Path
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
> line 19, in 
>    class Path(object):
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
> line 398, in Path
>    _unit_regular_polygons = WeakValueDictionary()
>  File "/usr/lib/python2.7/weakref.py", line 53, in __init__
>    UserDict.UserDict.__init__(self, *args, **kw)
> TypeError: unbound method __init__() must be called with UserDict
> instance as first argument (got WeakValueDictionary instance instead)
>
> ======================================================================
> ERROR: Verify that plot is available when pylab_import_all = True
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>  File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
> line 228, in skipper_func
>    return f(*args, **kwargs)
>  File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/decorators.py",
> line 225, in skipper_func
>    if skip_val():
>  File
> "/home/fperez/usr/lib/python2.7/site-packages/IPython/lib/tests/test_irunner_pylab_magic.py",
> line 21, in pylab_not_importable
>    import pylab
>  File "/home/fperez/usr/opt/lib/python2.7/site-packages/pylab.py",
> line 1, in 
>    from matplotlib.pylab import *
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/pylab.py",
> line 222, in 
>    from matplotlib import mpl  # pulls in most modules
>  File "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/mpl.py",
> line 1, in 
>    from matplotlib import artist
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/artist.py",
> line 6, in 
>    from transforms import Bbox, IdentityTransform, TransformedBbox,
> TransformedPath
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/transforms.py",
> line 47, in 
>    from path import Path
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
> line 19, in 
>    class Path(object):
>  File
> "/home/fperez/usr/opt/lib/python2.7/site-packages/matplotlib/path.py",
> line 398, in Path
>    _unit_regular_polygons = WeakValueDictionary()
>  File "/usr/lib/python2.7/weakref.py", line 53, in __init__
>    UserDict.UserDict.__init__(self, *args, **kw)
> TypeError: unbound method __init__() must be called with UserDict
> instance as first argument (got WeakValueDictionary instance instead)
>
> ----------------------------------------------------------------------
> Ran 38 tests in 2.755s
>
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Sat Apr 21 02:38:12 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 23:38:12 -0700
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: <4F9253EB.1030705@creativetrax.com>
References: 
	<4F922104.90009@creativetrax.com>
	
	<4F9253EB.1030705@creativetrax.com>
Message-ID: 

On Fri, Apr 20, 2012 at 11:30 PM, Jason Grout
 wrote:
> It would be great if we could have a video conference between the three
> projects (IPython, William's Sage Workspace project, and our Sage Cell
> Server) to exchange ideas. ?You're much further along than either of us,
> so what I see is (a) we learn from you problems you faced and overcame
> and (b) you learn from us possibly ways the IPython protocol/machinery
> could be made more general to handle our use-cases (like a "custom"
> message type with user-defined arbitrary content).

Certainly!  Let's talk when the time gets closer and we can set up a
call, we'd be happy to discuss this.

Cheers,

f


From fperez.net at gmail.com  Sat Apr 21 02:46:17 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 20 Apr 2012 23:46:17 -0700
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
	
Message-ID: 

On Fri, Apr 20, 2012 at 11:34 PM, MinRK  wrote:
> Yes, there's even an Issue for it. ?It's deepreload's fault somehow.
> ?Perhaps deep reload tests should be handled in a subprocess, so they are
> isolated.

Ah, thanks!  Had forgotten that one.  Yes, we should move those to
their own group.

Cheers,

f


From a.klein at science-applied.nl  Sat Apr 21 05:31:01 2012
From: a.klein at science-applied.nl (Almar Klein)
Date: Sat, 21 Apr 2012 11:31:01 +0200
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: 
References: 
	<4F922104.90009@creativetrax.com>
	
Message-ID: 

Hi IPython developers,

Let me explain about our choice to implement Yoton instead of using pyzmq.


Thanks for the pointer, I saw the announcement of IEP, but not the
> yoton one.  It looks extremely similar to zmq (down to the project
> motto "Easiest. Messaging. Ever", for zeromq a while back I think it
> read "Fastest. Messaging. Ever").


I definitely had ZMQ inspire me for the design. The motto is indeed
shamelessly stolen from zmq's motton; it should not be taken too seriously,
but more as a wink to the zmq project ;)



> But I don't quite see the point of
> reimplementing pyzmq: zeromq is already very good, has fantastic
> python bindings, and there's a very good reason to want your low-level
> networking machinery to be implemented in C++ instead of Python: the
> GIL.
>

The main reason for developing a pure python communication layer was
because I wanted IEP to work with any Python version installed on a system
(Python v2.4 and up), without having to install any extra packages. The
idea is that you can just install the binary version of IEP and get
started.

I already had working code for this (you recalled our discussion from
september 2010) when I learned about zmq and pyzmq. I have seen the
documentation of IPython for the communication between the kernel and
client, and although it all made a lot of sense, I foresaw some
incompatibilities with our ideas about the relation between the kernel and
the client. I cannot recall the details, but I believe the main difference
is that IPython considers the kernel to be "in charge", while IEP considers
the kernel to be a "slave".

Considering that I already had working code, I decided to refactor it to
make the code more clear, and introducing functionality to connect multiple
context in a network: I really like IPython's idea about remote and
collaborative computing. As always, I underestimated the work that was
needed to get it right. In this case I underestimated it *a lot*. But
anyway, this resulted in Yoton.



> Most of ipython's communications code continues to function regardless
> of the load on the python process and whatever may be happening to the
> GIL, and that's a pretty darn important point (there are obviously
> parts that need the gil, but a good amount happens independent of it).
>  Believe, we know: we used Twisted for years, and we saw the impact
> this could have for use cases like engines meant to do heavy-duty
> computing.
>

Indeed, if the kernel is running extension code, the kernel and IDE cannot
communicate. But we can still distinguish between the kernel being dead and
the kernel running extension code. If we connect via localhost, the
socket-pair always remains intact unless the process dies (or the socket is
closed explicitly of course). So if we do not receive heart beat signals,
but we do not detect a socket problem, we know that the process is running
extension code (this is also visualised in the shell-icon in IEP). This
does mean we need a connection via localhost, so we always use a broker
that runs on the same machine as the kernel. This has other advantages; we
also read the kernel's stdout and stderr via a pipe and send these messages
to the IDE via Yoton.

Certainly, zmq has a better performance, but Yoton easily matches the
requirements needed for kernel-ide communication.


But hey, as long as they have fun working on it, that's all that
> matters.  I'm not being sarcastic here, I mean this: people should
> feel free to play even with projects that seem redundant/pointless to
> others, and I'm not criticizing them at all.  *I* don't see the point
> for myself, but that doesn't matter.  And who knows, the ease of
> development of Python may take them in interesting new directions that
> zmq isn't exploring, so in the long run it may prove to be a very
> useful project.
>

Thanks for your understanding :)  Yes we did have a lot of fun. And a lot
of pain and frustration, but we learned a lot in the end.

One more thing that might be of interest. Yoton has a feature for the
pub-sub pattern to slow down the sending side if the receiver cannot keep
up processing messages. Therefore we can guarantee that the user will not
miss any messages. Try running "while True: print(time.time())".

Regards,
  Almar

-- 
Almar Klein, PhD
Science Applied
phone: +31 6 19268652
e-mail: a.klein at science-applied.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jtaylor.debian at googlemail.com  Sat Apr 21 09:21:21 2012
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Sat, 21 Apr 2012 15:21:21 +0200
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
Message-ID: <4F92B451.2070002@googlemail.com>

On 04/21/2012 07:37 AM, Fernando Perez wrote:
> Is anyone else seeing this?

the ubuntu daily build runs the test with the almost full dependencies
(only no pymongodb daemon) so it sees this failure too:
https://code.launchpad.net/~jtaylor/+recipe/ipython-daily

I wasn't aware your shining panda instance does not have all
dependencies so I did not report it. I'll check that in future.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From jtaylor.debian at googlemail.com  Sat Apr 21 09:27:42 2012
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Sat, 21 Apr 2012 15:27:42 +0200
Subject: [IPython-dev] status of 0.12.1?
In-Reply-To: 
References: 
	
	<4F8CA9EE.5080807@googlemail.com>
	
	<489FAD95-F772-4CA5-A9A0-C21DF2865394@gmail.com>
	
	
	
	
	
	
	
	
	<0B29AA11-B793-4998-A321-BA2DCDFD6C96@gmail.com>
	
Message-ID: <4F92B5CE.1040909@googlemail.com>

On 04/21/2012 06:08 AM, Fernando Perez wrote:
> On Fri, Apr 20, 2012 at 8:56 PM, Min RK  wrote:
>> If you want to take it, just so there's different set of eyes on the Final than the RCs, I might be a bit more comfortable
> 
> No problem, I'll start now and may finish tomorrow (gotta go soon).
> 
> Many, many thanks for doing *all* the git legwork to make this
> possible.  While it entailed no new code, I know you had to spend a
> ton of time managing the merges, and I really appreciate it.  It will
> be great to have a cleaner, more solid 0.12.1 out for users and
> especially for Ubuntu LTS that at least carries the notebook out.
> 

I also highly appreciate the work you did for this release.
It will make the default ipython experience for Ubuntu users much
smoother and greatly simplifies support for the package.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From jason-sage at creativetrax.com  Sat Apr 21 12:50:54 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Sat, 21 Apr 2012 11:50:54 -0500
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: 
References: 
	<4F922104.90009@creativetrax.com>
	
	
Message-ID: <4F92E56E.70204@creativetrax.com>

On 4/21/12 4:31 AM, Almar Klein wrote:
> I believe the main difference is that IPython considers the kernel to be
> "in charge", while IEP considers the kernel to be a "slave".

Can you elaborate on this distinction?

Also, as a pure-python library, I'm also curious how easy it would be to 
port parts of yoton to Google App Engine (for instance, to run a kernel 
on GAE).  You wouldn't have a BSD socket, but you could substitute in a 
wrapper around the GAE Channel class, I think (or at least hope).

In order to get IPython messages to go through a GAE instance out to a 
web client, it seems that we need to write a zmq<->GAE channel bridge 
(similar to the zmq<->websockets bridge in the IPython notebook [1], IIRC).

Thanks,

Jason

[1] 
https://github.com/ipython/ipython/blob/master/IPython/frontend/html/notebook/handlers.py#L367, 
see also 
https://github.com/ipython/ipython/blob/master/IPython/frontend/html/notebook/zmqhttp.py


From fperez.net at gmail.com  Sat Apr 21 13:39:11 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 21 Apr 2012 10:39:11 -0700
Subject: [IPython-dev] Test failure in master
In-Reply-To: <4F92B451.2070002@googlemail.com>
References: 
	<4F92B451.2070002@googlemail.com>
Message-ID: 

On Sat, Apr 21, 2012 at 6:21 AM, Julian Taylor
 wrote:
> the ubuntu daily build runs the test with the almost full dependencies
> (only no pymongodb daemon) so it sees this failure too:
> https://code.launchpad.net/~jtaylor/+recipe/ipython-daily
>
> I wasn't aware your shining panda instance does not have all
> dependencies so I did not report it. I'll check that in future.

Ah, great!  Yes, please do, it will be great to have these extra eyes.
 I wonder: would it be possible to have your recipe submit its results
to shining panda before being destroyed?  That way they'd show up
automatically in the summary report

Cheers,

f


From a.klein at science-applied.nl  Sat Apr 21 17:40:12 2012
From: a.klein at science-applied.nl (Almar Klein)
Date: Sat, 21 Apr 2012 23:40:12 +0200
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: <4F92E56E.70204@creativetrax.com>
References: 
	<4F922104.90009@creativetrax.com>
	
	
	<4F92E56E.70204@creativetrax.com>
Message-ID: 

>
> > I believe the main difference is that IPython considers the kernel to be
> > "in charge", while IEP considers the kernel to be a "slave".
>
> Can you elaborate on this distinction?
>

It's been a while since I looked at some code or docs of IPython, so please
correct me if I'm wrong. IPython, being historically only an interpreter,
is very kernel-centric; most of the smartness is in the kernel, and the
"clients" are relatively thin. IEP comes from the other direction; most of
the smartness is in the IDE, and the kernel is made to be as small and
light as possible.

One example is that the IEP kernel does not know about autocompletion. The
kernel has an RPC component that on demand provides the IDE with a list of
names in a certain namespace, and the IDE uses that to provide
autocompletion to the user.

Although IEP initially also did the magic commands on the IDE side, we know
think it's better to do these in the kernel, so our kernel is getting
"smarter" :)  But for the rest we try to keep the kernel stupid, and have
it provide the IDE with all the information to do the smart stuff. Anyway,
that's the idea.



> Also, as a pure-python library, I'm also curious how easy it would be to
> port parts of yoton to Google App Engine (for instance, to run a kernel
> on GAE).  You wouldn't have a BSD socket, but you could substitute in a
> wrapper around the GAE Channel class, I think (or at least hope).
>

I've never done anything using Google Apps Engine, so I'm not sure how
difficult it would be to deploy yoton on it. As for the connection, Yoton
implements an abstract class
Connection
that
represents the thing that connects two contexts. Currently the
TcpConnection (which wraps a BSD socket) is the only subclass. In principal
it should be possible to wrap any kind of connection in it.

Regards,
  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Sun Apr 22 02:06:41 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 21 Apr 2012 23:06:41 -0700
Subject: [IPython-dev] [ANN] IPython 0.12.1 is out
Message-ID: 

Hi folks,

I'm happy to announce, on behalf of the whole team, the release of the
bugfix-only IPython 0.12.1 as our new stable version.  This release
consists strictly of commits from the main development branch that
were back-ported over the 0.12 release, so no new code has been
written for this release that is not in the current git master (aside
from a few minor last-minute details that will be shortly merged).

Over 70 separate issues were closed (the final count is likely higher,
as we auto-generated the summary).  The full list of these can be
found here:

http://ipython.org/ipython-doc/stable/whatsnew/github-stats-0.12.html

You can find downloads either on our page or on PyPI:

http://ipython.org/download.html
pypi.python.org/pypi/ipython

I'd like to thank everyone whose commits we merged for this release
(please let me know if we omitted your name, this list is
auto-generated from the git logs):

* anatoly techtonik
* Andr? Matos
* Andrew Giessel
* Bradley M. Froehle
* Fernando Perez
* Matthew Brett
* MinRK
* Paul Ivanov
* Puneeth Chaganti
* Robert Kern
* Thomas Kluyver
* Thomas Spura
* Walter Doerwald
* wilsaj

and especially Min who did all the detailed work of cherry-picking the
~ 125 commits that went into this release.

As usual, please report any problems on our mailing list or our issue tracker.

Cheers,

f


From fperez.net at gmail.com  Sun Apr 22 03:19:37 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 00:19:37 -0700
Subject: [IPython-dev] Back to focus on the road to 0.13
Message-ID: 

Hi all,

ouch, releases always manage to eat up a ton of time; between
distutils and testing for windows, I ended up burning a solid ~10
hours getting 0.12.1 out (and that's after Min had done all the 'real'
work of cherry-picking commits).  Oh well, having a cleaner 0.12.x in
ubuntu LTS should be worth it.

But now that 0.12.1 is out, we can focus on what needs to happen for
0.13.  I'm going to start implementing the cell-level magics this
week, using '%%' as the sigil (I'd forgotten that I wanted to keep '$'
as an *explicit* sigil for aliases, which currently can't be
disambiguated because they have no sigil; besides %% was winning from
other arguments).  We have a healthy list of PRs pending, and also
currently 8 issues marked as blockers:

https://github.com/ipython/ipython/issues?labels=prio-blocker&milestone=7&page=1&state=open

and 13 more high-priority ones:

https://github.com/ipython/ipython/issues?labels=prio-high&milestone=7&page=1&state=open

We'd love to have a hand with either reviewing open PRs, or submitting
code against these high/blocker issues!

Cheers,

f


From takowl at gmail.com  Sun Apr 22 08:05:04 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 22 Apr 2012 13:05:04 +0100
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
	<4F92B451.2070002@googlemail.com>
	
Message-ID: 

On 21 April 2012 18:39, Fernando Perez  wrote:
> Ah, great! ?Yes, please do, it will be great to have these extra eyes.
> ?I wonder: would it be possible to have your recipe submit its results
> to shining panda before being destroyed? ?That way they'd show up
> automatically in the summary report

If we did this the same way the Mac & Windows buildbots do it, the
build process would need to upload a zip file to a publically
accessible space (such as the ipython.org server), then ping a URL on
Shiningpanda to download the results. I don't know if the launchpad
builders are allowed to make arbitrary HTTP connections, though - it
might be limited to installing build dependencies from the
repositories.

Thomas


From jtaylor.debian at googlemail.com  Sun Apr 22 08:18:27 2012
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Sun, 22 Apr 2012 14:18:27 +0200
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
	<4F92B451.2070002@googlemail.com>
	
	
Message-ID: <4F93F713.1050901@googlemail.com>

On 04/22/2012 02:05 PM, Thomas Kluyver wrote:
> On 21 April 2012 18:39, Fernando Perez  wrote:
>> Ah, great!  Yes, please do, it will be great to have these extra eyes.
>>  I wonder: would it be possible to have your recipe submit its results
>> to shining panda before being destroyed?  That way they'd show up
>> automatically in the summary report
> 
> If we did this the same way the Mac & Windows buildbots do it, the
> build process would need to upload a zip file to a publically
> accessible space (such as the ipython.org server), then ping a URL on
> Shiningpanda to download the results. I don't know if the launchpad
> builders are allowed to make arbitrary HTTP connections, though - it
> might be limited to installing build dependencies from the
> repositories.
> 
> Thomas

internet connections are not allowed during the build so this can not be
done directly.
One could add the test results to a packages and let shinning-panda
download and extract the package (only ar and tar need to be installed
for that)
If you deem it worth it could modify the recipe to do that.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From tomspur at fedoraproject.org  Sun Apr 22 12:21:30 2012
From: tomspur at fedoraproject.org (Thomas Spura)
Date: Sun, 22 Apr 2012 18:21:30 +0200
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
	<4F92B451.2070002@googlemail.com>
	
	
Message-ID: 

On Sun, Apr 22, 2012 at 2:05 PM, Thomas Kluyver  wrote:
> On 21 April 2012 18:39, Fernando Perez  wrote:
>> Ah, great! ?Yes, please do, it will be great to have these extra eyes.
>> ?I wonder: would it be possible to have your recipe submit its results
>> to shining panda before being destroyed? ?That way they'd show up
>> automatically in the summary report
>
> If we did this the same way the Mac & Windows buildbots do it, the
> build process would need to upload a zip file to a publically
> accessible space (such as the ipython.org server), then ping a URL on
> Shiningpanda to download the results. I don't know if the launchpad
> builders are allowed to make arbitrary HTTP connections, though - it
> might be limited to installing build dependencies from the
> repositories.

How exactly does that work?
Maybe the test results can also be build with an script out of the
build logs and then get zipped for shiningpanda?
Didn't found docs about that right now...

There is no way to access the internet in the Fedora build system either...

Greetings,
    Tom


From fperez.net at gmail.com  Sun Apr 22 16:50:53 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 13:50:53 -0700
Subject: [IPython-dev] Fwd: [SciPy-User] ANN: IEP 3.0.beta - the
 Interactive Editor for Python
In-Reply-To: 
References: 
	<4F922104.90009@creativetrax.com>
	
	
	<4F92E56E.70204@creativetrax.com>
	
Message-ID: 

Hi Almar,

On Sat, Apr 21, 2012 at 2:40 PM, Almar Klein  wrote:

> It's been a while since I looked at some code or docs of IPython, so please
> correct me if I'm wrong. IPython, being historically only an interpreter, is
> very kernel-centric; most of the smartness is in the kernel, and the
> "clients" are relatively thin. IEP comes from the other direction; most of
> the smartness is in the IDE, and the kernel is made to be as small and light
> as possible.

Well, it's not so much that the clients are thin (both the Qt console
and the notebook are fairly complex in their own right), it's rather
that the clients can be very *different* and yet they can all work
with the same kernel (in fact, they can do it simultaneously).

For this to work, there has to be a very strong separation between
what's the job of the kernel and what's the job of the clients, and it
does mean that we put a ton of the functionality in the kernels
themselves.  For us, the kernel does everything related to code
execution and managing the namespace itself, while the clients expose
a *workflow* against that kernel.  That workflow may be quite
different for each client, but they all share many features thanks to
those features residing in the kernel.

I think your intuition was correct, I just want to clarify our perspective.

And thanks for the details on your design, it's always useful to
compare takes on a similar problem.  And additionally, while yoton
being in pure python will impose some limitations (performance, GIL),
it also makes it very appealing for easy deployment in scenarios such
as GAE, as Jason points out.

Cheers,

f


From takowl at gmail.com  Sun Apr 22 18:36:57 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 22 Apr 2012 23:36:57 +0100
Subject: [IPython-dev] Test failure in master
In-Reply-To: <4F93F713.1050901@googlemail.com>
References: 
	<4F92B451.2070002@googlemail.com>
	
	
	<4F93F713.1050901@googlemail.com>
Message-ID: 

On 22 April 2012 13:18, Julian Taylor  wrote:
> One could add the test results to a packages and let shinning-panda
> download and extract the package (only ar and tar need to be installed
> for that)
> If you deem it worth it could modify the recipe to do that.

It's not essential - we already have the tests run in a Debian based
environment on Shiningpanda, and separately on a Mac with most of the
dependencies installed (which did pick up this failure). But if it's
easy to do, more tests never hurt.

Thomas S:
> How exactly does that work?

There's a de facto standard XML format for test results, based on the
JUnit tool for Java. A nose plugin (Xunit) can write those files, and
we can make Shiningpanda download them and parse them as the results
of a test run.

> Maybe the test results can also be build with an script out of the build logs and then get zipped for shiningpanda?

The tool we use at present constructs the results as the tests run. I
imagine it would be possible to parse the build logs to create the
XML, but it's probably not worth the effort of writing a tool to do so
unless we feel really limited by our other test methods.

Thanks,
Thomas


From fperez.net at gmail.com  Sun Apr 22 19:18:57 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 16:18:57 -0700
Subject: [IPython-dev] Welcoming Matthias Bussonier (and belatedly,
	Paul Ivanov) to the core team
Message-ID: 

Hi all,

I'd like to welcome Matthias (@Carreau), as well as belatedly catch up
with doing the same for Paul (@ivanov), to the core team.  Matthias
and Paul have been regularly very active participants in the project,
consistently producing pull requests that ended up being merged,
closing bugs, etc.  We felt it was time to ask them to join the core
team.

In practice core members and everyone else on IPython really follow
very similar worfklows, since we still use the pull request mechanism
for review of everyone's work, and we almost never commit directly to
the main repo.  This means that *in practice*, not much changes for
core members, other than the ability to merge pull requests that have
been well reviewed.

But we hope that by being a member of the core team, contributors will
be encouraged to participate even further, and to get more involved
with the project not only by contributing their own code (which we
greatly appreciate!), but also by reviewing the pull requests made by
others, participating in design discussions, etc.  We have a small
team, so every new member who brings enthusiasm and energy can really
make a difference.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 00:06:20 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:06:20 -0700
Subject: [IPython-dev] clear_output() of only last line?
In-Reply-To: <4F85B4A5.4030001@stsci.edu>
References: <4F85B4A5.4030001@stsci.edu>
Message-ID: 

Hi Mike,

On Wed, Apr 11, 2012 at 9:43 AM, Michael Droettboom  wrote:
> I'm working on getting our animated progressbars in astropy to work in the
> IPython notebook.? I found the recipe of using "clear_output()" in this bug
> report:
>
> https://github.com/ipython/ipython/issues/1527
>
> That gets things closer, but unlike "\r" at a standard terminal,
> clear_output() clears the entire output chunk, not just the last line.? So
> if we have multiple progress bars, or progress bars interspersed with
> logging messages, all but the last progress bar gets deleted.
>
> For example:
>
> from IPython.core.display import clear_output
>
> print "Working"
> for i in range(80):
> ??? clear_output()
> ??? print "\r" + ("=" * i)
>
> Any suggestions?

Have you had a chance to look at the new Animations_and_Progress
notebook in master?  It shows examples of how to make both text-based
and graphical progressbars in the nb.

I hope that helps, and sorry for the delayed reply!

Best,

f


From fperez.net at gmail.com  Mon Apr 23 00:20:49 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:20:49 -0700
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
Message-ID: 

Hi Thomas,

thanks for bringing this up...  I just haven't been able to keep up :)

On Wed, Apr 11, 2012 at 7:48 AM, Thomas Kluyver  wrote:
> Pierre Haessig has spotted inconsistencies in the effect of a
> statement like 'from __future__ import division'. If it's in a startup
> file with a .py extension, it's local to that file; if the same file
> has a .ipy extension, it affects the interactive environment. It
> occurs to me that we haven't given this proper consideration. There
> are a lot of places that we execute external code:
>
> * At startup: `exec_lines`, `exec_files`, startup files. There are .py
> and .ipy files, and they should definitely behave the same way with
> future statements.
> * `%run` (and `%run -i`), `%rerun`, `%edit`, macros, `%loadpy`
> * Extensions
> * Code loaded by the autoreload extension
>
> My gut feeling is that of those, `exec_lines`, `%loadpy`, macros and
> maybe `%run -i` should 'leak' future statements to the interactive
> environment. Everything else should keep future statements to itself
> (except by using `run_cell()` to run them through IPython). But I'm
> asking here to get other opinions.

- loadpy doesn't really do anything special regarding execution, it
only *reads* the data.  Once loaded, the cell will be executed just as
if it had been typed interactively.  So loadpy doesn't belong in this
discussion.

- run: yes, if a __future__ is imported in a file that was %run, then
it should become active interactively.  The reason is to keep the
semantics of operating with things defined in that script consistent.
Say a script had `z=x/y` in it, you want to be able to later type at
the command-line `x/y` and get the same division semantics that were
set inside the script.

- the fact that .ipy files behave different from .py ones is a bug in
my view.  Those should be executed like modules, without leaking
__future__ imports.

- if users want a __future__ import  active in their startup sequence,
they should put it in `exec_lines` (or they can achieve the same
result by manually duing get_ipython().run_cell('from
__future__...')`.  I think it's OK that this is a special case, b/c
__future__ imports behave specially in Python itself.

In summary, it seems to me that we have:

- one small bug: the mismatch between .ipy and .py behavior.

- one small enahcement request: for %run to be executed in a way that
propagates any __future__ statements back to the interactive
environment. I'd note that if actually doing this complicates run in
any major way, we should simply drop the idea and document the
behavior in a special note.

- a visible note in the config section of the docs explaining that the
right way to set __future__ imports is via the `exec_lines` variable.

How does this sound?

f


From fperez.net at gmail.com  Mon Apr 23 00:26:23 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:26:23 -0700
Subject: [IPython-dev] Embedding IPython+zmq
In-Reply-To: <808F6AA8-5CD4-452C-937C-D907276710C8@anl.gov>
References: <4F86EE45.2040909@creativetrax.com>
	<808F6AA8-5CD4-452C-937C-D907276710C8@anl.gov>
Message-ID: 

Hi Ray,

On Thu, Apr 12, 2012 at 9:44 AM, Ray Osborn  wrote:
> I didn't want to hijack the other thread so I thought I would create a new one. I just read the page that Jason Grout references about making the two-process iPython standard. I have, I think successfully, adapted Robert Kern's method of creating a kernel that fakes zmq messages within a single process, so that I can embed the iPython shell in a Qt GUI. I need this because I want to display items in the iPython shell namespace in a separate widget and need to trigger Qt signals when certain items in the namespace are changed. As Robert admits, this requires hacking into the zmq code and is probably not as elegant as it could be.
>
> Are there plans to refactor the two-process code to enable a single-process mode easily? Or perhaps zmq messaging could be used to facilitate communication to external GUI widgets, such as providing a way of mapping messages to signals? This is not my area of expertise, so let me know if I have not explained the problem clearly enough.

You have, and indeed I would *love* to have the time for a clean
refactoring of the various kernel/client classes to make it really
trivial to switch from in-process to out-of-process.  In my mind, all
clients should have simply a .kernel attribute that could be either a
LocalKernel or a RemoteKernel, with otherwise identical APIs.  Then,
changing the Qt console from using a remote to a local kernel would be
a trivial one-liner.

We've done bits and pieces of that, but nobody so far has had the
bandwidth to spend on doing this entire refactoring cleanly.
Ultimately that could even produce a notebook that runs without zeromq
(albeit opening only one .ipynb per http port)...

So in summary: yes, we want to do it.  No, it's not on our immediate
work list due to lack of time/higher priorities.

If anyone is interested in doing this right, we'd be happy to review
relevant pull requests, though.

Until then, hacks like what you're using will be the solution, I'm afraid.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 00:36:44 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:36:44 -0700
Subject: [IPython-dev] traitlets question [from Re: msg spec]
In-Reply-To: 
References: 
Message-ID: 

Hey Satra,

sorry for the delayed reply...

On Tue, Apr 17, 2012 at 7:27 PM, Satrajit Ghosh  wrote:
>
>> > I've been using traitlets to do the message validation, and I'm running
>> > into
>> > the fact that we are unclear about what keys can be None and what
>> > cannot.
>> > ?Should I start with the assumption that *anything* can be None, or not?
>>
>> I think that's a reasonable starting point. Once the tests are in
>> solid shape we can revisit this and go over the spec/implementation
>> more carefully.
>
>
> question out of not following the traitlets code. a long while back it was
> deemed that traitlets needed to have every element defined. is that no
> longer necessary? i.e could a trait be undefined (or set to None).

Dunno, sorry.  Brian?

> and to follow up, does traitlets have event delegation?

They have basic static and dynamic _foo_changed and on_trait_change
event handler like enthought.Traits do, but I think that Traits
support a more complex delegation pattern that we don't.

Brian or Robert will correct me if the above is mistaken...

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 00:41:32 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:41:32 -0700
Subject: [IPython-dev] zmq when using ipython
In-Reply-To: <4F87BD0C.3010901@creativetrax.com>
References: <4F86E525.1090801@creativetrax.com>
	
	<4F86EAFA.9010103@creativetrax.com>
	
	<4F87BD0C.3010901@creativetrax.com>
Message-ID: 

Hi Jason,

On Thu, Apr 12, 2012 at 10:43 PM, Jason Grout
 wrote:
> Last year, we talked about wrapping such user messages so that the
> message the client would actually see this:
>
> {"header": {...},
> "parent_header": {...},
> "msg_type": "extension",
> "content": user_message, # message above
> }
>
> Is there a way to send such messages from userspace code?

I don't think we ever implemented the extension message type, but I'm
+1 on doing so.  I would suggest that our objects would implement a
generic extension message handler that simply logs them by default.
But users could provide their own extension handler function, which
we'd then call instead of the default.

How does this sound?

f


From fperez.net at gmail.com  Mon Apr 23 00:43:32 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:43:32 -0700
Subject: [IPython-dev] IdleX adds IPython support
In-Reply-To: 
References: 
	
	
Message-ID: 

On Sun, Mar 11, 2012 at 10:56 AM, Satrajit Ghosh  wrote:
> i would also add pycharm having support for 0.12. again i don't know the
> full extent of support, but i can pull up an ipython 0.12 console from
> within pycharm.

That's good to hear!  I tried using pycharm after seeing your
screencaps with it, but unfortunately under linux Java font rendering
is bad to the point of being intolerable.  That, and the fact that it
needed about 10 minutes to open my IPython directory, made me end up
not really going much further with the tool.  But it seems that on
OSX/Windows it's a very good tool, from what I've seen in demos.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 00:45:10 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:45:10 -0700
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
Message-ID: 

On Sat, Mar 10, 2012 at 4:52 PM, Satrajit Ghosh  wrote:
>
> the culprit was node, so instead of sending it directly, i pickled it and
> sent it. and unpickled it inside the function ?after i had done any of the
> configuring that i needed to do.

Why not define another task that does the prep you need, and declare
it as a dependency of this task?  Would that not work?

Best,

f


From cschin at infoecho.net  Mon Apr 23 00:51:00 2012
From: cschin at infoecho.net (Chen-Shan Chin)
Date: Sun, 22 Apr 2012 21:51:00 -0700
Subject: [IPython-dev] zmq when using ipython
In-Reply-To: <4F87BD0C.3010901@creativetrax.com>
References: <4F86E525.1090801@creativetrax.com>
	
	<4F86EAFA.9010103@creativetrax.com>
	
	<4F87BD0C.3010901@creativetrax.com>
Message-ID: <01428207-8D9B-4093-9C04-ED9FA2AE17F4@infoecho.net>


Hi, Fernando:

I really like the extension idea.  This is probably also what I was looking for when I try to hack ipython to be able to take message that is not initialized from a normal input cell (e.g. a python code is associated some customized UI element sent by javascript...)

--Jason C.

On Apr 12, 2012, at 10:43 PM, Jason Grout wrote:

> On 4/12/12 10:48 PM, Fernando Perez wrote:
>> On Thu, Apr 12, 2012 at 7:47 AM, Jason Grout
>>   wrote:
>>> Thanks, this corrects a big misunderstanding I had (and maybe is also an
>>> answer to Volker's post).  I thought *everything* in the new ipython
>>> depended on the messaging protocol and connections with zmq.
>> 
>> The biggest change for Sage is going to be that customizing IPython
>> now uses a richer but also more complex configuration system:
>> 
>> http://ipython.org/ipython-doc/dev/config/index.html
>> 
>> It appears a bit daunting at first, and it does have a few rough
>> edges, but by and large we've found that it works very well in letting
>> us configure all the many moving parts IPython now has while using the
>> full power of the Python language (since the config files are regular
>> .py).
>> 
>> It's *possible* that we've removed/forgotten something that Sage used
>> in its command-line client.  If that's the case let us know, as we
>> didn't intend to remove any of the features that made the kind of
>> aggressive customization Sage does impossible.  Such a change would be
>> just an accidental oversight, so just report it and we'll do our best
>> to restore any missing functionality.
>> 
> 
> Mike Hansen posted a working patch for the latest Sage build: 
> http://trac.sagemath.org/sage_trac/ticket/12719
> 
> I've been using it for about a day now, and haven't noticed any real 
> problems.
> 
> I'm also interested in the forking issue.  With the Sage Cell Server 
> [1], we end up forking a running Sage instance to create a new worker 
> process (i.e., kernel).  We're still thinking through the design and how 
> we can best take advantage of the new IPython functionality, so I'll 
> probably have more coherent questions later.  However, one feature we'd 
> love to see is allowing user code to send messages in the same channel 
> as normal output messages.  For example, I'd like to send the following 
> message, which our custom IPython client already interprets to make a 
> Sage interact with javascript controls:
> 
> {"msg_type": "interact_prepare",
>  "content": {"interact_id": "2096051932116277084",
>              "layout": {"top_center": ["n"]},
>              "update": {"n": ["n"]},
>              "controls": {"n": {"control_type": "slider", "raw": true, 
> "default": 0.0, "step": 0.004, "label": null, "subtype": "continuous", 
> "range": [0.0, 1.0], "display_value": true}}
>             }
> }
> 
> Last year, we talked about wrapping such user messages so that the 
> message the client would actually see this:
> 
> {"header": {...},
> "parent_header": {...},
> "msg_type": "extension",
> "content": user_message, # message above
> }
> 
> Is there a way to send such messages from userspace code?
> 
> Thanks,
> 
> Jason
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From fperez.net at gmail.com  Mon Apr 23 00:53:28 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:53:28 -0700
Subject: [IPython-dev] ShiningPanda
In-Reply-To: 
References: 
	
	
	
	
	
	
Message-ID: 

On Sun, Jan 29, 2012 at 3:00 PM, Aaron Meurer  wrote:
> I'd like to clarify something about the SymPy review bot.

Aaron, I just wanted to thank you for providing us with the details of
the sympy review bot, it looks like a really fantastic system.  And
I'd *love* to set IPython up to use something similar, it would be
great to come to a PR and already know that it passes the test suite
(especially important if it's a PR that addresses a platform-specific
issue on say Windows or OSX, which not all developers have convenient
access to).

best,

f


From fperez.net at gmail.com  Mon Apr 23 00:56:10 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 21:56:10 -0700
Subject: [IPython-dev] more on embedding Qt/IPythonWidget
In-Reply-To: 
References: 
Message-ID: 

Hi Piotr,

On Tue, Jan 24, 2012 at 7:33 AM, Piotr Zolnierczuk
 wrote:
> Hi again,
>
> I am trying to understand the IPythonWidget (eventually I would like to
> embed it in my Qt app)
> I am playing with Robert Kern's example? https://gist.github.com/1416977
> and it does what I need. What I would like to add is the ability to connect
> to the kernel from another qtconsole.
>
> So I tried to replace his QtKernelManager ?with the "stock" one (changed one
> line in his example.py)
>
> -from qtkernelmanager import QtKernelManager
> +from IPython.frontend.qt.kernelmanager import QtKernelManager
>
> But now there's no prompt. What am I missing?

Did you manage to solve this issue?  Just a few days ago Ray Osborn
posted he'd successfully done the same thing, perhaps we could at
least get some sharing of the 'hack that works' posted in a canonical
place, until there's time for a cleaner refactoring...

f


From fperez.net at gmail.com  Mon Apr 23 02:00:24 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 23:00:24 -0700
Subject: [IPython-dev] How to add tests in IPython/external
In-Reply-To: <4E1C8BC3.80107@bostream.nu>
References: <4E1C8BC3.80107@bostream.nu>
Message-ID: 

Hi Jorgen,

On Tue, Jul 12, 2011 at 11:00 AM, J?rgen Stenarson
 wrote:
> I'm considering adding some tests to Itpl in IPython/external to
> investigate unicode issues in that module. But after adding a tests
> submodule in the Itpl directory I can't run those tests because nose
> crashes when it can't import pexepct. Any ideas on how to solve this?

did you eventually figure this out?  If the problem persists, we can
try to give you a hand.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 02:19:04 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 23:19:04 -0700
Subject: [IPython-dev] Package-specific extensions: an idea
In-Reply-To: 
References: 
	
	
	
	
	
Message-ID: 

Robert,

On Fri, Jun 17, 2011 at 11:26 AM, Robert Kern  wrote:
>
> Except that you don't want to register a separate function for each extension.
> You want to have a single function that deals with all of the delayed
> activations. And you want to implement this logic exactly once, not reimplement
> it in each extension that needs it.
>
> How about this: we implement the logic that goes through the list of
> ('module_name', 'extension_name') pairs in an
> InteractiveShell.check_deferred_extensions() method. We keep the configuration
> in c.Global.deferred_extensions as I had it. We make no modifications to the
> run_cell() method. Upon configuration, *if* deferred_extensions is not empty,
> then the InteractiveShell uses
> self.register_post_execute(self.check_deferred_extensions).

we seem to have dropped the ball on this one due to lack of bandwidth
with the whirlwind of last year's work.  I actually like the basic
idea you were proposing, as long as we use the post_execute hook to do
the registration rather than adding a hard-coded check on sys.modules
to every single execution.

If wrapped into a single flag that can be set by users who want this
capability, as you propose, I think it could be very useful.

If you're still interested, feel free to put up a  PR with an
implementation and we can go ahead with review.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 02:32:04 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 23:32:04 -0700
Subject: [IPython-dev] Failing to start qtconsole
In-Reply-To: 
References: 
Message-ID: 

Hi Hannes,

On Tue, Mar 6, 2012 at 9:08 AM, Hannes Bretschneider
 wrote:
> I use PyQt 4.9.1 (installed from source), with the Qt4 headers available
> through apt-get in Ubuntu 11.10.
>
> I'd appreciate some guidance what went wrong.

I've never seen a traceback like what you presented... Could you try
with the ubuntu native pyqt package?  That's what I've used and I've
never seen a problem.

If there's some weird interaction between the system Qt and a
hand-built PyQt, I'm afraid that's way beyond my Qt expertise; I
suggest asking on the PyQt list directly where I think your chances
are better.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 02:33:03 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 22 Apr 2012 23:33:03 -0700
Subject: [IPython-dev] using ipkernel to provide access to an
 application's internals
In-Reply-To: 
References: 
Message-ID: 

Hi Ivan,

On Fri, Feb 24, 2012 at 6:15 AM, Ivan Zhang  wrote:
>
> Say I have a typical python gui that creates some python objects, modifies
> them, etc...Now instead of just running this code directly, say I want to
> run this inside the IPKernel so that all of its data can then be inspected
> by someone connecting an qtconsole to it.

This capability is almost ready to be merged:

https://github.com/ipython/ipython/pull/1640

should be done in a day or two.

Cheers,

f


From robert.kern at gmail.com  Mon Apr 23 06:20:36 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 23 Apr 2012 11:20:36 +0100
Subject: [IPython-dev] traitlets question [from Re: msg spec]
In-Reply-To: 
References: 
	
Message-ID: 

On 4/23/12 5:36 AM, Fernando Perez wrote:
>
> On Tue, Apr 17, 2012 at 7:27 PM, Satrajit Ghosh  wrote:

>> and to follow up, does traitlets have event delegation?
>
> They have basic static and dynamic _foo_changed and on_trait_change
> event handler like enthought.Traits do, but I think that Traits
> support a more complex delegation pattern that we don't.

I'm not sure exactly which feature Satrajit is referring to, but I'm pretty sure 
that traitlets doesn't have it. :-)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Mon Apr 23 06:25:31 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 23 Apr 2012 11:25:31 +0100
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
Message-ID: 

On 23 April 2012 05:20, Fernando Perez  wrote:
> - run: yes, if a __future__ is imported in a file that was %run, then
> it should become active interactively. ?The reason is to keep the
> semantics of operating with things defined in that script consistent.
> Say a script had `z=x/y` in it, you want to be able to later type at
> the command-line `x/y` and get the same division semantics that were
> set inside the script.

On the other hand, %run (without -i) works a bit like doing `from foo
import *`, in which case you don't get semantic changes from
__future__ statements in foo. I'm inclined to agree with you that I'd
expect __future__ statements in %run to affect the interactive
environment, but I don't think it's entirely clear cut.

Thomas


From fperez.net at gmail.com  Mon Apr 23 06:44:11 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 23 Apr 2012 03:44:11 -0700
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
	
Message-ID: 

On Mon, Apr 23, 2012 at 3:25 AM, Thomas Kluyver  wrote:
> On the other hand, %run (without -i) works a bit like doing `from foo
> import *`, in which case you don't get semantic changes from
> __future__ statements in foo. I'm inclined to agree with you that I'd
> expect __future__ statements in %run to affect the interactive
> environment, but I don't think it's entirely clear cut.

I was thinking more about what the more natural workflow would be,
since we pitch %run more closely related to "run a script as if you
had typed all that interactively" than "a shorthand for 'import *'".
But you are right, one could think of it either way...

f


From satra at mit.edu  Mon Apr 23 07:12:03 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 07:12:03 -0400
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
Message-ID: 

hi fernando,

we just refactored our logging and config system! always good to clean up
some code right :)

cheers,

satra


On Mon, Apr 23, 2012 at 12:45 AM, Fernando Perez wrote:

> On Sat, Mar 10, 2012 at 4:52 PM, Satrajit Ghosh  wrote:
> >
> > the culprit was node, so instead of sending it directly, i pickled it and
> > sent it. and unpickled it inside the function  after i had done any of
> the
> > configuring that i needed to do.
>
> Why not define another task that does the prep you need, and declare
> it as a dependency of this task?  Would that not work?
>
> Best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From satra at mit.edu  Mon Apr 23 07:14:05 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 07:14:05 -0400
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
Message-ID: 

ps. the reason i couldn't make it another task is because of the
distributed context. the engine on which the dependent task runs cannot be
controlled (i think).

cheers,

satra

On Mon, Apr 23, 2012 at 7:12 AM, Satrajit Ghosh  wrote:

> hi fernando,
>
> we just refactored our logging and config system! always good to clean up
> some code right :)
>
> cheers,
>
> satra
>
>
>
> On Mon, Apr 23, 2012 at 12:45 AM, Fernando Perez wrote:
>
>> On Sat, Mar 10, 2012 at 4:52 PM, Satrajit Ghosh  wrote:
>> >
>> > the culprit was node, so instead of sending it directly, i pickled it
>> and
>> > sent it. and unpickled it inside the function  after i had done any of
>> the
>> > configuring that i needed to do.
>>
>> Why not define another task that does the prep you need, and declare
>> it as a dependency of this task?  Would that not work?
>>
>> Best,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From satra at mit.edu  Mon Apr 23 07:21:23 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 07:21:23 -0400
Subject: [IPython-dev] IdleX adds IPython support
In-Reply-To: 
References: 
	
	
	
Message-ID: 

>
> That's good to hear!  I tried using pycharm after seeing your
> screencaps with it, but unfortunately under linux Java font rendering
> is bad to the point of being intolerable.  That, and the fact that it
> needed about 10 minutes to open my IPython directory, made me end up
> not really going much further with the tool.  But it seems that on
> OSX/Windows it's a very good tool, from what I've seen in demos.
>

hi fernando,

i use it routinely on our ubuntu cluster and works fine (even the font
rendering). the first-time indexing time (not startup) in a directory can
be long because it builds a set of indices depending on your project
interpreter, but after that it's instantaneous. btw, 2.5 was released last
friday.

it does give a very harsh warning about not having the Sun JDK and
incompatibility with OpenJDK, but i've ignored that from day one!

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From mirage007 at gmail.com  Mon Apr 23 07:57:44 2012
From: mirage007 at gmail.com (Ivan Zhang)
Date: Mon, 23 Apr 2012 07:57:44 -0400
Subject: [IPython-dev] using ipkernel to provide access to an
 application's internals
In-Reply-To: 
References: 
	
Message-ID: 

Wow, cool, had no idea this was work in progress, will look into it,
looking forward to it.

Thanks,

-i

On Apr 23, 2012 2:33 AM, "Fernando Perez"  wrote:

Hi Ivan,


On Fri, Feb 24, 2012 at 6:15 AM, Ivan Zhang  wrote:
>
> Say I have a typical p...
This capability is almost ready to be merged:

https://github.com/ipython/ipython/pull/1640

should be done in a day or two.

Cheers,

f
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From satra at mit.edu  Mon Apr 23 08:00:16 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 08:00:16 -0400
Subject: [IPython-dev] traitlets question [from Re: msg spec]
In-Reply-To: 
References: 
	
	
Message-ID: 

>
> I'm not sure exactly which feature Satrajit is referring to, but I'm
> pretty sure
> that traitlets doesn't have it. :-)
>

aha. i should have been more careful! i'm looking for sync traits (which
can be done with on_trait_change).

robert: since you are listening here :) i'm still trying to find a way to
address this bug [1].

cheers,

satra

[1] https://github.com/enthought/traits/issues/16
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From robert.kern at gmail.com  Mon Apr 23 08:50:46 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 23 Apr 2012 13:50:46 +0100
Subject: [IPython-dev] traitlets question [from Re: msg spec]
In-Reply-To: 
References: 
	
	
	
Message-ID: 

On 4/23/12 1:00 PM, Satrajit Ghosh wrote:
>     I'm not sure exactly which feature Satrajit is referring to, but I'm pretty sure
>     that traitlets doesn't have it. :-)
>
> aha. i should have been more careful! i'm looking for sync traits (which can be
> done with on_trait_change).

I still don't follow you. There is nothing that we call "sync traits". There is 
the HasTraits.sync_trait() method, but that's a verb phrase "sync[hronize one] 
trait [with another]". sync_trait() is not implemented in traitlets. Are you 
referring to DelegatesTo traits? Those are also not implemented in traitlets.

> robert: since you are listening here :) i'm still trying to find a way to
> address this bug [1].

This is not the list to talk about it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From satra at mit.edu  Mon Apr 23 09:21:53 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 09:21:53 -0400
Subject: [IPython-dev] traitlets question [from Re: msg spec]
In-Reply-To: 
References: 
	
	
	
	
Message-ID: 

hi robert,

On 4/23/12 1:00 PM, Satrajit Ghosh wrote:
> >     I'm not sure exactly which feature Satrajit is referring to, but I'm
> pretty sure
> >     that traitlets doesn't have it. :-)
> >
> > aha. i should have been more careful! i'm looking for sync traits (which
> can be
> > done with on_trait_change).
>
> I still don't follow you. There is nothing that we call "sync traits".
> There is
> the HasTraits.sync_trait() method, but that's a verb phrase "sync[hronize
> one]
> trait [with another]". sync_trait() is not implemented in traitlets.


the sync_trait method is exactly what is was referring to.


> Are you
> referring to DelegatesTo traits? Those are also not implemented in
> traitlets.
>

and we would like to use DelegatesTo as well.

thanks for the info.

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From benjaminrk at gmail.com  Mon Apr 23 12:47:53 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 23 Apr 2012 09:47:53 -0700
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
	
Message-ID: 

On Mon, Apr 23, 2012 at 04:14, Satrajit Ghosh  wrote:

> ps. the reason i couldn't make it another task is because of the
> distributed context. the engine on which the dependent task runs cannot be
> controlled (i think).


Nope.  The engines can be controlled quite specifically.  LoadBalancedViews
have a 'targets' attribute, just like DirectViews, but they select which
*subset* of targets across which the tasks may be distributed.  If this is
a single engine, then you know exactly where it will run.

-MinRK



>
> cheers,
>
> satra
>
>
> On Mon, Apr 23, 2012 at 7:12 AM, Satrajit Ghosh  wrote:
>
>> hi fernando,
>>
>> we just refactored our logging and config system! always good to clean up
>> some code right :)
>>
>> cheers,
>>
>> satra
>>
>>
>>
>> On Mon, Apr 23, 2012 at 12:45 AM, Fernando Perez wrote:
>>
>>> On Sat, Mar 10, 2012 at 4:52 PM, Satrajit Ghosh  wrote:
>>> >
>>> > the culprit was node, so instead of sending it directly, i pickled it
>>> and
>>> > sent it. and unpickled it inside the function  after i had done any of
>>> the
>>> > configuring that i needed to do.
>>>
>>> Why not define another task that does the prep you need, and declare
>>> it as a dependency of this task?  Would that not work?
>>>
>>> Best,
>>>
>>> f
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jorgen.stenarson at kroywen.se  Mon Apr 23 13:29:12 2012
From: jorgen.stenarson at kroywen.se (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Mon, 23 Apr 2012 19:29:12 +0200
Subject: [IPython-dev] How to add tests in IPython/external
In-Reply-To: 
References: <4E1C8BC3.80107@bostream.nu>
	
Message-ID: <4F959168.6060302@kroywen.se>

Fernando Perez skrev 2012-04-23 08:00:

> >  Hi Jorgen,
> >
> >  On Tue, Jul 12, 2011 at 11:00 AM, J?rgen Stenarson
> >     wrote:
>> >>    I'm considering adding some tests to Itpl in IPython/external to
>> >>    investigate unicode issues in that module. But after adding a tests
>> >>    submodule in the Itpl directory I can't run those tests because nose
>> >>    crashes when it can't import pexepct. Any ideas on how to solve this?
> >
> >  did you eventually figure this out?  If the problem persists, we can
> >  try to give you a hand.
> >

I did not persue this problem further because Thomas Kluyver said he was
going to do some major changes here. Looking back in my archive I see
that this was unfortunately not communicated on list. See below for copy
of relevant text.

Oops, I manage to just send this to Fernando. Here it is for the whole list.


/J?rgen


On 12 July 2011 19:00, J?rgen Stenarson
wrote:

      I'm considering adding some tests to Itpl in IPython/external to
      investigate unicode issues in that module. But after adding a tests
      submodule in the Itpl directory I can't run those tests because nose
      crashes when it can't import pexepct. Any ideas on how to solve this?


Just to note: I'm aiming in the future to replace all use of Itpl with
the advanced string formatting introduced in 2.6. There's a couple of
places it will still be in 0.11, but it will probably be done for 0.12.

Thanks,
Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From takowl at gmail.com  Mon Apr 23 13:51:57 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 23 Apr 2012 18:51:57 +0100
Subject: [IPython-dev] How to add tests in IPython/external
In-Reply-To: <4F959168.6060302@kroywen.se>
References: <4E1C8BC3.80107@bostream.nu>
	
	<4F959168.6060302@kroywen.se>
Message-ID: 

(My earlier message):
> Just to note: I'm aiming in the future to replace all use of Itpl with the
> advanced string formatting introduced in 2.6. There's a couple of places it
> will still be in 0.11, but it will probably be done for 0.12.

I seem to have succeeded with this. Itpl is now only referenced from
modules in quarantine & deathrow. If no-one objects, I'll prepare a PR
to remove our bundled Itpl, and note in the quarantined modules that
if they are updated, they should use the new formatting system. The
affected modules are ipy_editors and ipy_render.

Thomas


From asmeurer at gmail.com  Mon Apr 23 14:41:39 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 23 Apr 2012 12:41:39 -0600
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
	
	
Message-ID: 

I thought %run was more like executing the script from the command
line.  In other words, if you have

if __name__ == '__main__':
   blah

then blah will be run for %run (but it wouldn't in an import).

There are probably other invariants that hold in one situation but not
the other.

Aaron Meurer

On Mon, Apr 23, 2012 at 4:44 AM, Fernando Perez  wrote:
> On Mon, Apr 23, 2012 at 3:25 AM, Thomas Kluyver  wrote:
>> On the other hand, %run (without -i) works a bit like doing `from foo
>> import *`, in which case you don't get semantic changes from
>> __future__ statements in foo. I'm inclined to agree with you that I'd
>> expect __future__ statements in %run to affect the interactive
>> environment, but I don't think it's entirely clear cut.
>
> I was thinking more about what the more natural workflow would be,
> since we pitch %run more closely related to "run a script as if you
> had typed all that interactively" than "a shorthand for 'import *'".
> But you are right, one could think of it either way...
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From asmeurer at gmail.com  Mon Apr 23 14:46:34 2012
From: asmeurer at gmail.com (Aaron Meurer)
Date: Mon, 23 Apr 2012 12:46:34 -0600
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
	
	
	
Message-ID: 

I thought of another one: import * respects __all__.

Maybe it would be useful to have an %import magic, which would just
temporarily add the file path to the front of the python path and
import the file.  I don't know if it would be useful, though, and the
differences between it and %run would be subtle and probably not
really understood by non-expert Python users.

Aaron Meurer

On Mon, Apr 23, 2012 at 12:41 PM, Aaron Meurer  wrote:
> I thought %run was more like executing the script from the command
> line. ?In other words, if you have
>
> if __name__ == '__main__':
> ? blah
>
> then blah will be run for %run (but it wouldn't in an import).
>
> There are probably other invariants that hold in one situation but not
> the other.
>
> Aaron Meurer
>
> On Mon, Apr 23, 2012 at 4:44 AM, Fernando Perez  wrote:
>> On Mon, Apr 23, 2012 at 3:25 AM, Thomas Kluyver  wrote:
>>> On the other hand, %run (without -i) works a bit like doing `from foo
>>> import *`, in which case you don't get semantic changes from
>>> __future__ statements in foo. I'm inclined to agree with you that I'd
>>> expect __future__ statements in %run to affect the interactive
>>> environment, but I don't think it's entirely clear cut.
>>
>> I was thinking more about what the more natural workflow would be,
>> since we pitch %run more closely related to "run a script as if you
>> had typed all that interactively" than "a shorthand for 'import *'".
>> But you are right, one could think of it either way...
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev


From satra at mit.edu  Mon Apr 23 15:03:30 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 15:03:30 -0400
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
	
	
Message-ID: 

hi min,

 ps. the reason i couldn't make it another task is because of the
>> distributed context. the engine on which the dependent task runs cannot be
>> controlled (i think).
>
>
> Nope.  The engines can be controlled quite specifically.
>  LoadBalancedViews have a 'targets' attribute, just like DirectViews, but
> they select which *subset* of targets across which the tasks may be
> distributed.  If this is a single engine, then you know exactly where it
> will run.
>

this is more academic at this point, but just to be clear on the
functionality.

i have X engines running on X machines. and i'm distributing a task to some
arbitrary engine using LoadBalancedView. is there a way for me to send task
A and B and say run B on the same engine as A. if i remember target
attributes were setup during creation of an engine? i don't want to run A
on any particular engine.

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Mon Apr 23 15:32:05 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 23 Apr 2012 12:32:05 -0700
Subject: [IPython-dev] IdleX adds IPython support
In-Reply-To: 
References: 
	
	
	
	
Message-ID: 

On Mon, Apr 23, 2012 at 4:21 AM, Satrajit Ghosh  wrote:
> i use it routinely on our ubuntu cluster and works fine (even the font
> rendering). the first-time indexing time (not startup) in a directory can be
> long because it builds a set of indices depending on your project
> interpreter, but after that it's instantaneous. btw, 2.5 was released last
> friday.
>
> it does give a very harsh warning about not having the Sun JDK and
> incompatibility with OpenJDK, but i've ignored that from day one!

Mhh, good to know.  I should give it another try.

BTW, the license I was given is good for any IPython contributor, so
if you'd like to try it, please let me know and I'll give you the
license code.  Email me off list at: fernando.perez at berkeley.edu.  I'd
forgotten to mention this on the list, sorry for the delay.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 15:48:49 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 23 Apr 2012 12:48:49 -0700
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
	
	
	
	
Message-ID: 

On Mon, Apr 23, 2012 at 11:46 AM, Aaron Meurer  wrote:
> Maybe it would be useful to have an %import magic, which would just
> temporarily add the file path to the front of the python path and
> import the file. ?I don't know if it would be useful, though, and the
> differences between it and %run would be subtle and probably not
> really understood by non-expert Python users.

Yes, I think something like that is more likely to cause confusion than help.

As long as we have *a* policy re. __future__ in %run, we can just
document it, which is really what's missing: an explicit and
documented discussion of what to do re. __future__ in the various
parts of ipython...

Cheers,

f


From benjaminrk at gmail.com  Mon Apr 23 15:50:11 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 23 Apr 2012 12:50:11 -0700
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
	
	
	
Message-ID: 

On Mon, Apr 23, 2012 at 12:03, Satrajit Ghosh  wrote:

> hi min,
>
>  ps. the reason i couldn't make it another task is because of the
>>> distributed context. the engine on which the dependent task runs cannot be
>>> controlled (i think).
>>
>>
>> Nope.  The engines can be controlled quite specifically.
>>  LoadBalancedViews have a 'targets' attribute, just like DirectViews, but
>> they select which *subset* of targets across which the tasks may be
>> distributed.  If this is a single engine, then you know exactly where it
>> will run.
>>
>
> this is more academic at this point, but just to be clear on the
> functionality.
>
> i have X engines running on X machines. and i'm distributing a task to
> some arbitrary engine using LoadBalancedView. is there a way for me to send
> task A and B and say run B on the same engine as A.
>

No,


>  i don't want to run A on any particular engine.
>

Yes, this is what the `follow` dependencies accomplish, and what Fernando
referred to in his initial comment: Controlling the *destination* of tasks,
as a function of the destination of prior tasks:

def task_one(b):
    "store b in global a, for some reason"
    global a
    a = b

def task_two(c):
    "product of a and c, must be run following task_one"
    return c*a

rc = parallel.Client()

view = rc.load_balanced_view()

# submit task_one anywhere
ar_one = view.apply_async(task_one, 5)

# submit task_two to the same engine as task_one (even though we don't know
where that is, yet)
with view.temp_flags(follow=ar_one):
    ar_two = view.apply_async(task_two, 10)

ar_two.get()

print "task one ran on engine %i" % ar_one.engine_id
print "task two ran on engine %i" % ar_two.engine_id

We call these Graph Dependencies, and you can read a bit more about them in
the docs
.

Not knowing how your code actually works, it's possible this could be done
with functional dependencies (functions that run prior to the real one).
 This may not be true for you, though, as the dependency function is called
before the task is *run*, but not before it is unserialized.

> if i remember target attributes were setup during creation of an engine?

I don't know what this could refer to.  Nothing is setup on the client side
or in the schedulers during the creation of an engine.

-MinRK


>
> cheers,
>
> satra
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From fperez.net at gmail.com  Mon Apr 23 15:51:22 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 23 Apr 2012 12:51:22 -0700
Subject: [IPython-dev] How to add tests in IPython/external
In-Reply-To: 
References: <4E1C8BC3.80107@bostream.nu>
	
	<4F959168.6060302@kroywen.se>
	
Message-ID: 

On Mon, Apr 23, 2012 at 10:51 AM, Thomas Kluyver  wrote:
> I seem to have succeeded with this. Itpl is now only referenced from
> modules in quarantine & deathrow. If no-one objects, I'll prepare a PR
> to remove our bundled Itpl, and note in the quarantined modules that
> if they are updated, they should use the new formatting system. The
> affected modules are ipy_editors and ipy_render.

Great!  Please go for it.

Cheers,

f


From fperez.net at gmail.com  Mon Apr 23 15:57:34 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 23 Apr 2012 12:57:34 -0700
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
	
	
	
	
Message-ID: 

On Mon, Apr 23, 2012 at 12:50 PM, MinRK  wrote:
>> if i remember target attributes were setup during creation of an engine?
>
> I don't know what this could refer to. ?Nothing is setup on the client side
> or in the schedulers during the creation of an engine.

I think Satra is confusing the engine startup with the creation of the
view objects from the main Client, which take a targets attribute...

f


From satra at mit.edu  Mon Apr 23 16:07:05 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 16:07:05 -0400
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
	
	
	
	
Message-ID: 

thanks min,

def task_one(b):
>     "store b in global a, for some reason"
>     global a
>     a = b
>
> def task_two(c):
>     "product of a and c, must be run following task_one"
>     return c*a
>
> rc = parallel.Client()
>
> view = rc.load_balanced_view()
>
> # submit task_one anywhere
> ar_one = view.apply_async(task_one, 5)
>
> # submit task_two to the same engine as task_one (even though we don't
> know where that is, yet)
> with view.temp_flags(follow=ar_one):
>     ar_two = view.apply_async(task_two, 10)
>

this is exactly what i needed. thanks for this.


>  We call these Graph Dependencies, and you can read a bit more about them in
> the docs
> .
>

just read them and they are plenty clear.

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From satra at mit.edu  Mon Apr 23 16:08:39 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 23 Apr 2012 16:08:39 -0400
Subject: [IPython-dev] load_balanced_view
In-Reply-To: 
References: 
	
	
	
	
	
	
	
	
Message-ID: 

>
> >> if i remember target attributes were setup during creation of an engine?
> >
> > I don't know what this could refer to.  Nothing is setup on the client
> side
> > or in the schedulers during the creation of an engine.
>
> I think Satra is confusing the engine startup with the creation of the
> view objects from the main Client, which take a targets attribute...
>

it's indeed my day to be confused and confusing! thanks fernando for the
clarification.

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From hemmecke at gmail.com  Mon Apr 23 17:32:48 2012
From: hemmecke at gmail.com (Ralf Hemmecke)
Date: Mon, 23 Apr 2012 23:32:48 +0200
Subject: [IPython-dev] ipython notebook
Message-ID: <4F95CA80.7000108@gmail.com>

Hello!

I am currently looking for a notebook interface to the computer algebra 
system fricas (http://fricas.sourceforge.net/).

I was wondering whether it would be easily possible to use the power of 
the notebook that comes with ipython for this purpose.

I expect that the string that is entered into the notebook is sent (as 
is) to a fricas process running in the background. The output (just 
ASCII) could be either displayed directly or one can let fricas output 
tex or mathml and display that.

Is the notebook suited for this purpose or is it too much tied to python?

If the ipython notebook is not suited, I also welcome suggestions for 
other (active) notebook projects. I've looked into SageNotebook and 
Codenode so far.

Ralf


From jorgen.stenarson at kroywen.se  Mon Apr 23 18:04:31 2012
From: jorgen.stenarson at kroywen.se (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Tue, 24 Apr 2012 00:04:31 +0200
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
	
	
	
	
	
Message-ID: <4F95D1EF.9090706@kroywen.se>

Fernando Perez skrev 2012-04-23 21:48:
> On Mon, Apr 23, 2012 at 11:46 AM, Aaron Meurer  wrote:
> >  Maybe it would be useful to have an %import magic, which would just
> >  temporarily add the file path to the front of the python path and
> >  import the file.  I don't know if it would be useful, though, and the
> >  differences between it and %run would be subtle and probably not
> >  really understood by non-expert Python users.
>
> Yes, I think something like that is more likely to cause confusion than help.
>
> As long as we have *a* policy re. __future__ in %run, we can just
> document it, which is really what's missing: an explicit and
> documented discussion of what to do re. __future__ in the various
> parts of ipython...
To me it makes most sense to not import any __future__ into the global 
namespace unless explicitly asked for by some option (like %run -i).  In 
short I would prefer to be explicit about what ends up globally.

/J?rgen



From fperez.net at gmail.com  Tue Apr 24 01:56:28 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 23 Apr 2012 22:56:28 -0700
Subject: [IPython-dev] ipython notebook
In-Reply-To: <4F95CA80.7000108@gmail.com>
References: <4F95CA80.7000108@gmail.com>
Message-ID: 

Hi Ralf,

On Mon, Apr 23, 2012 at 2:32 PM, Ralf Hemmecke  wrote:
> I am currently looking for a notebook interface to the computer algebra
> system fricas (http://fricas.sourceforge.net/).
>
> I was wondering whether it would be easily possible to use the power of
> the notebook that comes with ipython for this purpose.
>
> I expect that the string that is entered into the notebook is sent (as
> is) to a fricas process running in the background. The output (just
> ASCII) could be either displayed directly or one can let fricas output
> tex or mathml and display that.
>
> Is the notebook suited for this purpose or is it too much tied to python?

Indeed, it would be possible to do so; recently an enterprising user
did just that as a test with Ruby:

https://github.com/isotope11/iruby

He said it was a bit of a hack; I haven't looked at the code.  But the
point is that it's possible, even if doing so may take a bit of work.

The notebook file format itself doesn't have any python-specific
dependencies.  As for such an app, you will most likely write parts in
Python, simply because that will be a little easier and gives you lots
of infrastructure for free.

Unfortunately we haven't had the time to write a detailed design
document that explains how the entire architecture works for someone
in your position.  So I'm afraid that making progress will require a
fair amount of reading the source and experimenting.  But don't be
afraid to ask here!

>From our perspective, this is *absolutely* something we'd like to see
out there.  Furthermore, as you make progress we may find ways of
refactoring our code to make your task easier.

Just today I was talking to a statistician friend and we were
discussing the possibility of an R notebook frontend.  That's also
something that would be great to have, and likely very useful.

Cheers,

f


From hemmecke at gmail.com  Tue Apr 24 13:42:09 2012
From: hemmecke at gmail.com (Ralf Hemmecke)
Date: Tue, 24 Apr 2012 19:42:09 +0200
Subject: [IPython-dev] ipython notebook
In-Reply-To: 
References: <4F95CA80.7000108@gmail.com>
	
Message-ID: <4F96E5F1.9030707@gmail.com>

Thank you, Fernando.

>> Is the notebook suited for this purpose or is it too much tied to
>> python?
>
> Indeed, it would be possible to do so; recently an enterprising user
> did just that as a test with Ruby:
>
> https://github.com/isotope11/iruby

 From what I figured out so far, but Josh Adams modified ipython
(https://github.com/isotope11/ipython/tree/ruby_kernel) in order to
start ruby. Not that I understood everything, I wonder why ipython does
not include such kind of talking to an external interpreter.

> Unfortunately we haven't had the time to write a detailed design
> document that explains how the entire architecture works for someone
> in your position.

Understandable, but may I still ask for the big picture.
Let me guess.

(a) The client side is done with JaveScript. This is responsible for
code highlighting, tab completion and what not.

(b) JavaScript is probably talking (via 0MQ) to some python process on
the server in order to get information of what to highlight and what to
complete

(c) The input string is sent (probably in json format) to the server.

(d) The server sends a json format of the output back to the client.

Of course, I want different highlighting. Which function is responsible
for that? How (for simplicity) could I simply switch that off?

Same question for completion.

I have to start an external process (fricas in my case) and send input
to it and receive its output. Which functions are responsible for that?

> So I'm afraid that making progress will require a fair amount of
> reading the source and experimenting.  But don't be afraid to ask
> here!

If I don't disturb people here too much...
What I don't see at the moment is in how far the notebook is tied to 
python. In some .ipynb file, I've seen "language: python" which seemed 
to be connected to one cell. That made me wonder whether you've already 
thought about having an input cell in another language. Since I would 
have to add the fricas language, where would I do this?

>> From our perspective, this is *absolutely* something we'd like to
>> see out there.

That's good. At least it sounds like I will be going a supported path.

> Just today I was talking to a statistician friend and we were
> discussing the possibility of an R notebook frontend.  That's also
> something that would be great to have, and likely very useful.

Yep. In fact, it would be wonderful, if one could have a multi-language 
environment in one notebook. The SageNotebook has something like this, 
but it cannot currently run without Sage and I am actually looking for 
something lighter.

BTW, I installed ipython into a virtualenv. However, I was unable to get 
it running with ZeroMQ installed in my HOME. I would be grateful, if I 
can learn about installing ipython with with ZeroMQ not under /usr/local.

Regards
Ralf


From dasneutron at gmail.com  Tue Apr 24 13:45:13 2012
From: dasneutron at gmail.com (Piotr Zolnierczuk)
Date: Tue, 24 Apr 2012 13:45:13 -0400
Subject: [IPython-dev] more on embedding Qt/IPythonWidget
In-Reply-To: 
References: 
	
Message-ID: 

I got a tip  from Raymond Bosman (back in February) that I thought
went posted to this mailing list, so I assumed it was non-issue
anymore.


Here it is:
========================================================

Hi Piotr,

I came across the same problem that no prompt was available in the
IPython Qt Widget.
The solution is to add:

km.start_channels()

The example code becomes:


from IPython.frontend.qt.console.ipython_widget import IPythonWidget
from IPython.lib import guisupport

from IPython.frontend.qt.kernelmanager import QtKernelManager

def main():
   app = guisupport.get_app_qt4()

   km = QtKernelManager()
   km.start_kernel()
   km.start_channels()
   control = IPythonWidget()
   control.gui_completion = True
   control.kernel_manager = km
   control.show()

   guisupport.start_event_loop_qt4(app)


# Application entry point.
if __name__ == '__main__':
   main()

===========================

Cheers

Piotr


On Mon, Apr 23, 2012 at 12:56 AM, Fernando Perez  wrote:
> Hi Piotr,
>
> On Tue, Jan 24, 2012 at 7:33 AM, Piotr Zolnierczuk
>  wrote:
>> Hi again,
>>
>> I am trying to understand the IPythonWidget (eventually I would like to
>> embed it in my Qt app)
>> I am playing with Robert Kern's example? https://gist.github.com/1416977
>> and it does what I need. What I would like to add is the ability to connect
>> to the kernel from another qtconsole.
>>
>> So I tried to replace his QtKernelManager ?with the "stock" one (changed one
>> line in his example.py)
>>
>> -from qtkernelmanager import QtKernelManager
>> +from IPython.frontend.qt.kernelmanager import QtKernelManager
>>
>> But now there's no prompt. What am I missing?
>
> Did you manage to solve this issue? ?Just a few days ago Ray Osborn
> posted he'd successfully done the same thing, perhaps we could at
> least get some sharing of the 'hack that works' posted in a canonical
> place, until there's time for a cleaner refactoring...
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From mark.voorhies at ucsf.edu  Tue Apr 24 14:12:05 2012
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Tue, 24 Apr 2012 11:12:05 -0700
Subject: [IPython-dev] ipython notebook
In-Reply-To: 
References: <4F95CA80.7000108@gmail.com>
	
Message-ID: <201204241112.06207.mark.voorhies@ucsf.edu>

On Monday 23 April 2012 10:56:28 pm Fernando Perez wrote:
> Just today I was talking to a statistician friend and we were
> discussing the possibility of an R notebook frontend.  That's also
> something that would be great to have, and likely very useful.
> 

FWIW, importing rpy within IPython is a pretty useful workflow, particularly
if what you're using from R is high level functions and plotting (the hairy
bit is if you're doing a lot of model fitting, which requires calls to
rpy.set_default_mode to switch among modes for passing objects
between python and R).  IPython.core.display.SVG plays nicely with
RSvgDevice for displaying R generated plots in the notebook.

E.g.
from IPython.core.display import SVG
import rpy
rpy.r.require("RSvgDevice")
rpy.r.svg("temp.svg")
rpy.r.plot(rpy.r.rnorm(100),rpy.r.rnorm(100), xlab = "x", ylab = "y", main = "")
rpy.r.dev_off()
SVG(filename = "temp.svg")

It _would_ be nice to see how easily the Emacs ESS mode could be ported
to the notebook (the nice features being interactive help in separate buffers
(as with IPython's paging), separate buffers for editing interactively defined
functions (as with %edit), some signal control (as with being able to interrupt
the kernel), and keybindings for some of R's more awkward syntax (e.g., <-).

--Mark

P.S. I have been teaching for the last two weeks and saving annotated IPython
notebooks from each session.  So far, I've been posting these for the students
as "log -o" python files + landscape-oriented pdf's printed from Firefox (landscape
seems to get around most of the cell truncation issues), but I hope to have some
time this weekend to use these notebooks as a test case for static html export
based on Fernando's script.  (This shouldn't be a blocker on anyone else taking
a crack at the problem -- I suspect that there are several different optima depending
on how WYSIWYG the output should be relative to the browser's vs. IPython's view
of the layout).



From benjaminrk at gmail.com  Tue Apr 24 14:24:44 2012
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 24 Apr 2012 11:24:44 -0700
Subject: [IPython-dev] ipython notebook
In-Reply-To: <4F96E5F1.9030707@gmail.com>
References: <4F95CA80.7000108@gmail.com>
	
	<4F96E5F1.9030707@gmail.com>
Message-ID: 

On Tue, Apr 24, 2012 at 10:42, Ralf Hemmecke  wrote:

> Thank you, Fernando.
>
> >> Is the notebook suited for this purpose or is it too much tied to
> >> python?
> >
> > Indeed, it would be possible to do so; recently an enterprising user
> > did just that as a test with Ruby:
> >
> > https://github.com/isotope11/iruby
>
>  From what I figured out so far, but Josh Adams modified ipython
> (https://github.com/isotope11/ipython/tree/ruby_kernel) in order to
> start ruby.
>

The *only* modification he needed to make to IPython was to change the
CodeMirror language to Ruby.
The rest was just subclassing and configuration.

> Not that I understood everything, I wonder why ipython does not include
such kind of talking to an external interpreter.

IPython does not include kernels in a dozen other languages because each
one must have an implementation of the kernel.

There are two ways to implement a new kernel:

1. translate the existing kernel into the target language. This would have
to include complying with message spec and serialization, etc.
2. subclass existing Kernel, to call out to a subprocess for execution.
 This would save a lot of work, particularly if all you want to do is
simple stdin/stdout, but would be inefficient, as it adds yet another
process/network hop to the mix, and only appropriate for supporting
languages that don't do much interactively.

This is all work that has to be done, and it is work that is going to be
done by people who want to use these other languages in the notebook (i.e.
not IPython devs). We are happy to help contributors with this work, but it
will not be us who write it.


>
> > Unfortunately we haven't had the time to write a detailed design
> > document that explains how the entire architecture works for someone
> > in your position.
>
> Understandable, but may I still ask for the big picture.
> Let me guess.
>
> (a) The client side is done with JavaScript. This is responsible for
> code highlighting, tab completion and what not.
>

correct


>
> (b) JavaScript is probably talking (via 0MQ) to some python process on
> the server in order to get information of what to highlight and what to
> complete
>

Close: js talks websockets, which are translated to zmq by the nb server.

Completion is handled in the kernel, but highlighting is purely js
(CodeMirror).


>
> (c) The input string is sent (probably in json format) to the server.
>
> (d) The server sends a json format of the output back to the client.
>
> Of course, I want different highlighting. Which function is responsible
> for that? How (for simplicity) could I simply switch that off?
>

Highlighting is done by CodeMirror, and the language is selected
here
.
We have not exposed UI to change the highlighting language, but it
wouldn't be difficult.  If your language is not already supported by
CodeMirror,
I think you have to add the highlighting spec.


>
> Same question for completion.
>
> I have to start an external process (fricas in my case) and send input
> to it and receive its output. Which functions are responsible for that?
>

The launch_kernel function in a KernelManager


>
> > So I'm afraid that making progress will require a fair amount of
> > reading the source and experimenting.  But don't be afraid to ask
> > here!
>
> If I don't disturb people here too much...
> What I don't see at the moment is in how far the notebook is tied to
> python. In some .ipynb file, I've seen "language: python" which seemed
> to be connected to one cell. That made me wonder whether you've already
> thought about having an input cell in another language. Since I would
> have to add the fricas language, where would I do this?
>

This code would be *entirely* in the javascript - it's just a metadata
field on the Cell,
and it would only be a few lines to make it editable.


>
> >> From our perspective, this is *absolutely* something we'd like to
> >> see out there.
>
> That's good. At least it sounds like I will be going a supported path.
>
> > Just today I was talking to a statistician friend and we were
> > discussing the possibility of an R notebook frontend.  That's also
> > something that would be great to have, and likely very useful.
>
> Yep. In fact, it would be wonderful, if one could have a multi-language
> environment in one notebook. The SageNotebook has something like this,
> but it cannot currently run without Sage and I am actually looking for
> something lighter.
>
> BTW, I installed ipython into a virtualenv. However, I was unable to get
> it running with ZeroMQ installed in my HOME. I would be grateful, if I
> can learn about installing ipython with with ZeroMQ not under /usr/local.
>

This should be easy, just configure zeromq with the virtualenv prefix (or
any other prefix, for that matter):

version=2.1.11

curl -O http://download.zeromq.org/zeromq-$version.tar.gz
tar -xzf zeromq-$version.tar.gz
cd zeromq-$version
./configure --prefix=$VIRTUAL_ENV
make && make install

# and install pyzmq, ensuring it knows where to look for libzmq:

pip install pyzmq --install-option="--zmq=$VIRTUAL_ENV"

-MinRK



> Regards
> Ralf
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jason-sage at creativetrax.com  Tue Apr 24 14:40:27 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Tue, 24 Apr 2012 13:40:27 -0500
Subject: [IPython-dev] ipython notebook
In-Reply-To: <4F96E5F1.9030707@gmail.com>
References: <4F95CA80.7000108@gmail.com>
	
	<4F96E5F1.9030707@gmail.com>
Message-ID: <4F96F39B.7060808@creativetrax.com>

On 4/24/12 12:42 PM, Ralf Hemmecke wrote:
> Yep. In fact, it would be wonderful, if one could have a multi-language
> environment in one notebook. The SageNotebook has something like this,
> but it cannot currently run without Sage and I am actually looking for
> something lighter.

The (new) Sage notebook has some very trivial dependencies on Sage (more 
convenience than real dependencies, and it should be very 
straightforward to remove them...).  However, you're right that it is 
quite a bit heavier than the ipython notebook, but it also provides a 
scalable centralized architecture for a multi-user server, tracking 
users, notebooks, etc.  It's a heavier-weight solution to a more complex 
problem. If you just want a personal notebook that is an interface to a 
single personal installation, the IPython notebook is probably a better fit.

Jason



From fperez.net at gmail.com  Tue Apr 24 14:49:35 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 24 Apr 2012 11:49:35 -0700
Subject: [IPython-dev] ipython notebook
In-Reply-To: <4F96E5F1.9030707@gmail.com>
References: <4F95CA80.7000108@gmail.com>
	
	<4F96E5F1.9030707@gmail.com>
Message-ID: 

On Tue, Apr 24, 2012 at 10:42 AM, Ralf Hemmecke  wrote:
> I wonder why ipython does
> not include such kind of talking to an external interpreter.

Min gave you already a set of solid responses, I just want to
reiterate on this point: we want to make it *possible* for other
developers to do this kind of work, but we have limited resources and
only build things as they are needed.  The way an out-of-process
kernel will happen is if someone (e.g. you) does the work and produces
a functioning implementation.

Most likely in the process, you'll find that certain things could be
easier if IPython itself adjusted certain things; in that case and
driven by a concrete use case, we'll be happy to fine-tune things.
But we simply don't have the resources to create an abstract
architecture for third-party kernels in a vacuum (and besides, I don't
believe such approaches produce useful tools).

Cheers,

f


From fperez.net at gmail.com  Tue Apr 24 16:39:35 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 24 Apr 2012 13:39:35 -0700
Subject: [IPython-dev] more on embedding Qt/IPythonWidget
In-Reply-To: 
References: 
	
	
Message-ID: 

On Tue, Apr 24, 2012 at 10:45 AM, Piotr Zolnierczuk
 wrote:
> I got a tip ?from Raymond Bosman (back in February) that I thought
> went posted to this mailing list, so I assumed it was non-issue
> anymore.
>

I may have missed it, but thanks for posting it again, it's a good example!

Cheers,

f


From takowl at gmail.com  Tue Apr 24 18:34:48 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 24 Apr 2012 23:34:48 +0100
Subject: [IPython-dev] ipython notebook
In-Reply-To: <4F96E5F1.9030707@gmail.com>
References: <4F95CA80.7000108@gmail.com>
	
	<4F96E5F1.9030707@gmail.com>
Message-ID: 

On 24 April 2012 18:42, Ralf Hemmecke  wrote:
> Yep. In fact, it would be wonderful, if one could have a multi-language
> environment in one notebook. The SageNotebook has something like this,
> but it cannot currently run without Sage and I am actually looking for
> something lighter.

To add to what others have said, we're looking at cell level magics,
which would be a suitable interface for having different languages in
the cells of one notebook - you would prefix a cell with something
like "%%R". It's still some way off, but you might well see this in a
future release.

Thomas


From mdroe at stsci.edu  Wed Apr 25 11:11:18 2012
From: mdroe at stsci.edu (Michael Droettboom)
Date: Wed, 25 Apr 2012 11:11:18 -0400
Subject: [IPython-dev] clear_output() of only last line?
In-Reply-To: 
References: <4F85B4A5.4030001@stsci.edu>
	
Message-ID: <4F981416.6020001@stsci.edu>

That's helpful.  What I'm hoping to get to work in progress bars 
interspersed without other logging all in the same output chunk -- and 
with an API that makes sense and works either inside IPython notebook or 
out.  The ability to send down custom HTML and Javascript (which I 
didn't know about) may bear some fruit... will have experiment and think 
on that.

Thanks,
Mike

On 04/23/2012 12:06 AM, Fernando Perez wrote:
> Hi Mike,
>
> On Wed, Apr 11, 2012 at 9:43 AM, Michael Droettboom  wrote:
>> I'm working on getting our animated progressbars in astropy to work in the
>> IPython notebook.  I found the recipe of using "clear_output()" in this bug
>> report:
>>
>> https://github.com/ipython/ipython/issues/1527
>>
>> That gets things closer, but unlike "\r" at a standard terminal,
>> clear_output() clears the entire output chunk, not just the last line.  So
>> if we have multiple progress bars, or progress bars interspersed with
>> logging messages, all but the last progress bar gets deleted.
>>
>> For example:
>>
>> from IPython.core.display import clear_output
>>
>> print "Working"
>> for i in range(80):
>>      clear_output()
>>      print "\r" + ("=" * i)
>>
>> Any suggestions?
> Have you had a chance to look at the new Animations_and_Progress
> notebook in master?  It shows examples of how to make both text-based
> and graphical progressbars in the nb.
>
> I hope that helps, and sorry for the delayed reply!
>
> Best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From mdroe at stsci.edu  Wed Apr 25 13:21:45 2012
From: mdroe at stsci.edu (Michael Droettboom)
Date: Wed, 25 Apr 2012 13:21:45 -0400
Subject: [IPython-dev] clear_output() of only last line?
In-Reply-To: <4F981416.6020001@stsci.edu>
References: <4F85B4A5.4030001@stsci.edu>
	
	<4F981416.6020001@stsci.edu>
Message-ID: <4F9832A9.4080700@stsci.edu>

I've hacked the fixConsole stuff in the JavaScript to mangle the text 
based on the "\r" in it.  This seems to work quite well, but I can't get 
the saving/restoring of notebooks to work correctly.  I have a pull 
request here:

https://github.com/ipython/ipython/pull/1659

Mike

On 04/25/2012 11:11 AM, Michael Droettboom wrote:
> That's helpful.  What I'm hoping to get to work in progress bars
> interspersed without other logging all in the same output chunk -- and
> with an API that makes sense and works either inside IPython notebook or
> out.  The ability to send down custom HTML and Javascript (which I
> didn't know about) may bear some fruit... will have experiment and think
> on that.
>
> Thanks,
> Mike
>
> On 04/23/2012 12:06 AM, Fernando Perez wrote:
>> Hi Mike,
>>
>> On Wed, Apr 11, 2012 at 9:43 AM, Michael Droettboom   wrote:
>>> I'm working on getting our animated progressbars in astropy to work in the
>>> IPython notebook.  I found the recipe of using "clear_output()" in this bug
>>> report:
>>>
>>> https://github.com/ipython/ipython/issues/1527
>>>
>>> That gets things closer, but unlike "\r" at a standard terminal,
>>> clear_output() clears the entire output chunk, not just the last line.  So
>>> if we have multiple progress bars, or progress bars interspersed with
>>> logging messages, all but the last progress bar gets deleted.
>>>
>>> For example:
>>>
>>> from IPython.core.display import clear_output
>>>
>>> print "Working"
>>> for i in range(80):
>>>       clear_output()
>>>       print "\r" + ("=" * i)
>>>
>>> Any suggestions?
>> Have you had a chance to look at the new Animations_and_Progress
>> notebook in master?  It shows examples of how to make both text-based
>> and graphical progressbars in the nb.
>>
>> I hope that helps, and sorry for the delayed reply!
>>
>> Best,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From hans_meine at gmx.net  Wed Apr 25 15:56:05 2012
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 25 Apr 2012 21:56:05 +0200
Subject: [IPython-dev] Where should __future__ statements affect the
	interactive environment.
In-Reply-To: <4F95D1EF.9090706@kroywen.se>
References: 
	
	<4F95D1EF.9090706@kroywen.se>
Message-ID: <1367608.AiK7xuXvST@hmeine-pc>

Am Dienstag, 24. April 2012, 00:04:31 schrieb J?rgen Stenarson:
> To me it makes most sense to not import any __future__ into the global
> namespace unless explicitly asked for by some option (like %run -i).  In
> short I would prefer to be explicit about what ends up globally.

I felt exactly the same, namely that '%run foo.py' should behave similar to 
'python foo.py' and *not* import the __future__s, while '%run -i foo.py' is 
more like execfile and should import them.

Even if -i is mostly used (AFAICS) for transferring bindings /into/ the script 
context, it is actually documented as "run the file in IPython's namespace 
instead of an empty one", which would IMHO lead to the above semantics.

Best regards,
  Hans



From benjaminrk at gmail.com  Wed Apr 25 16:50:01 2012
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 25 Apr 2012 13:50:01 -0700
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: <1367608.AiK7xuXvST@hmeine-pc>
References: 
	
	<4F95D1EF.9090706@kroywen.se> <1367608.AiK7xuXvST@hmeine-pc>
Message-ID: 

On Wed, Apr 25, 2012 at 12:56, Hans Meine  wrote:

> Am Dienstag, 24. April 2012, 00:04:31 schrieb J?rgen Stenarson:
> > To me it makes most sense to not import any __future__ into the global
> > namespace unless explicitly asked for by some option (like %run -i).  In
> > short I would prefer to be explicit about what ends up globally.
>
> I felt exactly the same, namely that '%run foo.py' should behave similar to
> 'python foo.py' and *not* import the __future__s, while '%run -i foo.py' is
> more like execfile and should import them.
>

execfile does *not* bring in __future__ imports, but I agree with the idea.
 I expect `run -i` to be
as close to interactive typing as we can get, but would be surprised if
regular `%run foo` included __future__ imports,
which are meant to be per-file.


> Even if -i is mostly used (AFAICS) for transferring bindings /into/ the
> script
> context, it is actually documented as "run the file in IPython's namespace
> instead of an empty one", which would IMHO lead to the above semantics.
>
> Best regards,
>  Hans
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From damianavila at gmail.com  Wed Apr 25 22:05:05 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Wed, 25 Apr 2012 23:05:05 -0300
Subject: [IPython-dev] html notebook converter
Message-ID: <4F98AD51.6030009@gmail.com>

Guys,

I have been reading the notebook converter script posted in github.
Searching for tools to a better conversion from rst to html, i have 
found a static site generator called Nikola (developed from a guy in my 
country).
So, I taked some code... I make some changes... and you can see the 
result here: http://damian.staticloud.com
This is the notebook tour example inside IPython docs...
It is a very preliminar result... you can see that some kind of cells 
are not converted... but also you can see the improvement in the 
"loooking" of the notebook.
I am using a jinja template (Nikola uses mako and jinja), and IPython is 
migrating to this templating engine, so i would be very easy to get the 
same look as the "live" notebook in the future.
Up to now, I am using docutils and pygments as dependencies.

I will be working on this... and I will published it when I clean the 
mess...

Regards.

Dami?n.



From fperez.net at gmail.com  Thu Apr 26 01:38:00 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 Apr 2012 22:38:00 -0700
Subject: [IPython-dev] html notebook converter
In-Reply-To: <4F98AD51.6030009@gmail.com>
References: <4F98AD51.6030009@gmail.com>
Message-ID: 

Hi Damian,

On Wed, Apr 25, 2012 at 7:05 PM, Dami?n Avila  wrote:
> I have been reading the notebook converter script posted in github.
> Searching for tools to a better conversion from rst to html, i have
> found a static site generator called Nikola (developed from a guy in my
> country).
> So, I taked some code... I make some changes... and you can see the
> result here: http://damian.staticloud.com

great, thanks for diving into this!  It actually looks very nice,
though I'm curious: why not use sphinx?  While it will be great to
allow people to plug in their own converters, I think that given how
so much of the python world uses sphinx, that would be a good target
to consider.  Additionally, sphinx produces high-quality pdf via
latex, which is another reason why good sphinx support is worthwhile.

When considering this problem, I'd thought that we'd probably offer
out of the box a basic html export using only docutils, and a richer
one using sphinx.  How does that sound to you?

And in any case, though in the end most people really want html/pdf,
having the intermediate rst export available as an explicit target is
always a good idea.  That way, people may decide to only use our
converter for rst and then plug it into the rest of their worfklow.

But in any case, I'm glad to see you looking into this problem and
making good progress!

Best,

f


From ellisonbg at gmail.com  Thu Apr 26 01:43:27 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 25 Apr 2012 22:43:27 -0700
Subject: [IPython-dev] html notebook converter
In-Reply-To: 
References: <4F98AD51.6030009@gmail.com>
	
Message-ID: 

On Wed, Apr 25, 2012 at 10:38 PM, Fernando Perez  wrote:
> Hi Damian,
>
> On Wed, Apr 25, 2012 at 7:05 PM, Dami?n Avila  wrote:
>> I have been reading the notebook converter script posted in github.
>> Searching for tools to a better conversion from rst to html, i have
>> found a static site generator called Nikola (developed from a guy in my
>> country).
>> So, I taked some code... I make some changes... and you can see the
>> result here: http://damian.staticloud.com
>
> great, thanks for diving into this! ?It actually looks very nice,
> though I'm curious: why not use sphinx? ?While it will be great to
> allow people to plug in their own converters, I think that given how
> so much of the python world uses sphinx, that would be a good target
> to consider. ?Additionally, sphinx produces high-quality pdf via
> latex, which is another reason why good sphinx support is worthwhile.
>
> When considering this problem, I'd thought that we'd probably offer
> out of the box a basic html export using only docutils, and a richer
> one using sphinx. ?How does that sound to you?

Our basic HTML exporter should simply use something like jinja and
should not go through reST as an intermediary format.  This will
enable us to write the HTML using the same css classes the actual
notebook uses - we will get the right look with very little effort.
We will probably have to tweak the css somewhat because we do some css
tricks in Javascript, but a good bit is covered by our static css
stylesheets.

But I do agree that sphinx writes very nice html and we should
definitely have a writer that goes through that route.  I suppose we
could also support regular docutils, but I don't see as much benefit
in supporting that.

> And in any case, though in the end most people really want html/pdf,
> having the intermediate rst export available as an explicit target is
> always a good idea. ?That way, people may decide to only use our
> converter for rst and then plug it into the rest of their worfklow.
>
> But in any case, I'm glad to see you looking into this problem and
> making good progress!
>
> Best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From fperez.net at gmail.com  Thu Apr 26 01:48:18 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 25 Apr 2012 22:48:18 -0700
Subject: [IPython-dev] html notebook converter
In-Reply-To: 
References: <4F98AD51.6030009@gmail.com>
	
	
Message-ID: 

On Wed, Apr 25, 2012 at 10:43 PM, Brian Granger  wrote:
> Our basic HTML exporter should simply use something like jinja and
> should not go through reST as an intermediary format. ?This will
> enable us to write the HTML using the same css classes the actual
> notebook uses - we will get the right look with very little effort.

I guess I was thinking of documents with plaintext cells that contain
rst in them, and for those it's kind of unavoidable to use docutils if
we want them to render well... There's no point in writing our own
rst->html conversion code when docutils does precisely that.

Cheers,

f


From ellisonbg at gmail.com  Thu Apr 26 02:01:19 2012
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 25 Apr 2012 23:01:19 -0700
Subject: [IPython-dev] html notebook converter
In-Reply-To: 
References: <4F98AD51.6030009@gmail.com>
	
	
	
Message-ID: 

On Wed, Apr 25, 2012 at 10:48 PM, Fernando Perez  wrote:
> On Wed, Apr 25, 2012 at 10:43 PM, Brian Granger  wrote:
>> Our basic HTML exporter should simply use something like jinja and
>> should not go through reST as an intermediary format. ?This will
>> enable us to write the HTML using the same css classes the actual
>> notebook uses - we will get the right look with very little effort.
>
> I guess I was thinking of documents with plaintext cells that contain
> rst in them, and for those it's kind of unavoidable to use docutils if
> we want them to render well... There's no point in writing our own
> rst->html conversion code when docutils does precisely that.

Ahh, yes, the direct to HTML route will only work when the stuff in
the plaintext cells is pure HTML or something like markdown that is
trivial to convert to HTML.  Good point.

Cheers,

Brian

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com


From warren.weckesser at enthought.com  Thu Apr 26 12:20:45 2012
From: warren.weckesser at enthought.com (Warren Weckesser)
Date: Thu, 26 Apr 2012 11:20:45 -0500
Subject: [IPython-dev] SciPy 2012 - The Eleventh Annual Conference on
 Scientific Computing with Python
In-Reply-To: 
References: 
Message-ID: 

Dear all,

(Sorry if you receive this announcement multiple times.)

Registration for SciPy 2012, the eleventh annual Conference on Scientific
Computing with Python, is open! Go to
https://conference.scipy.org/scipy2012/register/index.php

We would like to remind you that the submissions for talks, posters and
tutorials are open *until April 30th, *which is just around the corner. For
more information see:

http://conference.scipy.org/scipy2012/tutorials.php
http://conference.scipy.org/scipy2012/talks/index.php

For talks or posters, all we need is an abstract.  Tutorials require more
significant preparation.  If you are preparing a tutorial, please send a
brief note to Jonathan Rocher (jrocher at enthought.com) to indicate your
intent.

We look forward to seeing many of you this summer.

Kind regards,

The SciPy 2012 organizers
scipy2012 at scipy.org



On Wed, Apr 4, 2012 at 4:30 PM, Warren Weckesser <
warren.weckesser at enthought.com> wrote:

> SciPy 2012, the eleventh annual Conference on Scientific Computing with
> Python, will be held July 16?21, 2012, in Austin, Texas.
>
> At this conference, novel scientific applications and libraries related to
> data acquisition, analysis, dissemination and visualization using Python
> are presented. Attended by leading figures from both academia and industry,
> it is an excellent opportunity to experience the cutting edge of scientific
> software development.
>
> The conference is preceded by two days of tutorials, during which
> community experts provide training on several scientific Python packages.
>  Following the main conference will be two days of coding sprints.
>
> We invite you to give a talk or present a poster at SciPy 2012.
>
> The list of topics that are appropriate for the conference includes (but
> is not limited to):
>
>    - new Python libraries for science and engineering;
>    - applications of Python in solving scientific or computational
>    problems;
>    - high performance, parallel and GPU computing with Python;
>    - use of Python in science education.
>
>
>
> Specialized Tracks
>
> Two specialized tracks run in parallel to the main conference:
>
>    - High Performance Computing with Python
>    Whether your algorithm is distributed, threaded, memory intensive or
>    latency bound, Python is making headway into the problem.  We are looking
>    for performance driven designs and applications in Python.  Candidates
>    include the use of Python within a parallel application, new architectures,
>    and ways of making traditional applications execute more efficiently.
>
>
>    - Visualization
>    They say a picture is worth a thousand words--we?re interested in
>    both!  Python provides numerous visualization tools that allow scientists
>    to show off their work, and we want to know about any new tools and
>    techniques out there.  Come show off your latest graphics, whether it?s an
>    old library with a slick new feature, a new library out to challenge the
>    status quo, or simply a beautiful result.
>
>
>
> Domain-specific Mini-symposia
>
> Mini-symposia on the following topics are also being organized:
>
>    - Computational bioinformatics
>    - Meteorology and climatology
>    - Astronomy and astrophysics
>    - Geophysics
>
>
>
> Talks, papers and posters
>
> We invite you to take part by submitting a talk or poster abstract.
>  Instructions are on the conference website:
>
>    
> http://conference.scipy.org/scipy2012/talks.php
>  
> Selected talks are included as papers in the peer-reviewed conference
> proceedings, to be published online.
>
>
> Tutorials
>
> Tutorials will be given July 16?17.  We invite instructors to submit
> proposals for half-day tutorials on topics relevant to scientific computing
> with Python.  See
>
>   http://conference.scipy.org/scipy2012/tutorials.php
>  
> for information about submitting a tutorial proposal.  To encourage
> tutorials of the highest quality, the instructor (or team of instructors)
> is given a $1,000 stipend for each half day tutorial.
>
>
> Student/Community Scholarships
>
> We anticipate providing funding for students and for active members of the
> SciPy community who otherwise might not be able to attend the conference.
>  See
>
>   http://conference.scipy.org/scipy2012/student.php
>  
> for scholarship application guidelines.
>
>
> Be a Sponsor
>
> The SciPy conference could not run without the generous support of the
> institutions and corporations who share our enthusiasm for Python as a tool
> for science.  Please consider sponsoring SciPy 2012.  For more information,
> see
>
>   http://conference.scipy.org/scipy2012/sponsor/index.php
>
>
> Important dates:
>
>    Monday, April 30: Talk abstracts and tutorial proposals due.
>    Monday, May 7: Accepted tutorials announced.
>    Monday, May 13: Accepted talks announced.
>
>   Monday, June 18: Early registration ends. (Price increases after this
> date.)
>    Sunday, July 8: Online registration ends.
>
>   Monday-Tuesday, July 16 - 17: Tutorials
>    Wednesday-Thursday, July 18 - July 19: Conference
>    Friday-Saturday, July 20 - July 21: Sprints
>
> We look forward to seeing you all in Austin this year!
>
> The SciPy 2012 Team
> http://conference.scipy.org/scipy2012/organizers.php
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From jtaylor.debian at googlemail.com  Thu Apr 26 16:26:30 2012
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Thu, 26 Apr 2012 22:26:30 +0200
Subject: [IPython-dev] Test failure in master
In-Reply-To: 
References: 
	<4F92B451.2070002@googlemail.com>
	
	
	<4F93F713.1050901@googlemail.com>
	
Message-ID: <4F99AF76.1020906@googlemail.com>

On 04/23/2012 12:36 AM, Thomas Kluyver wrote:
> On 22 April 2012 13:18, Julian Taylor  wrote:
>> One could add the test results to a packages and let shinning-panda
>> download and extract the package (only ar and tar need to be installed
>> for that)
>> If you deem it worth it could modify the recipe to do that.
> 
> It's not essential - we already have the tests run in a Debian based
> environment on Shiningpanda, and separately on a Mac with most of the
> dependencies installed (which did pick up this failure). But if it's
> easy to do, more tests never hurt.
> 

as the daily hasn't built since a week because of failures I went ahead
and moved the result to a package and in turn ignore the testsuite
result in the build process.
The xml files go into a ipython-testresults package and are extract to
usr/share/ipython/testresults
The package is published here:
https://launchpad.net/~jtaylor/+archive/ipython-dev/+packages
though to get the url you may have to some scraping as it is built from
bzr not git with the revision number in the filename.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: 

From damianavila at gmail.com  Thu Apr 26 16:35:34 2012
From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=)
Date: Thu, 26 Apr 2012 17:35:34 -0300
Subject: [IPython-dev] html notebook converter
In-Reply-To: 
References: <4F98AD51.6030009@gmail.com>
	
Message-ID: <4F99B196.6090908@gmail.com>

Fernando, thank for your answer... sorry for the delay... i have been 
very busy at work today.

El 26/04/12 02:38, Fernando Perez escribi?:
> When considering this problem, I'd thought that we'd probably offer
> out of the box a basic html export using only docutils, and a richer
> one using sphinx.  How does that sound to you?

It sounds great!!! I believe that we can get the two exporters, one 
based in docutils (maybe this one when I get it fully functional) and 
another based into Sphinx (in this case, I don't have too much 
experience with it, but this is a good "excuse" to swim in the 
architecture of this tool).

> And in any case, though in the end most people really want html/pdf,
> having the intermediate rst export available as an explicit target is
> always a good idea.  That way, people may decide to only use our
> converter for rst and then plug it into the rest of their worfklow.

I am agree with you that the intermediate export to rst is the right 
way, because there are a lot of tools to process this rst file and get 
different views of the notebook. We can not provide all this views, but 
we can give us (as users) a general "starter" format, and I believe that 
rst is the correct one.

> But in any case, I'm glad to see you looking into this problem and
> making good progress!
>
> Best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

I am glad to help a little bit...

Regards.

Dami?n.



From takowl at gmail.com  Sat Apr 28 18:27:47 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 28 Apr 2012 23:27:47 +0100
Subject: [IPython-dev] Where should __future__ statements affect the
 interactive environment.
In-Reply-To: 
References: 
	
	<4F95D1EF.9090706@kroywen.se> <1367608.AiK7xuXvST@hmeine-pc>
	
Message-ID: 

I've resolved the difference between .py and .ipy files with respect
to __future__, but making %run -i share the shell's __future__ will
take a bit more work, because of the other options that %run can have.

The preliminary pull request is here:
https://github.com/ipython/ipython/pull/1671

Thomas


From hans_meine at gmx.net  Sun Apr 29 10:38:20 2012
From: hans_meine at gmx.net (Hans Meine)
Date: Sun, 29 Apr 2012 16:38:20 +0200
Subject: [IPython-dev] Where should __future__ statements affect the
	interactive environment.
In-Reply-To: 
References: 
	
	<4F95D1EF.9090706@kroywen.se> <1367608.AiK7xuXvST@hmeine-pc>
	
	
Message-ID: <7BA93649-2253-4116-9A0C-7BDB63994CED@gmx.net>

Am 29.04.2012 um 00:27 schrieb Thomas Kluyver:

> I've resolved the difference between .py and .ipy files with respect
> to __future__, but making %run -i share the shell's __future__ will
> take a bit more work, because of the other options that %run can have.

Great.

I would say the latter has low priority, since ? as Min pointed out ? even execfile does not share __future__s.  Even if some of use expect %run -i to do, it does not seem to be clear-cut, and adding appropriate statements to the %run file is not much work and makes the intention clear for all cases anyhow.  (In the face of ambiguity, refuse the temptation to guess.)

Best regards,
  Hans

From fperez.net at gmail.com  Mon Apr 30 04:02:37 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 30 Apr 2012 01:02:37 -0700
Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from
	notebooks
Message-ID: 

Hi folks,

This pdf file:

http://archive.ipython.org/tmp/IntroNumPy.pdf

was produced without a single manual change, straight from the
notebook in the tarball next to it.  We're getting to the point where
we can really produce very nice PDFs straight from the notebook!

If you want to see this in master, please come along and help with the
code review (it also has the instructions to reproduce it yourself):

https://github.com/ipython/nbconvert/pull/12

I hope we'll have this moved over soon, so we can polish it further
and can make notebooks part of a regular publishing workflow.

Ouch, my wrists hurt now.  But it was a weekend well spent, I think...

Cheers,

f


From cschin at infoecho.net  Mon Apr 30 04:19:24 2012
From: cschin at infoecho.net (Jason Chin)
Date: Mon, 30 Apr 2012 01:19:24 -0700
Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from
	notebooks
In-Reply-To: 
References: 
Message-ID: 

Fernando,
Look great.(even on a phone)
Congratulate! any dependence? Latex?
Lot of random thing at work, but I am looking forward to sending my co-worker some high quality output. 

--Jason Chin, sent from a mobile device


On Apr 30, 2012, at 1:02 AM, Fernando Perez  wrote:

> Hi folks,
> 
> This pdf file:
> 
> http://archive.ipython.org/tmp/IntroNumPy.pdf
> 
> was produced without a single manual change, straight from the
> notebook in the tarball next to it.  We're getting to the point where
> we can really produce very nice PDFs straight from the notebook!
> 
> If you want to see this in master, please come along and help with the
> code review (it also has the instructions to reproduce it yourself):
> 
> https://github.com/ipython/nbconvert/pull/12
> 
> I hope we'll have this moved over soon, so we can polish it further
> and can make notebooks part of a regular publishing workflow.
> 
> Ouch, my wrists hurt now.  But it was a weekend well spent, I think...
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From fperez.net at gmail.com  Mon Apr 30 04:25:19 2012
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 30 Apr 2012 01:25:19 -0700
Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from
	notebooks
In-Reply-To: 
References: 
	
Message-ID: 

On Mon, Apr 30, 2012 at 1:19 AM, Jason Chin  wrote:
> Look great.(even on a phone)
> Congratulate! any dependence? Latex?

Thanks!  Yes, sorry that I forgot to list the dependencies:

- latex: unavoidable, since this is after all to produce pdflatex-based PDF :)

- pandoc: unavoidable, needed to convert markdown to latex (we're not
about to reinvent that)

- inkscape: completely avoidable on OSX.  that's what I'm using to
convert svg figures to pdf.  I just didn't know off-hand what
command-line utility would do that on OSX; once someone tells me what
the syntax should be to get svg -> pdf on OSX, removing the inkscape
dependency is a trivial 2-minute fix.

So on linux, this is just an apt-get away, on OSX it should be easy
once I know what svg2pdf tool to call.  On Windows, no clue if pandoc
and inkscape can be installed (on windows it will also likely be slow
as hell, as I open many subprocesses and subprocess creation on
windows is absurdly slow).

Cheers,

f


From jason-sage at creativetrax.com  Mon Apr 30 07:22:57 2012
From: jason-sage at creativetrax.com (Jason Grout)
Date: Mon, 30 Apr 2012 06:22:57 -0500
Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from
 notebooks
In-Reply-To: 
References: 
	
	
Message-ID: <4F9E7611.6030908@creativetrax.com>

On 4/30/12 3:25 AM, Fernando Perez wrote:
> - inkscape: completely avoidable on OSX.  that's what I'm using to
> convert svg figures to pdf.  I just didn't know off-hand what
> command-line utility would do that on OSX; once someone tells me what
> the syntax should be to get svg ->  pdf on OSX, removing the inkscape
> dependency is a trivial 2-minute fix.

How about cairosvg?  I haven't tried it (and it may not be complete 
enough), but it looks like an interesting, python-based option:

http://cairosvg.org/

Thanks,

Jason



From bussonniermatthias at gmail.com  Mon Apr 30 10:51:04 2012
From: bussonniermatthias at gmail.com (Matthias BUSSONNIER)
Date: Mon, 30 Apr 2012 16:51:04 +0200
Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from
	notebooks
In-Reply-To: 
References: 
	
	
Message-ID: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com>


Le 30 avr. 2012 ? 10:25, Fernando Perez a ?crit :

> - inkscape: completely avoidable on OSX.  that's what I'm using to
> convert svg figures to pdf.  I just didn't know off-hand what
> command-line utility would do that on OSX; once someone tells me what
> the syntax should be to get svg -> pdf on OSX, removing the inkscape
> dependency is a trivial 2-minute fix.

You can use Inkscape as a command line tool on mac, you 'just' need
to use the full path of a script inside the Inkscape bundle, which 
most of the time is :
/Applications/Inkscape.app/Contents/Resources/bin/inkscape
-- 
Matthias




From punchagan at gmail.com  Mon Apr 30 11:47:19 2012
From: punchagan at gmail.com (Puneeth Chaganti)
Date: Mon, 30 Apr 2012 21:17:19 +0530
Subject: [IPython-dev] Opening Notebooks from the cmd line
Message-ID: 

Hi All,

This is my first mail to the list, so, I'd like to begin with thanking
you for the awesome piece of software!  Thank you very much!

I was looking for comments on the utility of a feature and the
implementation, before I spent too much time on it.  I would like to be
able to open a notebook directly from the command line, without showing
the dashboard.  I couldn't find such a feature possible, and I came up
with some rough code[1] to do the same.  Would such a feature be useful?
Would it be better to clean up the code and send it as a PR to get
comments on it?

Thanks!
Puneeth

[1] - https://github.com/punchagan/ipython/tree/notebook-files-cmd-args


From satra at mit.edu  Mon Apr 30 13:08:10 2012
From: satra at mit.edu (Satrajit Ghosh)
Date: Mon, 30 Apr 2012 13:08:10 -0400
Subject: [IPython-dev] Opening Notebooks from the cmd line
In-Reply-To: 
References: 
Message-ID: 

hi puneeth


> I was looking for comments on the utility of a feature and the
> implementation, before I spent too much time on it.  I would like to be
> able to open a notebook directly from the command line, without showing
> the dashboard.  I couldn't find such a feature possible, and I came up
> with some rough code[1] to do the same.  Would such a feature be useful?
>

+1 this would be a useful feature as it cuts down on a few strokes.
especially if i can just point to a notebook without cd-ing to that
directory.


> Would it be better to clean up the code and send it as a PR to get
> comments on it?
>

it's good to announce on the list, but it is always useful to have some of
the pertinent discussion be attached to the code on a github PR.

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From takowl at gmail.com  Mon Apr 30 13:16:31 2012
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 30 Apr 2012 18:16:31 +0100
Subject: [IPython-dev] Opening Notebooks from the cmd line
In-Reply-To: 
References: 
	
Message-ID: 

On 30 April 2012 18:08, Satrajit Ghosh  wrote:
> +1 this would be a useful feature as it cuts down on a few strokes.
> especially if i can just point to a notebook without cd-ing to that
> directory.

There's even an issue open for it:

https://github.com/ipython/ipython/issues/945

I have a vague memory there was some obstacle to doing this
'properly', but I can't remember what it was, and maybe it's been
overcome in the meantime. I'm sure there'll be more detailed feedback
coming soon.

Thanks, Puneeth!

Thomas


From pivanov314 at gmail.com  Mon Apr 30 13:23:14 2012
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Mon, 30 Apr 2012 10:23:14 -0700
Subject: [IPython-dev] Opening Notebooks from the cmd line
In-Reply-To: 
References: 
Message-ID: 

On Mon, Apr 30, 2012 at 8:47 AM, Puneeth Chaganti  wrote:
> I was looking for comments on the utility of a feature and the
> implementation, before I spent too much time on it. ?I would like to be
> able to open a notebook directly from the command line, without showing
> the dashboard. ?I couldn't find such a feature possible, and I came up
> with some rough code[1] to do the same. ?Would such a feature be useful?
> Would it be better to clean up the code and send it as a PR to get
> comments on it?
>
> [1] - https://github.com/punchagan/ipython/tree/notebook-files-cmd-args

Hi Puneeth,
this indeed would be useful, and I just filed an enhancement request [1] that
could leverage such a feature and go one step further by running a
specific notebook in
place (ipython can already do that with .py files, so no reason to not be able
to do it with .ipynb files)

This was prompted by a feature request Stefan Krastanov filed [2]
against nbconvert

1. https://github.com/ipython/ipython/issues/1675
2. https://github.com/ipython/nbconvert/issues/13

best,
-- 
Paul Ivanov
314 address only used for lists, ?off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7


From benjaminrk at gmail.com  Mon Apr 30 13:28:20 2012
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 30 Apr 2012 10:28:20 -0700
Subject: [IPython-dev] Opening Notebooks from the cmd line
In-Reply-To: 
References: 
	
	
Message-ID: 

On Mon, Apr 30, 2012 at 10:16, Thomas Kluyver  wrote:

> On 30 April 2012 18:08, Satrajit Ghosh  wrote:
> > +1 this would be a useful feature as it cuts down on a few strokes.
> > especially if i can just point to a notebook without cd-ing to that
> > directory.
>
> There's even an issue open for it:
>
> https://github.com/ipython/ipython/issues/945
>
> I have a vague memory there was some obstacle to doing this
> 'properly', but I can't remember what it was, and maybe it's been
> overcome in the meantime. I'm sure there'll be more detailed feedback
> coming soon.
>

I think one part of the 'doing it properly' issue was fixing the random
urls, so it is easier to determine the url for a given notebook from one
session to the next.  The random urls have always been considered a
temporary solution, but the first attempt at consistent urls could result
in data loss during notebook renaming, so it was scrapped. (note: the UUID
urls should probably stay, but they should be deterministic for a given
filename).

-MinRK


>
> Thanks, Puneeth!
>
> Thomas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 

From warren.weckesser at enthought.com  Mon Apr 30 21:16:04 2012
From: warren.weckesser at enthought.com (Warren Weckesser)
Date: Mon, 30 Apr 2012 20:16:04 -0500
Subject: [IPython-dev] SciPy 2012 Abstract and Tutorial Deadlines Extended
Message-ID: 

SciPy 2012 Conference Deadlines Extended

Didn't quite finish your abstract or tutorial yet?  Good news: the SciPy
2012 organizers have extended the deadline until Friday, May 4.  Proposals
for tutorials and abstracts for talks and posters are now due by midnight
(Austin time, CDT), May 4.

For the many of you who have already submitted an abstract or tutorial:
thanks!   If you need to make corrections to an abstract or tutorial that
you have already submitted, you may resubmit it by the same deadline.

The SciPy 2012 Organizers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: