From fperez.net at gmail.com Tue May 1 20:42:56 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 1 May 2012 17:42:56 -0700 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> Message-ID: On Mon, Apr 30, 2012 at 7:51 AM, Matthias BUSSONNIER wrote: > You can use Inkscape as a command line tool on mac, you 'just' need > to use the full path of a script inside the Inkscape bundle, which > most of the time is : > /Applications/Inkscape.app/Contents/Resources/bin/inkscape BTW, for those who may want to try this out, Matthias already made a PR with this fix for OSX, and Stefan vdW fixed up also some of the formatting I didn't finish last weekend, so now it's producing very clean, tight output. We really should merge this soon into the mainline repo, I hope to have a bit of time to clean it up so we can proceed. Cheers, f From fperez.net at gmail.com Tue May 1 21:04:16 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 1 May 2012 18:04:16 -0700 Subject: [IPython-dev] Brilliant buildbot triggering from PRs... Message-ID: Hey, Major brownie points for whoever can see if something like this is doable for our Shining Panda setup: https://github.com/mozilla/pdf.js-bot If we could do in a PR comment area @bot test with the output summary ending up right there a few minutes later, with summary stats in the comment field and links to the full reports, that would be *AWESOME*. Major time saver, and it would significantly decrease the chances that merging a PR will introduce a regression in a platform the reviewers don't have. Cheers, f From takowl at gmail.com Wed May 2 07:16:39 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 2 May 2012 12:16:39 +0100 Subject: [IPython-dev] Brilliant buildbot triggering from PRs... In-Reply-To: References: Message-ID: On 2 May 2012 02:04, Fernando Perez wrote: > Major brownie points for whoever can see if something like this is > doable for our Shining Panda setup: > > https://github.com/mozilla/pdf.js-bot I think it's possible, and I can look into getting something set up, but by my calculations, we're currently using approximately all our free build time on Shiningpanda. So if we were to do this, we'd have to consider: - Paying Shiningpanda for extra build time. At $0.48 /hour, we're looking at a couple of cents for a test run (2-3 minutes). - Using another service to run the tests. Sympy's review bot is co-ordinated on Google appengine, and I think they run the tests on one of their own computers, with the results being automatically uploaded. The site says they run into the free Quota on appengine, though. - Scaling back our current tests - e.g. we could go back to only testing with multiple python versions every other day, which would free up a chunk of our build time. Or we could mark some tests as 'slow' and run them less frequently. Thanks, Thomas From takowl at gmail.com Wed May 2 08:23:31 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 2 May 2012 13:23:31 +0100 Subject: [IPython-dev] Brilliant buildbot triggering from PRs... In-Reply-To: References: Message-ID: On 2 May 2012 12:16, Thomas Kluyver wrote: > I think it's possible, and I can look into getting something set up, > but by my calculations, we're currently using approximately all our > free build time on Shiningpanda. So if we were to do this, we'd have > to consider: On second thoughts, let's start with something simple. I'll put together a script so we can run `ipy-test-pr 1657`, and it will put the test log on gist. Then if we want to make it more automated, we can adapt the code later. I'll post some code today so others can collaborate on it. Thomas From fperez.net at gmail.com Wed May 2 09:11:28 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 2 May 2012 06:11:28 -0700 Subject: [IPython-dev] Brilliant buildbot triggering from PRs... In-Reply-To: References: Message-ID: On Wed, May 2, 2012 at 5:23 AM, Thomas Kluyver wrote: > On second thoughts, let's start with something simple. I'll put > together a script so we can run `ipy-test-pr 1657`, and it will put > the test log on gist. Then if we want to make it more automated, we > can adapt the code later. Awesome. Once it's up and running we'll look into the options regarding payment, etc. Matthew Brett also mentioned a plain buildbot as a possible alternative that's free, though it requires more manual setup. I'll let you lead on this, so let us know what you feel works best overall (you've done all the work here, so I don't want to suggest a specific option without really understanding the details). I'm about to go mostly offline for a few days, so don't wait for me on replies... Thanks! f From takowl at gmail.com Wed May 2 10:43:57 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 2 May 2012 15:43:57 +0100 Subject: [IPython-dev] Brilliant buildbot triggering from PRs... In-Reply-To: References: Message-ID: On 2 May 2012 13:23, Thomas Kluyver wrote: > On second thoughts, let's start with something simple. I'll put > together a script so we can run `ipy-test-pr 1657`, and it will put > the test log on gist. I've got a basic version of this working, see PR 1685: https://github.com/ipython/ipython/pull/1685 Thomas From jorgen.stenarson at kroywen.se Wed May 2 12:42:15 2012 From: jorgen.stenarson at kroywen.se (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Wed, 02 May 2012 18:42:15 +0200 Subject: [IPython-dev] spurious output during testing? Message-ID: <4FA163E7.2040902@kroywen.se> When I run the testsuite I get a bunch of messages during the IPython.parallell test: .[Application] keys mismatch [Application] Table _ exists and doesn't match db format, trying __1 Is this expected? If not what could be wrong in my setup? I run the tests in a virtualenv with only the necessary packages installed. /J?rgen From benjaminrk at gmail.com Wed May 2 14:05:56 2012 From: benjaminrk at gmail.com (MinRK) Date: Wed, 2 May 2012 11:05:56 -0700 Subject: [IPython-dev] spurious output during testing? In-Reply-To: <4FA163E7.2040902@kroywen.se> References: <4FA163E7.2040902@kroywen.se> Message-ID: On Wed, May 2, 2012 at 9:42 AM, J?rgen Stenarson < jorgen.stenarson at kroywen.se> wrote: > When I run the testsuite I get a bunch of messages during the > IPython.parallell test: > > .[Application] keys mismatch > [Application] Table _ exists and doesn't match db format, trying __1 > > Is this expected? If not what could be wrong in my setup? > It is probably due to stale files in the iptest profile. If you clear it out (profile_iptest/tasks.db), then it should stop. A permanent fix would be to change the tests to use a fully temporary profile_dir. -MinRK > > I run the tests in a virtualenv with only the necessary packages installed. > > /J?rgen > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed May 2 19:02:36 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 2 May 2012 16:02:36 -0700 Subject: [IPython-dev] Great slides on Parallel IPython from CU Boulder, no idea who made these... Message-ID: https://www.rc.colorado.edu/tutorials/parallelipython I have to give a talk tomorrow precisely on this at MSU East Lansing, and frankly, I might borrow a few slides from this :) We should link them from the site. If anyone knows who's behind this, please let us know so we can get explicit approval to host a copy and credit them! Cheers, f From fperez.net at gmail.com Wed May 2 19:03:26 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 2 May 2012 16:03:26 -0700 Subject: [IPython-dev] Brilliant buildbot triggering from PRs... In-Reply-To: References: Message-ID: On Wed, May 2, 2012 at 7:43 AM, Thomas Kluyver wrote: > > I've got a basic version of this working, see PR 1685: > > https://github.com/ipython/ipython/pull/1685 Fantastic. I won't have time to review until I return to California, feel free to move without me. Cheers, f From ellisonbg at gmail.com Wed May 2 23:13:53 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 2 May 2012 20:13:53 -0700 Subject: [IPython-dev] Great slides on Parallel IPython from CU Boulder, no idea who made these... In-Reply-To: References: Message-ID: Wow these are great! On Wed, May 2, 2012 at 4:02 PM, Fernando Perez wrote: > https://www.rc.colorado.edu/tutorials/parallelipython > > I have to give a talk tomorrow precisely on this at MSU East Lansing, > and frankly, I might borrow a few slides from this :) > > We should link them from the site. ?If anyone knows who's behind this, > please let us know so we can get explicit approval to host a copy and > credit them! > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jtaylor.debian at googlemail.com Thu May 3 16:14:12 2012 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Thu, 03 May 2012 22:14:12 +0200 Subject: [IPython-dev] pyside, pyqt import testing question Message-ID: <4FA2E714.8060903@googlemail.com> Hi, Debian received this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=671363 In short: the ipython-qtconsole package depends on qt or pyside. pyside is split into several components in Debian and sometime people don't have all installed. Debian packages can't depend on pyqt or (pyside.x and pyside.y) so the package uses this as a workaround to not pull everything: Depends: python-qt4 | python-pyside.qtcore, python-qt4 | python-pyside.qtgui, python-qt4 | python-pyside.qtsvg, This has the issue that when pyqt is installed but the pyside installation is incomplete ipython qtconsole won't start: File "/usr/lib/python2.7/dist-packages/IPython/external/qt.py", line 62, in from PySide import QtCore, QtGui, QtSvg external/qt.py only checks if import PySide succeeds and the fixes itself to that, even if PySide turns out to be incomplete later and pyqt would available as a fallback. Why is the check split up into two parts like that? Would there be any issues in making a complete PySide import test before settling on it? As this is a Debian only issue I'm fine with caring a Debian only patch for it if it will not have any unforeseen consequences. Cheers, Julian Taylor -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From takowl at gmail.com Thu May 3 16:48:24 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 3 May 2012 21:48:24 +0100 Subject: [IPython-dev] pyside, pyqt import testing question In-Reply-To: <4FA2E714.8060903@googlemail.com> References: <4FA2E714.8060903@googlemail.com> Message-ID: On 3 May 2012 21:14, Julian Taylor wrote: > external/qt.py only checks if import PySide succeeds and the fixes > itself to that, even if PySide turns out to be incomplete later and pyqt > would available as a fallback. > > Why is the check split up into two parts like that? > Would there be any issues in making a complete PySide import test before > settling on it? I think it makes sense to fix it in IPython so it checks for all the parts it needs, and I don't think that would cause any problems. I guess when it was written, we hadn't considered the possibility of PySide being partially installed. Thomas From ellisonbg at gmail.com Thu May 3 18:55:27 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 3 May 2012 15:55:27 -0700 Subject: [IPython-dev] Long awaited for JavaScript refactor Message-ID: Hi, I have posted a PR: https://github.com/ipython/ipython/pull/1697 that does some long awaited for refactoring of the JavaScript code for the Notebook. With this change, it should be possible for JavaScript widgets to call back into the kernel. I don't have an example yet but that is coming. There are a few PR's that this conflicts with: https://github.com/ipython/ipython/pull/1509 https://github.com/ipython/ipython/pull/1417 Because 1697 is such a major change, I would like to merge it first and then merge 1509/1417. I think 1697 will also help clean up these other two PRs. #1697 has significant potential to break the notebook, so it will need extensive testing. Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Thu May 3 19:56:44 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 3 May 2012 16:56:44 -0700 Subject: [IPython-dev] pyside, pyqt import testing question In-Reply-To: References: <4FA2E714.8060903@googlemail.com> Message-ID: On Thu, May 3, 2012 at 1:48 PM, Thomas Kluyver wrote: > On 3 May 2012 21:14, Julian Taylor wrote: > > external/qt.py only checks if import PySide succeeds and the fixes > > itself to that, even if PySide turns out to be incomplete later and pyqt > > would available as a fallback. > > > > Why is the check split up into two parts like that? > > Would there be any issues in making a complete PySide import test before > > settling on it? > > I think it makes sense to fix it in IPython so it checks for all the > parts it needs, and I don't think that would cause any problems. I > guess when it was written, we hadn't considered the possibility of > PySide being partially installed. > Yes, it's as simple as this - I had no idea that you could get less than all of PySide (a valueless source of complexity, but something we should deal with, I suppose). Our PySide import check should not only try to import PySide, but import everything we need from PySide. > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Fri May 4 18:33:51 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 04 May 2012 17:33:51 -0500 Subject: [IPython-dev] documenting new packages Message-ID: <4FA4594F.6080902@creativetrax.com> I found this page very helpful to get a high-level overview of the organization: http://ipython.org/ipython-doc/dev/development/reorg.html However, it's missing several directories: nbformat, parallel, testing, and zmq. If someone wants to reply with a sentence or two describing these directories, I can submit a pull request for the docs. Or if someone wants to just that to the docs directly, even better! Also, I'm trying to somehow view every message that goes back and forth to clients. I can start the kernel with --debug and see the execute_request/reply messages. But how do I see the stdout messages, etc., going from the kernel back to the frontend? Thanks, Jason From pivanov314 at gmail.com Fri May 4 18:51:37 2012 From: pivanov314 at gmail.com (Paul Ivanov) Date: Fri, 4 May 2012 15:51:37 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: <4FA4594F.6080902@creativetrax.com> References: <4FA4594F.6080902@creativetrax.com> Message-ID: Hey Jason, On Fri, May 4, 2012 at 3:33 PM, Jason Grout wrote: > Also, I'm trying to somehow view every message that goes back and forth > to clients. ?I can start the kernel with --debug and see the > execute_request/reply messages. ?But how do I see the stdout messages, > etc., going from the kernel back to the frontend? In python, get a kernel manager looking on the right session (see [1] for an example on how to do that), and then use: msgs = km.sub_channel.get_msgs(). Now, iterate over the msgs to figure out what type it is, and what contents it has. In vim-ipython, I do it like this [2] 1. https://github.com/ivanov/vim-ipython/blob/master/ftplugin/python/ipy.vim#L63 2. https://github.com/ivanov/vim-ipython/blob/master/ftplugin/python/ipy.vim#L281 best, -- Paul Ivanov 314 address only used for lists, ?off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 From pivanov314 at gmail.com Fri May 4 20:07:37 2012 From: pivanov314 at gmail.com (Paul Ivanov) Date: Fri, 4 May 2012 17:07:37 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: <4FA4594F.6080902@creativetrax.com> References: <4FA4594F.6080902@creativetrax.com> Message-ID: On Fri, May 4, 2012 at 3:33 PM, Jason Grout wrote: > I found this page very helpful to get a high-level overview of the > organization: > > http://ipython.org/ipython-doc/dev/development/reorg.html > > However, it's missing several directories: nbformat, parallel, testing, > and zmq. ?If someone wants to reply with a sentence or two describing > these directories, I can submit a pull request for the docs. ?Or if > someone wants to just that to the docs directly, even better! Thanks for point this out, Jason, I'm working on adding these changes right now. -- Paul Ivanov 314 address only used for lists, ?off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 From pivanov314 at gmail.com Fri May 4 21:37:59 2012 From: pivanov314 at gmail.com (Paul Ivanov) Date: Fri, 4 May 2012 18:37:59 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: References: <4FA4594F.6080902@creativetrax.com> Message-ID: On Fri, May 4, 2012 at 5:07 PM, Paul Ivanov wrote: > On Fri, May 4, 2012 at 3:33 PM, Jason Grout wrote: >> I found this page very helpful to get a high-level overview of the >> organization: >> >> http://ipython.org/ipython-doc/dev/development/reorg.html >> >> However, it's missing several directories: nbformat, parallel, testing, >> and zmq. ?If someone wants to reply with a sentence or two describing >> these directories, I can submit a pull request for the docs. ?Or if >> someone wants to just that to the docs directly, even better! > > Thanks for point this out, Jason, I'm working on adding these changes right now. Thanks, again, Jason, this is now in [1] 1. https://github.com/ipython/ipython/commit/6d4f6b56 -- Paul Ivanov 314 address only used for lists, ?off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 From jason-sage at creativetrax.com Fri May 4 22:10:41 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 04 May 2012 21:10:41 -0500 Subject: [IPython-dev] documenting new packages In-Reply-To: References: <4FA4594F.6080902@creativetrax.com> Message-ID: <4FA48C21.9040605@creativetrax.com> On 5/4/12 8:37 PM, Paul Ivanov wrote: > Thanks, again, Jason, this is now in [1] > > 1.https://github.com/ipython/ipython/commit/6d4f6b56 No, thank *you*. I now know that I should have ignored the kernel directory and concentrated on the zmq directory. Hopefully we'll have some pull requests coming soon to implement custom user messages. Or at least a message or two to the mailing list discussing design of such things. Thanks, Jason From jason-sage at creativetrax.com Fri May 4 22:14:33 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 04 May 2012 21:14:33 -0500 Subject: [IPython-dev] documenting new packages In-Reply-To: References: <4FA4594F.6080902@creativetrax.com> Message-ID: <4FA48D09.8010208@creativetrax.com> On 5/4/12 8:37 PM, Paul Ivanov wrote: > On Fri, May 4, 2012 at 5:07 PM, Paul Ivanov wrote: >> On Fri, May 4, 2012 at 3:33 PM, Jason Grout wrote: >>> I found this page very helpful to get a high-level overview of the >>> organization: >>> >>> http://ipython.org/ipython-doc/dev/development/reorg.html >>> >>> However, it's missing several directories: nbformat, parallel, testing, >>> and zmq. If someone wants to reply with a sentence or two describing >>> these directories, I can submit a pull request for the docs. Or if >>> someone wants to just that to the docs directly, even better! >> >> Thanks for point this out, Jason, I'm working on adding these changes right now. > > Thanks, again, Jason, this is now in [1] > > 1. https://github.com/ipython/ipython/commit/6d4f6b56 I'm curious: how often is the documentation at http://ipython.org/ipython-doc/dev/development/reorg.html refreshed? I thought maybe it was continually regenerated to reflect the latest commits, but I don't see your changes there. Thanks, Jason From jason-sage at creativetrax.com Fri May 4 22:16:05 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 04 May 2012 21:16:05 -0500 Subject: [IPython-dev] documenting new packages In-Reply-To: <4FA48D09.8010208@creativetrax.com> References: <4FA4594F.6080902@creativetrax.com> <4FA48D09.8010208@creativetrax.com> Message-ID: <4FA48D65.2000707@creativetrax.com> On 5/4/12 9:14 PM, Jason Grout wrote: > On 5/4/12 8:37 PM, Paul Ivanov wrote: >> On Fri, May 4, 2012 at 5:07 PM, Paul Ivanov wrote: >>> On Fri, May 4, 2012 at 3:33 PM, Jason Grout wrote: >>>> I found this page very helpful to get a high-level overview of the >>>> organization: >>>> >>>> http://ipython.org/ipython-doc/dev/development/reorg.html >>>> >>>> However, it's missing several directories: nbformat, parallel, testing, >>>> and zmq. If someone wants to reply with a sentence or two describing >>>> these directories, I can submit a pull request for the docs. Or if >>>> someone wants to just that to the docs directly, even better! >>> >>> Thanks for point this out, Jason, I'm working on adding these changes right now. >> >> Thanks, again, Jason, this is now in [1] >> >> 1. https://github.com/ipython/ipython/commit/6d4f6b56 > > I'm curious: how often is the documentation at > > http://ipython.org/ipython-doc/dev/development/reorg.html > > refreshed? I thought maybe it was continually regenerated to reflect > the latest commits, but I don't see your changes there. Answering my own question, the bottom of that page says: Last updated on Apr 18, 2012. Does that mean it is refreshed manually? So I really should go to the source and generate my own docs if I really want cutting edge docs... Jason From benjaminrk at gmail.com Sat May 5 00:05:26 2012 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 May 2012 21:05:26 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: <4FA48D65.2000707@creativetrax.com> References: <4FA4594F.6080902@creativetrax.com> <4FA48D09.8010208@creativetrax.com> <4FA48D65.2000707@creativetrax.com> Message-ID: On Fri, May 4, 2012 at 7:16 PM, Jason Grout wrote: > On 5/4/12 9:14 PM, Jason Grout wrote: > > On 5/4/12 8:37 PM, Paul Ivanov wrote: > >> On Fri, May 4, 2012 at 5:07 PM, Paul Ivanov > wrote: > >>> On Fri, May 4, 2012 at 3:33 PM, Jason Grout< > jason-sage at creativetrax.com> wrote: > >>>> I found this page very helpful to get a high-level overview of the > >>>> organization: > >>>> > >>>> http://ipython.org/ipython-doc/dev/development/reorg.html > >>>> > >>>> However, it's missing several directories: nbformat, parallel, > testing, > >>>> and zmq. If someone wants to reply with a sentence or two describing > >>>> these directories, I can submit a pull request for the docs. Or if > >>>> someone wants to just that to the docs directly, even better! > >>> > >>> Thanks for point this out, Jason, I'm working on adding these changes > right now. > >> > >> Thanks, again, Jason, this is now in [1] > >> > >> 1. https://github.com/ipython/ipython/commit/6d4f6b56 > > > > I'm curious: how often is the documentation at > > > > http://ipython.org/ipython-doc/dev/development/reorg.html > > > > refreshed? I thought maybe it was continually regenerated to reflect > > the latest commits, but I don't see your changes there. > > Answering my own question, the bottom of that page says: > > Last updated on Apr 18, 2012. > That's the sphinx build date, I believe. > > Does that mean it is refreshed manually? So I really should go to the > source and generate my own docs if I really want cutting edge docs... > It's entered manually. I didn't even know this file existed, and it was last edited in October 2009. -MinRK > > Jason > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pivanov314 at gmail.com Sat May 5 04:22:03 2012 From: pivanov314 at gmail.com (Paul Ivanov) Date: Sat, 5 May 2012 01:22:03 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: References: <4FA4594F.6080902@creativetrax.com> <4FA48D09.8010208@creativetrax.com> <4FA48D65.2000707@creativetrax.com> Message-ID: On Fri, May 4, 2012 at 9:05 PM, MinRK wrote: > It's entered manually. ?I didn't even know this file existed, and it was > last edited in October 2009. Weird, that *is* what git blame and git log show, but was this a typo or does Brian have a time machine we don't know about? dc983636 (Brian E. Granger 2009-09-27 22:57:44 -0700) As of the 0.11 release of IPython, the top-level packages and modules have been completely reorganized. This section describes the purpose of the top-level IPython subpackages. Given that 0.11 wasn't released until 2011... -- Paul Ivanov 314 address only used for lists, ?off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 From jason-sage at creativetrax.com Sat May 5 05:36:28 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Sat, 05 May 2012 04:36:28 -0500 Subject: [IPython-dev] custom user messages from the kernel Message-ID: <4FA4F49C.80505@creativetrax.com> I submitted a pull request to implement custom messages from user's code (i.e., from the kernel). This allows a user to send messages to a custom frontend with a new msg_type (so that existing frontends just ignore the message, right?). https://github.com/ipython/ipython/pull/1700 The pull request is more of a request for comment than a request for merge. Specific questions are in the pull request description. Since I'm not sure how to start such code conversations (is just submitting a pull request enough?), I thought I'd ask here for comments too. Thanks, Jason From takowl at gmail.com Sat May 5 09:21:59 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 5 May 2012 14:21:59 +0100 Subject: [IPython-dev] Testing pull requests Message-ID: The script for testing pull requests is now in master. To use it, find the number of the pull request (the last part of the URL), and run: python tools/test_pr.py 1234 This will create a .ipy_pr_tests/ directory in your home folder, check out the project, merge the specified pull request, and run the tests for each version of Python you have installed. If you add a -p option, it will also post the results as a comment on the pull request. You'll need requests installed, and if you want to post the results, keyring. Both are available on PyPI. Thanks, Thomas From carl.input at gmail.com Sat May 5 15:48:52 2012 From: carl.input at gmail.com (Carl Smith) Date: Sat, 5 May 2012 20:48:52 +0100 Subject: [IPython-dev] Notebook: Interacting with JavaScript Values Message-ID: Hi all I heard that there was some work done to enable accessing JS values within the Notebook. I'm just trying some stuff with Chrome extensions and wondered if anyone could offer any input on stuff I might want to look at. Thanks P.S. The notebook is awesome. From martin.zmk at gmail.com Sat May 5 19:28:05 2012 From: martin.zmk at gmail.com (lecast) Date: Sat, 5 May 2012 16:28:05 -0700 (PDT) Subject: [IPython-dev] experiment - remote execution of jquery and d3 code into the browser via ipython In-Reply-To: <996EF879-CFEE-4083-886F-89FB72EA3E54@infoecho.net> References: <4F6722E3.8020109@continuum.io> <4F67357C.8030501@continuum.io> <4F6747D4.3040206@creativetrax.com> <4F67CED9.7080700@continuum.io> <996EF879-CFEE-4083-886F-89FB72EA3E54@infoecho.net> Message-ID: <1336260485904-4955237.post@n6.nabble.com> I really like how flexible D3 can be, although it will take me a while to learn it. And the number one feature is that you can modify the graphics very easily in your browser. So today I started writing a module that would 'bring' D3 to IPython notebook, even though my experience with that is rather minimal. Surprisingly it goes well so far. I figured a way to insert multiple figures and update them between cells. The only problem now is that I cannot save the results. All HTML injections get saved, SVG objects are not. Any ideas how to circumvent that limitation? Demo with three figures being populated with random dots. https://www.youtube.com/watch?v=IVlRT4wT7mM -- View this message in context: http://python.6.n6.nabble.com/experiment-remote-execution-of-jquery-and-d3-code-into-the-browser-via-ipython-tp4633053p4955237.html Sent from the IPython - Development mailing list archive at Nabble.com. From jason-sage at creativetrax.com Sat May 5 20:47:00 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Sat, 05 May 2012 19:47:00 -0500 Subject: [IPython-dev] experiment - remote execution of jquery and d3 code into the browser via ipython In-Reply-To: <1336260485904-4955237.post@n6.nabble.com> References: <4F6722E3.8020109@continuum.io> <4F67357C.8030501@continuum.io> <4F6747D4.3040206@creativetrax.com> <4F67CED9.7080700@continuum.io> <996EF879-CFEE-4083-886F-89FB72EA3E54@infoecho.net> <1336260485904-4955237.post@n6.nabble.com> Message-ID: <4FA5CA04.1080800@creativetrax.com> On 5/5/12 6:28 PM, lecast wrote: > I really like how flexible D3 can be, although it will take me a while to > learn it. And the number one feature is that you can modify the graphics > very easily in your browser. So today I started writing a module that would > 'bring' D3 to IPython notebook, even though my experience with that is > rather minimal. Surprisingly it goes well so far. I figured a way to insert > multiple figures and update them between cells. The only problem now is that > I cannot save the results. All HTML injections get saved, SVG objects are > not. Any ideas how to circumvent that limitation? > > Demo with three figures being populated with random dots. > https://www.youtube.com/watch?v=IVlRT4wT7mM I'm curious what you think of the recent pull request to refactor the javascript to make it easier for javascript to call back to python: https://github.com/ipython/ipython/pull/1697 Thanks, Jason From benjaminrk at gmail.com Sun May 6 03:25:12 2012 From: benjaminrk at gmail.com (MinRK) Date: Sun, 6 May 2012 00:25:12 -0700 Subject: [IPython-dev] ipynb test script Message-ID: In case anyone is interested, I tossed together a crude script to run through a notebook and compare the output with that stored in the notebook: https://gist.github.com/2620735 -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sun May 6 03:43:15 2012 From: benjaminrk at gmail.com (MinRK) Date: Sun, 6 May 2012 00:43:15 -0700 Subject: [IPython-dev] ipynb test script In-Reply-To: References: Message-ID: On Sun, May 6, 2012 at 12:25 AM, MinRK wrote: > In case anyone is interested, I tossed together a crude script to run > through a notebook and compare the output with that stored in the notebook: > > https://gist.github.com/2620735 > And a simpler version that just steps through notebooks and notifies you of exceptions (handy validator): https://gist.github.com/2620876 > > -MinRK > -------------- next part -------------- An HTML attachment was scrubbed... URL: From krastanov.stefan at gmail.com Sun May 6 06:53:57 2012 From: krastanov.stefan at gmail.com (krastanov.stefan at gmail.com) Date: Sun, 6 May 2012 12:53:57 +0200 Subject: [IPython-dev] ipynb test script In-Reply-To: References: Message-ID: This will be immensely useful for what I am doing with the sympy examples. Thank you. I also referenced the gists on this feature request https://github.com/ipython/ipython/issues/1675 From krastanov.stefan at gmail.com Sun May 6 09:22:22 2012 From: krastanov.stefan at gmail.com (krastanov.stefan at gmail.com) Date: Sun, 6 May 2012 15:22:22 +0200 Subject: [IPython-dev] ipynb test script In-Reply-To: References: Message-ID: How can I force these scripts to use a certain configuration profile? Just adding extra_arguments=['--profile=blahblah'] to kernel_start does not seem to work. From krastanov.stefan at gmail.com Sun May 6 09:42:33 2012 From: krastanov.stefan at gmail.com (krastanov.stefan at gmail.com) Date: Sun, 6 May 2012 15:42:33 +0200 Subject: [IPython-dev] ipynb test script In-Reply-To: References: Message-ID: > How can I force these scripts to use a certain configuration profile? > Just adding extra_arguments=['--profile=blahblah'] to kernel_start > does not seem to work. It works. The error was in my understanding of how profiles work. Again, thanks for the script. From fperez.net at gmail.com Mon May 7 01:23:31 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 6 May 2012 22:23:31 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: References: <4FA4594F.6080902@creativetrax.com> <4FA48D09.8010208@creativetrax.com> <4FA48D65.2000707@creativetrax.com> Message-ID: On Sat, May 5, 2012 at 1:22 AM, Paul Ivanov wrote: > Weird, that *is* what git blame and git log show, but was this a typo or > does Brian have a time machine we don't know about? > > dc983636 (Brian E. Granger 2009-09-27 22:57:44 -0700) > > ? ?As of the 0.11 release of IPython, the top-level packages and modules have > ? ?been completely reorganized. ?This section describes the purpose of the > ? ?top-level IPython subpackages. > > Given that 0.11 wasn't released until 2011... That reorganization was done for 0.11 in 2009, so Brian's message really meant "for what will become 0.11 when released". For a while we had 0.10 and 0.11.dev living in parallel, with all new work going into the .11 dev branch. But since it took so long for all that massive amount of work to stabilize, we kept the 0.10 branch alive in the meantime to give users a stable but bugfixed source of updates. To answer Jason's question, unfortunately that document simply has fallen out of maintenance. I wish we had right now the manpower for a significant documentation update sprint, but we just don't at the moment (especially with some other burning concerns on our plate). Cheers, f From jason-sage at creativetrax.com Mon May 7 09:54:05 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Mon, 07 May 2012 08:54:05 -0500 Subject: [IPython-dev] documenting new packages In-Reply-To: References: <4FA4594F.6080902@creativetrax.com> <4FA48D09.8010208@creativetrax.com> <4FA48D65.2000707@creativetrax.com> Message-ID: <4FA7D3FD.9000407@creativetrax.com> On 5/7/12 12:23 AM, Fernando Perez wrote: > On Sat, May 5, 2012 at 1:22 AM, Paul Ivanov wrote: >> Weird, that *is* what git blame and git log show, but was this a typo or >> does Brian have a time machine we don't know about? >> >> dc983636 (Brian E. Granger 2009-09-27 22:57:44 -0700) >> >> As of the 0.11 release of IPython, the top-level packages and modules have >> been completely reorganized. This section describes the purpose of the >> top-level IPython subpackages. >> >> Given that 0.11 wasn't released until 2011... > > That reorganization was done for 0.11 in 2009, so Brian's message > really meant "for what will become 0.11 when released". For a while > we had 0.10 and 0.11.dev living in parallel, with all new work going > into the .11 dev branch. But since it took so long for all that > massive amount of work to stabilize, we kept the 0.10 branch alive in > the meantime to give users a stable but bugfixed source of updates. > > To answer Jason's question, unfortunately that document simply has > fallen out of maintenance. I wish we had right now the manpower for a > significant documentation update sprint, but we just don't at the > moment (especially with some other burning concerns on our plate). > I understand. Ultimately, we have the code, which is great. As long as you don't mind if we pepper the list with questions every now and then... It looks like Paul updated that particular page, so that's great! Jason From fperez.net at gmail.com Mon May 7 11:20:22 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 7 May 2012 08:20:22 -0700 Subject: [IPython-dev] documenting new packages In-Reply-To: <4FA7D3FD.9000407@creativetrax.com> References: <4FA4594F.6080902@creativetrax.com> <4FA48D09.8010208@creativetrax.com> <4FA48D65.2000707@creativetrax.com> <4FA7D3FD.9000407@creativetrax.com> Message-ID: On Mon, May 7, 2012 at 6:54 AM, Jason Grout wrote: > I understand. ?Ultimately, we have the code, which is great. ?As long as > you don't mind if we pepper the list with questions every now and then... No, not at all! And hopefully such questions will highlight the areas most in need of updating, so we can put resources in there... > It looks like Paul updated that particular page, so that's great! just like Paul did this time :) Thanks, Paul! Cheers, f From ellisonbg at gmail.com Mon May 7 12:42:20 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 7 May 2012 09:42:20 -0700 Subject: [IPython-dev] Notebook: Interacting with JavaScript Values In-Reply-To: References: Message-ID: We don't support accessing Javascript values from Python. Here is a rough sketch of what we will soon support (under review): * You can write Python objects that generate JavaScript code. * That JavaScript code can create widgets and UI elements on the notebook page. * That JavaScript code can execute further Python code by calling the IPython kernel. In those calls it can pass the values of JavaScript variables. Here is the pull request where this work is going on: https://github.com/ipython/ipython/pull/1697 Cheers, Brian On Sat, May 5, 2012 at 12:48 PM, Carl Smith wrote: > Hi all > > I heard that there was some work done to enable accessing JS values > within the Notebook. I'm just trying some stuff with Chrome extensions > and wondered if anyone could offer any input on stuff I might want to > look at. > > Thanks > > P.S. The notebook is awesome. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Mon May 7 19:13:39 2012 From: benjaminrk at gmail.com (MinRK) Date: Mon, 7 May 2012 16:13:39 -0700 Subject: [IPython-dev] roadmap and wishlist milestones Message-ID: Anyone object to my creating two new milestones on GitHub? 1. wishlist - feature requests, etc. with no concrete plans for implementation. Often a good place to start for newcomers who want to do something neat that wouldn't be done otherwise (at least not soon). 2. roadmap - Various things that we file under 'definitely planning to do, but probably not the next release (or two)'. Currently either/both of these are handled by the state "no milestone". Our use of milestones right now is quite lax, so I wouldn't consider them particularly authoritative, but I think it might help us a little bit when keeping track of our plans. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon May 7 19:17:46 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 8 May 2012 00:17:46 +0100 Subject: [IPython-dev] roadmap and wishlist milestones In-Reply-To: References: Message-ID: On 8 May 2012 00:13, MinRK wrote: > 1. wishlist - feature requests, etc. with no concrete plans for > implementation. ?Often a good place to start for newcomers who want to do > something neat that wouldn't be done otherwise (at least not soon). IMO this makes more sense as a tag than a milestone, but I won't bikeshed about it - if you think it should be a milestone, go for it. Thomas From benjaminrk at gmail.com Mon May 7 19:26:43 2012 From: benjaminrk at gmail.com (MinRK) Date: Mon, 7 May 2012 16:26:43 -0700 Subject: [IPython-dev] roadmap and wishlist milestones In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 4:17 PM, Thomas Kluyver wrote: > On 8 May 2012 00:13, MinRK wrote: > > 1. wishlist - feature requests, etc. with no concrete plans for > > implementation. Often a good place to start for newcomers who want to do > > something neat that wouldn't be done otherwise (at least not soon). > > IMO this makes more sense as a tag than a milestone, but I won't > bikeshed about it - if you think it should be a milestone, go for it. > I mainly say milestone, because that is how most 'real' bugtrackers I have seen deal with this sort of thing (of course, most such things don't have tags). This is somewhat redundant with our 'status' tags, but I don't think they are particularly effective at communicating anything. I would essentially like to replace 'status-dormant' with milestone=wishlist, and drop status-active. Note that we came up with our tagging system before GitHub had milestones. > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vickyscowcroft at gmail.com Mon May 7 19:29:39 2012 From: vickyscowcroft at gmail.com (vickys84) Date: Mon, 7 May 2012 16:29:39 -0700 (PDT) Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: <1336433379782-4959051.post@n6.nabble.com> I realise this is an old thread but I was getting the same error with a newer version of ipython with an EDP distribution. As this is the top search result on google I thought it would be helpful to post the solution here. [TerminalIPythonApp] Error in loading extension: kernmagic Check your config files in /Users/vs/.ipython/profile_default --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /Users/vs/env/normal/lib/python2.7/site-packages/IPython/core/extensions.pyc in load_extension(self, module_str) 85 if module_str not in sys.modules: 86 with prepended_to_syspath(self.ipython_extension_dir): ---> 87 __import__(module_str) 88 mod = sys.modules[module_str] 89 return self._call_load_ipython_extension(mod) ImportError: No module named kernmagic For anyone still having this problem you should comment out the line: c.TerminalIPythonApp.extensions = ['kernmagic'] in your ipython_config.py file. Hope this helps -- View this message in context: http://python.6.n6.nabble.com/Problem-with-0-12-beta-tp1647243p4959051.html Sent from the IPython - Development mailing list archive at Nabble.com. From fperez.net at gmail.com Mon May 7 21:02:48 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 7 May 2012 18:02:48 -0700 Subject: [IPython-dev] roadmap and wishlist milestones In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 4:26 PM, MinRK wrote: > This is somewhat redundant with our 'status' tags, but I don't think they > are particularly effective at communicating anything. ?I would essentially > like to replace 'status-dormant' with milestone=wishlist, and drop > status-active. ?Note that we came up with our tagging system before GitHub > had milestones. > No objection from me, as long as you go ahead and also remove those labels (perhaps write a script that maps the labeled issues to these new milestones). Cheers, f From ellisonbg at gmail.com Mon May 7 23:01:27 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 7 May 2012 20:01:27 -0700 Subject: [IPython-dev] roadmap and wishlist milestones In-Reply-To: References: Message-ID: I have stopped using the status-active and status-dormant tags for the notebook related issues because they were redundant with the information in the priority tags. I like the idea os getting rid of these and using milestones+priority instead. On Mon, May 7, 2012 at 4:13 PM, MinRK wrote: > Anyone object to my creating two new milestones on GitHub? > > 1. wishlist - feature requests, etc. with no concrete plans for > implementation. ?Often a good place to start for newcomers who want to do > something neat that wouldn't be done otherwise (at least not soon). I tend to like to minimize these types of issues to begin with. I like to have issues that have some chance of being acted on in a finite amount of time. But I realize that some people like to track everything and I can just ignore the wishlist. > 2. roadmap - Various things that we file under 'definitely planning to do, > but probably not the next release (or two)'. This is fine, but... > Currently either/both of these are handled by the state "no milestone". I am also fine with this less formal approach. > Our use of milestones right now is quite lax, so I wouldn't consider them > particularly authoritative, but I think it might help us a little bit when > keeping track of our plans. In my mind, it is OK to be lax on distant milestones - it is mainly the upcoming one that is worth being accurate on. But all of these feelings are rather weak, so I am fine with any decision along these lines. Cheers, Brian > -MinRK > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Tue May 8 00:53:21 2012 From: benjaminrk at gmail.com (MinRK) Date: Mon, 7 May 2012 21:53:21 -0700 Subject: [IPython-dev] roadmap and wishlist milestones In-Reply-To: References: Message-ID: Based on responses, I just converted all status-dormant Issues to milestone:wishlist, and removed the status tags. The only reason for 'roadmap' is do distinguish No Milestone as "we haven't decided" from wishlist=not right now, and roadmap=n+1 or n+2, but not n. I think it is probably simpler to just have n, n+1, and wishlist. I think in general, we should mark issues we should still *consider* resolving for the current release with the current milestone, and then as we get close to cutting it, we can make specific 'not yet' decisions, and start reassigning to n+1. I think we are at about that point now, where we can start going through many of the 0.13 issues and assigning them to 0.14 as we figure out how to focus our efforts towards release. Current breakdown of open issues: Milestones: 0.13: 76 0.14: 10 wishlist: 96 no milestone: 120 (including ~30 PRs, so closer to 90) Of those with no milestone: enhancement: 54 - many of these likely belong in wishlist bug: 31 -MinRK On Mon, May 7, 2012 at 8:01 PM, Brian Granger wrote: > I have stopped using the status-active and status-dormant tags for the > notebook related issues because they were redundant with the > information in the priority tags. I like the idea os getting rid of > these and using milestones+priority instead. > > On Mon, May 7, 2012 at 4:13 PM, MinRK wrote: > > Anyone object to my creating two new milestones on GitHub? > > > > 1. wishlist - feature requests, etc. with no concrete plans for > > implementation. Often a good place to start for newcomers who want to do > > something neat that wouldn't be done otherwise (at least not soon). > > I tend to like to minimize these types of issues to begin with. I > like to have issues that have some chance of being acted on in a > finite amount of time. But I realize that some people like to track > everything and I can just ignore the wishlist. > > > 2. roadmap - Various things that we file under 'definitely planning to > do, > > but probably not the next release (or two)'. > > This is fine, but... > > > Currently either/both of these are handled by the state "no milestone". > > I am also fine with this less formal approach. > > > Our use of milestones right now is quite lax, so I wouldn't consider them > > particularly authoritative, but I think it might help us a little bit > when > > keeping track of our plans. > > In my mind, it is OK to be lax on distant milestones - it is mainly > the upcoming one that is worth being accurate on. > > But all of these feelings are rather weak, so I am fine with any > decision along these lines. > > Cheers, > > Brian > > > -MinRK > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue May 8 01:55:11 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 7 May 2012 22:55:11 -0700 Subject: [IPython-dev] roadmap and wishlist milestones In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 8:01 PM, Brian Granger wrote: > I > like to have issues that have some chance of being acted on in a > finite amount of time. ?But I realize that some people like to track > everything and I can just ignore the wishlist. This kind of list can be a great entry point for *new* contributors, who may not necessarily want to dive into 'critical path' issues that core developers may already be working on. So there's real value in these, even if it has the somewhat unpleasant side effect of maintaining a high count of open issues that are unlikely to go down anytime soon. f From jason-sage at creativetrax.com Tue May 8 12:16:19 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Tue, 08 May 2012 11:16:19 -0500 Subject: [IPython-dev] Launching new kernels by forking existing kernel Message-ID: <4FA946D3.5070003@creativetrax.com> Hi all, (I'm resending this since I don't see it appearing on the list since I sent it yesterday; my apologies if someone gets it twice.) As part of the effort to rewrite the Sage Cell server [1] to better use the new IPython capabilities, we are trying to figure out how to emulate this behavior: In our current implementation, when a user requests a computation, the request goes to a worker process manager. The manager creates a temporary directory, forks off a worker process, and then the worker process executes the computation and returns results. After the worker process is done, the manager cleans up the directory, etc. The goal of our use-case is hundreds of short-lived, constantly regenerated kernels with almost zero startup cost (which means it is really bad to start a kernel from scratch, as it takes some time to import all the Sage modules). Each kernel interacts with a client for a short period of time (seconds to minutes). Poking around a bit, I'm wondering if this is pretty similar to some of the launchers in the parallel/launcher.py file. Would it be natural to implement our start-a-new-kernel-by-forking procedure when we receive a message as a launcher like in that file? Also, is it easy to redirect a kernel's zmq channels (after we fork, I presume we need to hook the new kernel up to separate channels with the client.) Thanks, Jason [1] aleph.sagemath.org. See also http://sage.math.washington.edu/home/jason/sagecell/index.html From ellisonbg at gmail.com Tue May 8 14:33:54 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 8 May 2012 11:33:54 -0700 Subject: [IPython-dev] Launching new kernels by forking existing kernel In-Reply-To: <4FA946D3.5070003@creativetrax.com> References: <4FA946D3.5070003@creativetrax.com> Message-ID: Jason, Whenever we fork processes using pyzmq, we have to create a new context and sockets after the fork. IOW, pyzmq context and sockets cannot move over fork boundary. I think this means that it will be essentially impossible to do what you are wanting to. Cheers, Brian On Tue, May 8, 2012 at 9:16 AM, Jason Grout wrote: > Hi all, > > (I'm resending this since I don't see it appearing on the list since I > sent it yesterday; my apologies if someone gets it twice.) > > As part of the effort to rewrite the Sage Cell server [1] to better use > the new IPython capabilities, we are trying to figure out how to emulate > this behavior: > > In our current implementation, when a user requests a computation, the > request goes to a worker process manager. ?The manager creates a > temporary directory, forks off a worker process, and then the worker > process executes the computation and returns results. ?After the worker > process is done, the manager cleans up the directory, etc. ?The goal of > our use-case is hundreds of short-lived, constantly regenerated kernels > with almost zero startup cost (which means it is really bad to start a > kernel from scratch, as it takes some time to import all the Sage > modules). ?Each kernel interacts with a client for a short period of > time (seconds to minutes). > > Poking around a bit, I'm wondering if this is pretty similar to some of > the launchers in the parallel/launcher.py file. ?Would it be natural to > implement our start-a-new-kernel-by-forking procedure when we receive a > message as a launcher like in that file? ?Also, is it easy to redirect a > kernel's zmq channels (after we fork, I presume we need to hook the new > kernel up to separate channels with the client.) > > Thanks, > > Jason > > [1] aleph.sagemath.org. ?See also > http://sage.math.washington.edu/home/jason/sagecell/index.html > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Tue May 8 15:04:46 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 8 May 2012 12:04:46 -0700 Subject: [IPython-dev] Launching new kernels by forking existing kernel In-Reply-To: References: <4FA946D3.5070003@creativetrax.com> Message-ID: On Tue, May 8, 2012 at 11:33 AM, Brian Granger wrote: > Jason, > > Whenever we fork processes using pyzmq, we have to create a new > context and sockets after the fork. IOW, pyzmq context and sockets > cannot move over fork boundary. I think this means that it will be > essentially impossible to do what you are wanting to. > I think that depends on what he means by this: > Also, is it easy to redirect a kernel's zmq channels (after we fork, I presume we need to hook the new kernel up to separate channels with the client.) If there is some kernel-launching facility that is forking off processes, and those forked processes then create the context and sockets for the kernel, then this should work just fine. What won't work is passing context and/or sockets across the fork boundary. This is the behavior we currently have: 1. KernelManager writes connection file. 2. KM starts Kernel via subprocess.Popen, passing the connection file via the command-line. If you replace step 2 with: 2a. KM sends special request to KernelStarter, including connection file (either via network, or in-process, I'm not sure of the architecture). 2b. KernelStarter starts kernel via *fork*, passing connection file, and being careful to avoid the subprocess inheriting zmq objects. Then I don't see any reason that shouldn't work. -MinRK > > Cheers, > > Brian > > On Tue, May 8, 2012 at 9:16 AM, Jason Grout > wrote: > > Hi all, > > > > (I'm resending this since I don't see it appearing on the list since I > > sent it yesterday; my apologies if someone gets it twice.) > > > > As part of the effort to rewrite the Sage Cell server [1] to better use > > the new IPython capabilities, we are trying to figure out how to emulate > > this behavior: > > > > In our current implementation, when a user requests a computation, the > > request goes to a worker process manager. The manager creates a > > temporary directory, forks off a worker process, and then the worker > > process executes the computation and returns results. After the worker > > process is done, the manager cleans up the directory, etc. The goal of > > our use-case is hundreds of short-lived, constantly regenerated kernels > > with almost zero startup cost (which means it is really bad to start a > > kernel from scratch, as it takes some time to import all the Sage > > modules). Each kernel interacts with a client for a short period of > > time (seconds to minutes). > > > > Poking around a bit, I'm wondering if this is pretty similar to some of > > the launchers in the parallel/launcher.py file. Would it be natural to > > implement our start-a-new-kernel-by-forking procedure when we receive a > > message as a launcher like in that file? Also, is it easy to redirect a > > kernel's zmq channels (after we fork, I presume we need to hook the new > > kernel up to separate channels with the client.) > > > > Thanks, > > > > Jason > > > > [1] aleph.sagemath.org. See also > > http://sage.math.washington.edu/home/jason/sagecell/index.html > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Tue May 8 17:02:47 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Tue, 08 May 2012 16:02:47 -0500 Subject: [IPython-dev] Launching new kernels by forking existing kernel In-Reply-To: References: <4FA946D3.5070003@creativetrax.com> Message-ID: <4FA989F7.3080007@creativetrax.com> On 5/8/12 2:04 PM, MinRK wrote: > > > On Tue, May 8, 2012 at 11:33 AM, Brian Granger > wrote: > > Jason, > > Whenever we fork processes using pyzmq, we have to create a new > context and sockets after the fork. IOW, pyzmq context and sockets > cannot move over fork boundary. I think this means that it will be > essentially impossible to do what you are wanting to. > > > I think that depends on what he means by this: > >> Also, is it easy to redirect a kernel's zmq channels (after we fork, I > presume we need to hook the new kernel up to separate channels with the > client.) > > If there is some kernel-launching facility that is forking off > processes, and those forked processes then create the context and > sockets for the kernel, then this should work just fine. What won't > work is passing context and/or sockets across the fork boundary. > > This is the behavior we currently have: > > 1. KernelManager writes connection file. > 2. KM starts Kernel via subprocess.Popen, passing the connection file > via the command-line. > > If you replace step 2 with: > > 2a. KM sends special request to KernelStarter, including connection file > (either via network, or in-process, I'm not sure of the architecture). > 2b. KernelStarter starts kernel via *fork*, passing connection file, and > being careful to avoid the subprocess inheriting zmq objects. That's similar to how we do things now (wrapping ipython 0.10). We have a managing process that receives a message, forks, and then the worker creates a new context (actually, with exactly the same zmq socket). What I'm trying to do is see if this model fits into the framework of the launchers already built into IPython (I'm trying to reuse the architecture and design of IPython as much as is reasonable, and contribute back where I can). So if I understand things correctly, the managing process (the one that is forking, if everything is local) is the controller, and the forked worker is an engine. The code in apps/launcher.py looks very much like the sorts of things I want to do---launch a controller or some engines. Is that where I should begin experimenting, making a subclass of LocalProcessLauncher? Thanks for you patience while I find my way around the code. Jason From benjaminrk at gmail.com Tue May 8 17:08:52 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 8 May 2012 14:08:52 -0700 Subject: [IPython-dev] Launching new kernels by forking existing kernel In-Reply-To: <4FA989F7.3080007@creativetrax.com> References: <4FA946D3.5070003@creativetrax.com> <4FA989F7.3080007@creativetrax.com> Message-ID: On Tue, May 8, 2012 at 2:02 PM, Jason Grout wrote: > On 5/8/12 2:04 PM, MinRK wrote: > > > > > > On Tue, May 8, 2012 at 11:33 AM, Brian Granger > > wrote: > > > > Jason, > > > > Whenever we fork processes using pyzmq, we have to create a new > > context and sockets after the fork. IOW, pyzmq context and sockets > > cannot move over fork boundary. I think this means that it will be > > essentially impossible to do what you are wanting to. > > > > > > I think that depends on what he means by this: > > > >> Also, is it easy to redirect a kernel's zmq channels (after we fork, I > > presume we need to hook the new kernel up to separate channels with the > > client.) > > > > If there is some kernel-launching facility that is forking off > > processes, and those forked processes then create the context and > > sockets for the kernel, then this should work just fine. What won't > > work is passing context and/or sockets across the fork boundary. > > > > This is the behavior we currently have: > > > > 1. KernelManager writes connection file. > > 2. KM starts Kernel via subprocess.Popen, passing the connection file > > via the command-line. > > > > If you replace step 2 with: > > > > 2a. KM sends special request to KernelStarter, including connection file > > (either via network, or in-process, I'm not sure of the architecture). > > 2b. KernelStarter starts kernel via *fork*, passing connection file, and > > being careful to avoid the subprocess inheriting zmq objects. > > That's similar to how we do things now (wrapping ipython 0.10). We have > a managing process that receives a message, forks, and then the worker > creates a new context (actually, with exactly the same zmq socket). > What I'm trying to do is see if this model fits into the framework of > the launchers already built into IPython (I'm trying to reuse the > architecture and design of IPython as much as is reasonable, and > contribute back where I can). > > So if I understand things correctly, the managing process (the one that > is forking, if everything is local) is the controller, and the forked > worker is an engine. The code in apps/launcher.py looks very much like > the sorts of things I want to do---launch a controller or some engines. > Is that where I should begin experimenting, making a subclass of > LocalProcessLauncher? > You can definitely try subclassing the LocalProcessLauncher to use fork instead of subprocess. I'm not sure how valuable that will be, given how much if the ProcessLauncher stuff is based on passing cl-args to a subprocess, but it's worth a look. > > Thanks for you patience while I find my way around the code. > > Jason > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed May 9 00:06:33 2012 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 8 May 2012 21:06:33 -0700 Subject: [IPython-dev] Automated pull request testing Message-ID: Hi, Remembering the previous thread, I just noticed this: http://about.travis-ci.org/blog/announcing-pull-request-support/ So, not yet, for free, but soonish, I imagine, See you, Matthew From takowl at gmail.com Wed May 9 09:42:01 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 9 May 2012 14:42:01 +0100 Subject: [IPython-dev] Automated pull request testing In-Reply-To: References: Message-ID: On 9 May 2012 05:06, Matthew Brett wrote: > Remembering the previous thread, I just noticed this: > > http://about.travis-ci.org/blog/announcing-pull-request-support/ Yep, that looks broadly similar to what we've now got to run locally, except that (for now) we need to run one command to start a test run. If we want, we could create a simple daemon that would periodically check the open pull requests and test them. In fact, our solution already beats Travis's in one minor respect: they will only test a mergeable pull request. Our tool will merge before testing if possible, but can test an unmergeable pull request if you want to (the difference is mentioned in the results). Thomas From fperez.net at gmail.com Wed May 9 19:55:10 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 9 May 2012 16:55:10 -0700 Subject: [IPython-dev] Testing pull requests In-Reply-To: References: Message-ID: Hey, On Sat, May 5, 2012 at 6:21 AM, Thomas Kluyver wrote: > The script for testing pull requests is now in master. To use it, find > the number of the pull request (the last part of the URL), and run: > > python tools/test_pr.py 1234 I'm getting this error: (master)longs[ipython]> python tools/test_pr.py 1710 Traceback (most recent call last): File "", line 1, in ImportError: No module named nose Traceback (most recent call last): File "", line 1, in ImportError: No module named nose Any idea why? (I do have nose installed). Cheers, f From takowl at gmail.com Wed May 9 20:02:24 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 10 May 2012 01:02:24 +0100 Subject: [IPython-dev] Testing pull requests In-Reply-To: References: Message-ID: On 10 May 2012 00:55, Fernando Perez wrote: > Traceback (most recent call last): > ?File "", line 1, in > ImportError: No module named nose > > Any idea why? ?(I do have nose installed). I get one copy of that for Python 2.6 - now that 2.6 is no longer supported in Ubuntu, the distribution package python-nose doesn't cover it. I don't know why you'd get two. For each version of Python, it should be simply trying to run: python2.7 -c "import nose" Thomas From fperez.net at gmail.com Wed May 9 20:17:49 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 9 May 2012 17:17:49 -0700 Subject: [IPython-dev] Testing pull requests In-Reply-To: References: Message-ID: On Wed, May 9, 2012 at 5:02 PM, Thomas Kluyver wrote: > I get one copy of that for Python 2.6 - now that 2.6 is no longer > supported in Ubuntu, the distribution package python-nose doesn't > cover it. I don't know why you'd get two. For each version of Python, > it should be simply trying to run: Ah, I didn't have nose for python3.2. That's now fixed, but oddly, it's still confused and tells me that it can't test with python 2.7: Not available for testing: python2.6, python2.7, python3.1 Obviously it does work, since I can run the test suite myself just fine. Any thoughts? f From takowl at gmail.com Thu May 10 05:39:28 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 10 May 2012 10:39:28 +0100 Subject: [IPython-dev] Testing pull requests In-Reply-To: References: Message-ID: On 10 May 2012 01:17, Fernando Perez wrote: > Obviously it does work, since I can run the test suite myself just > fine. ?Any thoughts? If you make a script with these contents: import subprocess subprocess.check_call(['python2.7', '-c', 'import nose']) What do you get from running it (with no virtualenv active)? That should be precisely the test it uses to check whether Python 2.7 is available. Thomas From carl.input at gmail.com Thu May 10 09:18:29 2012 From: carl.input at gmail.com (Carl Smith) Date: Thu, 10 May 2012 14:18:29 +0100 Subject: [IPython-dev] Notebook: Interacting with JavaScript Values In-Reply-To: References: Message-ID: Thanks Brian. I've been playing with the new code. The notebook's really cool. Thanks again. On May 7, 2012 5:42 PM, "Brian Granger" wrote: > We don't support accessing Javascript values from Python. Here is a > rough sketch of what we will soon support (under review): > > * You can write Python objects that generate JavaScript code. > * That JavaScript code can create widgets and UI elements on the notebook > page. > * That JavaScript code can execute further Python code by calling the > IPython kernel. In those calls it can pass the values of JavaScript > variables. > > Here is the pull request where this work is going on: > > https://github.com/ipython/ipython/pull/1697 > > Cheers, > > Brian > > On Sat, May 5, 2012 at 12:48 PM, Carl Smith wrote: > > Hi all > > > > I heard that there was some work done to enable accessing JS values > > within the Notebook. I'm just trying some stuff with Chrome extensions > > and wondered if anyone could offer any input on stuff I might want to > > look at. > > > > Thanks > > > > P.S. The notebook is awesome. > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu May 10 13:33:52 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 10 May 2012 10:33:52 -0700 Subject: [IPython-dev] Notebook: Interacting with JavaScript Values In-Reply-To: References: Message-ID: On Thu, May 10, 2012 at 6:18 AM, Carl Smith wrote: > Thanks Brian. I've been playing with the new code. The notebook's really > cool. Thanks again. Great, keep us posted on how it goes. Cheers, Brian > On May 7, 2012 5:42 PM, "Brian Granger" wrote: >> >> We don't support accessing Javascript values from Python. ?Here is a >> rough sketch of what we will soon support (under review): >> >> * You can write Python objects that generate JavaScript code. >> * That JavaScript code can create widgets and UI elements on the notebook >> page. >> * That JavaScript code can execute further Python code by calling the >> IPython kernel. ?In those calls it can pass the values of JavaScript >> variables. >> >> Here is the pull request where this work is going on: >> >> https://github.com/ipython/ipython/pull/1697 >> >> Cheers, >> >> Brian >> >> On Sat, May 5, 2012 at 12:48 PM, Carl Smith wrote: >> > Hi all >> > >> > I heard that there was some work done to enable accessing JS values >> > within the Notebook. I'm just trying some stuff with Chrome extensions >> > and wondered if anyone could offer any input on stuff I might want to >> > look at. >> > >> > Thanks >> > >> > P.S. The notebook is awesome. >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Fri May 11 02:00:36 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 May 2012 23:00:36 -0700 Subject: [IPython-dev] Testing pull requests In-Reply-To: References: Message-ID: On Thu, May 10, 2012 at 2:39 AM, Thomas Kluyver wrote: > > What do you get from running it (with no virtualenv active)? That > should be precisely the test it uses to check whether Python 2.7 is > available. Ah! It was picking up an old, stray install of python 2.7 from /usr/local (the original 2.7). That install didn't have nose, hence the error. Since that particular install didn't have nose installed, the error now makes sensse. Thanks! f From takowl at gmail.com Sat May 12 13:36:40 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 12 May 2012 18:36:40 +0100 Subject: [IPython-dev] Testing on Python 3.3 Message-ID: I've just tweaked our Shiningpanda build service: as of tonight, IPython will be built and tested on pre-release versions of Python 3.3, as well as the other Python versions we test against. I don't anticipate any major problems. To free up some build time, I've stopped a redundant test job (we had two jobs testing on Python 2.7). This one is now called ipython-quick, and can be manually triggered if the core developers want a test run during the day. Best wishes, Thomas From fperez.net at gmail.com Sat May 12 15:10:47 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 May 2012 12:10:47 -0700 Subject: [IPython-dev] Testing on Python 3.3 In-Reply-To: References: Message-ID: On Sat, May 12, 2012 at 10:36 AM, Thomas Kluyver wrote: > I've just tweaked our Shiningpanda build service: as of tonight, > IPython will be built and tested on pre-release versions of Python > 3.3, as well as the other Python versions we test against. I don't > anticipate any major problems. > > To free up some build time, I've stopped a redundant test job (we had > two jobs testing on Python 2.7). This one is now called ipython-quick, > and can be manually triggered if the core developers want a test run > during the day. Excellent! It will be great to be ready out of the gate when 3.3 comes out. Cheers, f From massimodisasha at gmail.com Sat May 12 16:40:14 2012 From: massimodisasha at gmail.com (Massimo Di Stefano) Date: Sat, 12 May 2012 16:40:14 -0400 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> Message-ID: <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> i'd love to try to generate the pdf , but i've this log : MacBook-Pro-di-Massimo:nbconvert epifanio$ python nbconvert.py -f latex IntroNumPy.ipynb Unknown format 'latex', known formats are: rst (default), html, quick-html any clue on what i'm doing wrong ? thanks! Massimo. Il giorno May 1, 2012, alle ore 8:42 PM, Fernando Perez ha scritto: > On Mon, Apr 30, 2012 at 7:51 AM, Matthias BUSSONNIER > wrote: >> You can use Inkscape as a command line tool on mac, you 'just' need >> to use the full path of a script inside the Inkscape bundle, which >> most of the time is : >> /Applications/Inkscape.app/Contents/Resources/bin/inkscape > > BTW, for those who may want to try this out, Matthias already made a > PR with this fix for OSX, and Stefan vdW fixed up also some of the > formatting I didn't finish last weekend, so now it's producing very > clean, tight output. We really should merge this soon into the > mainline repo, I hope to have a bit of time to clean it up so we can > proceed. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Sat May 12 16:52:25 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 May 2012 13:52:25 -0700 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> Message-ID: On Sat, May 12, 2012 at 1:40 PM, Massimo Di Stefano wrote: > MacBook-Pro-di-Massimo:nbconvert epifanio$ python nbconvert.py -f latex IntroNumPy.ipynb > Unknown format 'latex', known formats are: rst (default), html, quick-html > > any clue on what i'm doing wrong ? The code hasn't been merged into nbconvert master yet, so right now you need to use my branch: https://github.com/fperez/nbconvert/tree/latex-conv It's been sitting in the PR queue for a while without any review, so I might just go ahead and merge it soon. But in the meantime, just use my branch (I don't have time to look into the merge right now). Cheers, f From massimodisasha at gmail.com Sat May 12 19:08:07 2012 From: massimodisasha at gmail.com (Massimo Di Stefano) Date: Sat, 12 May 2012 19:08:07 -0400 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> Message-ID: git clone https://github.com/fperez/nbconvert.git gived me an nbconvert without latex support ? i had to manually copy and past the one from git browser :/ then running it i got this error : (process:7700): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. Traceback (most recent call last): File "nbconvert.py", line 712, in main(infile=args.infile[0], format=args.format) File "nbconvert.py", line 690, in main latexfname = converter.render() File "nbconvert.py", line 149, in render self.output = self.convert() File "nbconvert.py", line 514, in convert with open(os.path.join(os.path.dirname(myfile), 'preamble.tex')) as f: IOError: [Errno 2] No such file or directory: 'preamble.tex' Il giorno May 12, 2012, alle ore 4:52 PM, Fernando Perez ha scritto: > On Sat, May 12, 2012 at 1:40 PM, Massimo Di Stefano > wrote: >> MacBook-Pro-di-Massimo:nbconvert epifanio$ python nbconvert.py -f latex IntroNumPy.ipynb >> Unknown format 'latex', known formats are: rst (default), html, quick-html >> >> any clue on what i'm doing wrong ? > > The code hasn't been merged into nbconvert master yet, so right now > you need to use my branch: > > https://github.com/fperez/nbconvert/tree/latex-conv > > It's been sitting in the PR queue for a while without any review, so I > might just go ahead and merge it soon. But in the meantime, just use > my branch (I don't have time to look into the merge right now). > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Sat May 12 20:02:27 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 May 2012 17:02:27 -0700 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> Message-ID: On Sat, May 12, 2012 at 4:08 PM, Massimo Di Stefano wrote: > git clone https://github.com/fperez/nbconvert.git > > gived me an ?nbconvert without latex ?support > ? i had to manually copy and past the one from git browser :/ No, as I indicated above, the code is in the latex-conv branch, not in master. You need to check out that branch first if you want to use it: git checkout latex-conv Cheers, f From massimodisasha at gmail.com Sat May 12 20:07:20 2012 From: massimodisasha at gmail.com (Massimo Di Stefano) Date: Sat, 12 May 2012 20:07:20 -0400 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> Message-ID: <15AE12A1-160F-4D90-A95D-B58656B2A823@gmail.com> Thankjs Fernando! it give me the right nbserver, Tring it, latex is correctly detected as output format, but i still have the error below : No such file or directory: 'preamble.tex' (process:8259): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. Setting Language: .UTF-8 (process:8300): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. Setting Language: .UTF-8 (process:8343): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. Traceback (most recent call last): File "nbconvert.py", line 712, in main(infile=args.infile[0], format=args.format) File "nbconvert.py", line 690, in main latexfname = converter.render() File "nbconvert.py", line 149, in render self.output = self.convert() File "nbconvert.py", line 514, in convert with open(os.path.join(os.path.dirname(myfile), 'preamble.tex')) as f: IOError: [Errno 2] No such file or directory: 'preamble.tex' MacBook-Pro-di-Massimo:nbconvert epifanio$ Il giorno May 12, 2012, alle ore 8:02 PM, Fernando Perez ha scritto: > On Sat, May 12, 2012 at 4:08 PM, Massimo Di Stefano > wrote: >> git clone https://github.com/fperez/nbconvert.git >> >> gived me an nbconvert without latex support >> ? i had to manually copy and past the one from git browser :/ > > No, as I indicated above, the code is in the latex-conv branch, not in > master. You need to check out that branch first if you want to use > it: > > git checkout latex-conv > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Sat May 12 20:16:34 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 May 2012 17:16:34 -0700 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: <15AE12A1-160F-4D90-A95D-B58656B2A823@gmail.com> References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> <15AE12A1-160F-4D90-A95D-B58656B2A823@gmail.com> Message-ID: On Sat, May 12, 2012 at 5:07 PM, Massimo Di Stefano wrote: > Thankjs Fernando! > it give me the right nbserver, > > Tring it, latex is correctly detected as output format, but i still have the error below : > > No such file or directory: 'preamble.tex' Mmh, that's very odd. Could you do 'ls' in the git checkout directory? You should see this: [nbconvert]> ls decorators.py nbconvert.py* README.rst rst2ipynb.py* work_flow.txt dollarmath.py preamble.tex rst2ipynblib/ tests/ That's from a fresh clone I just made. And it works for me: blanca[junk]> ./nbconvert/nbconvert.py -f latex formatting.ipynb blanca[junk]> ls formatting_files/ formatting.ipynb formatting.tex nbconvert/ Untitled0.ipynb I'm kind of puzzled... From damianavila at gmail.com Sat May 12 20:21:14 2012 From: damianavila at gmail.com (=?windows-1252?Q?Dami=E1n_Avila?=) Date: Sat, 12 May 2012 21:21:14 -0300 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: <15AE12A1-160F-4D90-A95D-B58656B2A823@gmail.com> References: <64F8176B-E191-4C90-9BAE-18BBD7393706@gmail.com> <3E2D5CE3-391D-4850-9134-9B6D4BCD4CB4@gmail.com> <15AE12A1-160F-4D90-A95D-B58656B2A823@gmail.com> Message-ID: <4FAEFE7A.2030809@gmail.com> El 12/05/12 21:07, Massimo Di Stefano escribi?: > Thankjs Fernando! > it give me the right nbserver, > > Tring it, latex is correctly detected as output format, but i still have the error below : > > No such file or directory: 'preamble.tex' > > > > (process:8259): Gtk-WARNING **: Locale not supported by C library. > Using the fallback 'C' locale. > Setting Language: .UTF-8 > > (process:8300): Gtk-WARNING **: Locale not supported by C library. > Using the fallback 'C' locale. > Setting Language: .UTF-8 > > (process:8343): Gtk-WARNING **: Locale not supported by C library. > Using the fallback 'C' locale. > Traceback (most recent call last): > File "nbconvert.py", line 712, in > main(infile=args.infile[0], format=args.format) > File "nbconvert.py", line 690, in main > latexfname = converter.render() > File "nbconvert.py", line 149, in render > self.output = self.convert() > File "nbconvert.py", line 514, in convert > with open(os.path.join(os.path.dirname(myfile), 'preamble.tex')) as f: > IOError: [Errno 2] No such file or directory: 'preamble.tex' > MacBook-Pro-di-Massimo:nbconvert epifanio$ > > > Il giorno May 12, 2012, alle ore 8:02 PM, Fernando Perez ha scritto: > >> On Sat, May 12, 2012 at 4:08 PM, Massimo Di Stefano >> wrote: >>> git clone https://github.com/fperez/nbconvert.git >>> >>> gived me an nbconvert without latex support >>> ? i had to manually copy and past the one from git browser :/ >> No, as I indicated above, the code is in the latex-conv branch, not in >> master. You need to check out that branch first if you want to use >> it: >> >> git checkout latex-conv >> >> Cheers, >> >> f >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev Are you sure you are in the right branch? Delete your cloned repository and start again: |$ git clone https://github.com/fperez/nbconvert.git $ cd nbconvert|| |$ git checkout -b latex-conv origin/latex-conv $ git branch| * latex-conv master $ git pull | And let us know... Regards. Dami?n. -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimodisasha at gmail.com Sat May 12 20:44:34 2012 From: massimodisasha at gmail.com (Massimo Di Stefano) Date: Sat, 12 May 2012 20:44:34 -0400 Subject: [IPython-dev] High-quality pdf output (pdflatex) directly from notebooks In-Reply-To: References: Message-ID: <539AACEA-346D-4D82-8DFC-38CED26B89C4@gmail.com> It works fine :) Thanks! git lesson learned! i really appreciated your help guys! --Massimo. Il giorno Apr 30, 2012, alle ore 4:02 AM, Fernando Perez ha scritto: > Hi folks, > > This pdf file: > > http://archive.ipython.org/tmp/IntroNumPy.pdf > > was produced without a single manual change, straight from the > notebook in the tarball next to it. We're getting to the point where > we can really produce very nice PDFs straight from the notebook! > > If you want to see this in master, please come along and help with the > code review (it also has the instructions to reproduce it yourself): > > https://github.com/ipython/nbconvert/pull/12 > > I hope we'll have this moved over soon, so we can polish it further > and can make notebooks part of a regular publishing workflow. > > Ouch, my wrists hurt now. But it was a weekend well spent, I think... > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From erik.tollerud at gmail.com Mon May 14 05:25:28 2012 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Mon, 14 May 2012 02:25:28 -0700 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: This at first seemed to work, but I've discovered a test case where it fails. If I paste exactly what you have here into IPython, and then do ``raise ValueError('a message')``, it works exactly as expected. However, if instead I do ``raise Exception('a message')``, I get a traceback like what I've shown below. Further investigation has revealed that it only happens if I try to raise an exception of a class exactly the same as what's given in the first argument to set_custom_exc (Exception, in this case), while if I raise sub-exceptions, everything works just fine. Is this a bug, or am I mis-using it somehow? (if I execute the Min's example, and then do ``raise Exception``): ... the "hi " message repeats for the length of the recursion limit... hi Custom TB Handler failed, unregistering Custom TB Handler failed, unregistering Custom TB Handler failed, unregistering Custom TB Handler failed, unregistering ERROR: Internal Python error in the inspect module. Below is the traceback from this internal error. hi Traceback (most recent call last): File "/home/erik/src/ipython/IPython/core/ultratb.py", line 756, in structured_traceback Custom TB Handler failed, unregistering ERROR: Internal Python error in the inspect module. Below is the traceback from this internal error. Traceback (most recent call last): File "/home/erik/src/ipython/IPython/core/ultratb.py", line 756, in structured_traceback records = _fixed_getinnerframes(etb, context, tb_offset) File "/home/erik/src/ipython/IPython/core/ultratb.py", line 242, in _fixed_getinnerframes records = fix_frame_records_filenames(inspect.getinnerframes(etb, context)) File "/usr/lib/python2.7/inspect.py", line 1043, in getinnerframes framelist.append((tb.tb_frame,) + getframeinfo(tb, context)) File "/usr/lib/python2.7/inspect.py", line 1003, in getframeinfo filename = getsourcefile(frame) or getfile(frame) File "/usr/lib/python2.7/inspect.py", line 444, in getsourcefile filename = getfile(object) File "/usr/lib/python2.7/inspect.py", line 404, in getfile if isclass(object): File "/usr/lib/python2.7/inspect.py", line 65, in isclass return isinstance(object, (type, types.ClassType)) RuntimeError: maximum recursion depth exceeded while calling a Python object Unfortunately, your original traceback can not be constructed. The original exception: --------------------------------------------------------------------------- (, Exception(), )Traceback (most recent call last) (, Exception(), ): None On Mon, Apr 9, 2012 at 7:09 PM, MinRK wrote: > > > On Mon, Apr 9, 2012 at 18:39, Erik Tollerud wrote: >> >> Hello all, >> >> I noticed some odd (although I now realize quite sensible) behavior in >> IPython, and I'm curious if there's a workaround. ?I have an >> application log that I want to use to capture unhandled exceptions. >> When not using IPython, it's straightforward to just override >> sys.excepthook to first sent the exception to our logging system, and >> then continue the normal excepthook behavior. ?However, IPython seems >> to always use only its own excepthook in interactive mode >> (TerminalInteractiveShell.excepthook, at least in the standard >> terminal). >> >> So my question is: Is there any way to plug anything into the IPython >> exception handling system, either with sys.excepthook-style syntax, or >> some other way? ?As I said, it doesn't have to actually catch the >> exceptions - we just want entries placed in a log if an interactive >> session hits an unhandled exception, and it could then pass normally >> into the IPython formatting/pdb running system. > > > You use shell.set_custom_exc for this: > > # define custom exception handler: > > def custom_exc(shell, etype, evalue, tb, tb_offset=None): > ? ? # do you own thing: > ? ? print 'hi', etype > ? ? # also do what IPython would have done, if you want: > ? ? shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset) > > # Tell IPython to use it for any/all Exceptions: > get_ipython().set_custom_exc((Exception,), custom_exc) > >> >> -- >> Erik Tollerud >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Erik Tollerud From erik.tollerud at gmail.com Mon May 14 05:27:06 2012 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Mon, 14 May 2012 02:27:06 -0700 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: Further odd behavior that I'm not sure to classify as a bug or a feature. If I make a *script* to do this, the exact same thing happens. However, if I make a script that's identical, but raises ValueError (or any other subclass of Exception), it does *not* trigger the recursion loop, but also, the "hi" message does *not* appear - it seems to just use the standard traceback report and ignore the custom one. Is this intended? On Mon, Apr 9, 2012 at 7:09 PM, MinRK wrote: > > > On Mon, Apr 9, 2012 at 18:39, Erik Tollerud wrote: >> >> Hello all, >> >> I noticed some odd (although I now realize quite sensible) behavior in >> IPython, and I'm curious if there's a workaround. ?I have an >> application log that I want to use to capture unhandled exceptions. >> When not using IPython, it's straightforward to just override >> sys.excepthook to first sent the exception to our logging system, and >> then continue the normal excepthook behavior. ?However, IPython seems >> to always use only its own excepthook in interactive mode >> (TerminalInteractiveShell.excepthook, at least in the standard >> terminal). >> >> So my question is: Is there any way to plug anything into the IPython >> exception handling system, either with sys.excepthook-style syntax, or >> some other way? ?As I said, it doesn't have to actually catch the >> exceptions - we just want entries placed in a log if an interactive >> session hits an unhandled exception, and it could then pass normally >> into the IPython formatting/pdb running system. > > > You use shell.set_custom_exc for this: > > # define custom exception handler: > > def custom_exc(shell, etype, evalue, tb, tb_offset=None): > ? ? # do you own thing: > ? ? print 'hi', etype > ? ? # also do what IPython would have done, if you want: > ? ? shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset) > > # Tell IPython to use it for any/all Exceptions: > get_ipython().set_custom_exc((Exception,), custom_exc) > >> >> -- >> Erik Tollerud >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Erik Tollerud From Fernando.Perez at berkeley.edu Tue May 15 02:46:47 2012 From: Fernando.Perez at berkeley.edu (Fernando Perez) Date: Mon, 14 May 2012 23:46:47 -0700 Subject: [IPython-dev] An idea for `iypthon notebook` server CTRL+C (^C) In-Reply-To: <1336062225.98436.YahooMailClassic@web162302.mail.bf1.yahoo.com> References: <1336062225.98436.YahooMailClassic@web162302.mail.bf1.yahoo.com> Message-ID: Hi, many thanks for your kind words! I'm really sorry for not replying before, but unfortunately I'm eternally behind on my email. For this reason, I've forwarded your message to the ipython-dev mailing list, where we manage all ipython development activity. By having these discussions on-list, there's a higher chance that other developers can participate and contribute even if I'm not available at the time. You can sign up to the list here: http://mail.scipy.org/mailman/listinfo/ipython-dev Cheers, f On Thu, May 3, 2012 at 9:23 AM, Flubber wrote: > Good evening (19:12 / 7:12 PM) here. First of all, I wish you all my > respects and ... just ... speechless. Great work! A lot of times these > complex projects often have problems (obviously, because they are so > complex) but, I have to be honest with you, you guys have done an > outstanding job. > > I was watching the IPython in-depth video (YouTube URL: > http://www.youtube.com/watch?v=2G5YTlheCbw), there is mentioned at time: > 1:39:30 that if you press CTRL+C (KeyboardInterrupt as Python would mention > the exception to be) the server shutdown, but before this, you (Mr. > Fernando Perez) mentioned that the QT Console launches an side terminal in > which you will operate. Now, I have no idea if this was fixed in the newer > versions as I do not have a Tornado version bigger or equal to 2.1.0, but > if the server crashes upon CTRL+C (the `ipython notebook`), then, couldn't > you spawn the server in a mirror shell/terminal/console/whatever similar to > the QT one? > > For example in Linux if you want to start-up Kwrite editor and not keep > the terminal busy with it you execute: > $> kwrite & > > And you will get something like this: > $> kwrite & > [1] 2785 > $> ps ax | grep kwrite > 2785 pts/0 Sl 0:01 kwrite > > Where obviously after you execute `kwrite &`, the PID of the process that > has been forked/spawned will be reported in the terminal. As you can see > with the ps ax command. > > But the error reporting will still go in stdout in the terminal which > spawned. So, if you use kwrite & and an error occurs it will be reported > in the same terminal/console/shell that it spawned, which is kind of ugly > because it can overwrite your other commands (ls, cd whatever you do). > > This is just an idea or suggestion if you wish to call it like that. As I > said, I have no idea if you fixed it in the newer versions, if not, you > could spawn the server in a separate console where all the reporting and > errors are nicely displayed and does not overwrite the users space (from > the original console/terminal). > > Take care and keep up the good work, you guys are inspiring and just > awesome, I wish I had the balls (literally) to study so hard and do > projects like this. > > Best regards. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue May 15 06:42:06 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 15 May 2012 11:42:06 +0100 Subject: [IPython-dev] An idea for `iypthon notebook` server CTRL+C (^C) In-Reply-To: References: <1336062225.98436.YahooMailClassic@web162302.mail.bf1.yahoo.com> Message-ID: Hi, Flubber: > but if the server crashes upon CTRL+C (the `ipython notebook`), then, > couldn't you spawn the server in a mirror shell/terminal/console/whatever > similar to the QT one? I'm not quite clear about what you're suggesting. It's not a crash - pressing Ctrl+C in the terminal where you started the notebook is the way to shut it down. Ctrl+C within a notebook doesn't affect the server - the browser will handle it, generally as a copy command. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue May 15 08:13:43 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 15 May 2012 13:13:43 +0100 Subject: [IPython-dev] Fwd: [Ipython-build] Automatic IPython build reports Build failed in Jenkins: ipython-mac #107 In-Reply-To: <443275147.9.1337065346866.JavaMail.uipyth01@master3> References: <443275147.9.1337065346866.JavaMail.uipyth01@master3> Message-ID: We periodically get failure reports like this from the mac build bot. It looks like one or more of the xml files is an empty file ("Test report file /home/slave/jenkins/workspace/ipython-mac/results/IPython.parallel.xunit.xml was length 0"). When I re-run it (i.e. redownload the zip file), it works. Matthias, any ideas what might be causing it? Shiningpanda currently downloads your results at 9am French time - should I shift that time? Thanks, Thomas ---------- Forwarded message ---------- From: Date: 15 May 2012 08:02 Subject: [Ipython-build] Automatic IPython build reports Build failed in Jenkins: ipython-mac #107 To: ipython-build at lists.ipython.org See ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on Default Debian 6 - Squeeze .1 in workspace [ipython-mac] $ /bin/sh -xe /tmp/hudson2257515646098530057.sh + mkdir -p results + cd results + rm -f IPython.config.coverage.xml IPython.config.xunit.xml IPython.core.coverage.xml IPython.core.xunit.xml IPython.extensions.coverage.xml IPython.extensions.xunit.xml IPython.frontend.coverage.xml IPython.frontend.xunit.xml IPython.lib.coverage.xml IPython.lib.xunit.xml IPython.nbformat.coverage.xml IPython.nbformat.xunit.xml IPython.parallel.coverage.xml IPython.parallel.xunit.xml IPython.scripts.coverage.xml IPython.scripts.xunit.xml IPython.testing.coverage.xml IPython.testing.xunit.xml IPython.utils.coverage.xml IPython.utils.xunit.xml IPython.zmq.coverage.xml IPython.zmq.xunit.xml stderrandout.log test-latest.zip + wget -O test-latest.zip http://elacave.lmdb.eu/~carreau/ipd/test-latest.zip --2012-05-15 09:02:23-- ?http://elacave.lmdb.eu/~carreau/ipd/test-latest.zip Resolving elacave.lmdb.eu... 91.121.208.77, 2001:41d0:1:fd4d::1 Connecting to elacave.lmdb.eu|91.121.208.77|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 95865 (94K) [application/zip] Saving to: ?test-latest.zip? ? ? 0K .......... .......... .......... .......... .......... 53% 2.35M 0s ? ?50K .......... .......... .......... .......... ... ? ? ? 100% 5.22M=0.03s 2012-05-15 09:02:23 (3.16 MB/s) - ?test-latest.zip? saved [95865/95865] + unzip test-latest.zip Archive: ?test-latest.zip ?inflating: IPython.config.coverage.xml ?inflating: IPython.config.xunit.xml ?inflating: IPython.core.coverage.xml ?inflating: IPython.core.xunit.xml ?inflating: IPython.extensions.coverage.xml ?inflating: IPython.extensions.xunit.xml ?inflating: IPython.frontend.coverage.xml ?inflating: IPython.frontend.xunit.xml ?inflating: IPython.lib.coverage.xml ?inflating: IPython.lib.xunit.xml ?inflating: IPython.nbformat.coverage.xml ?inflating: IPython.nbformat.xunit.xml ?extracting: IPython.parallel.coverage.xml ?extracting: IPython.parallel.xunit.xml ?inflating: IPython.scripts.coverage.xml ?inflating: IPython.scripts.xunit.xml ?inflating: IPython.testing.coverage.xml ?inflating: IPython.testing.xunit.xml ?inflating: IPython.utils.coverage.xml ?inflating: IPython.utils.xunit.xml ?inflating: IPython.zmq.coverage.xml ?inflating: IPython.zmq.xunit.xml ?inflating: stderrandout.log + touch IPython.config.coverage.xml IPython.core.coverage.xml IPython.extensions.coverage.xml IPython.frontend.coverage.xml IPython.lib.coverage.xml IPython.nbformat.coverage.xml IPython.parallel.coverage.xml IPython.scripts.coverage.xml IPython.testing.coverage.xml IPython.utils.coverage.xml IPython.zmq.coverage.xml + touch IPython.config.xunit.xml IPython.core.xunit.xml IPython.extensions.xunit.xml IPython.frontend.xunit.xml IPython.lib.xunit.xml IPython.nbformat.xunit.xml IPython.parallel.xunit.xml IPython.scripts.xunit.xml IPython.testing.xunit.xml IPython.utils.xunit.xml IPython.zmq.xunit.xml Publishing Cobertura coverage report... Publishing Cobertura coverage results... FATAL: Unable to parse /sp/mnt/jenkins/ipython/jobs/ipython-mac/builds/2012-05-15_09-02-19/coverage6.xml hudson.util.IOException2: Cannot parse coverage results ? ? ? ?at hudson.plugins.cobertura.CoberturaCoverageParser.parse(CoberturaCoverageParser.java:89) ? ? ? ?at hudson.plugins.cobertura.CoberturaCoverageParser.parse(CoberturaCoverageParser.java:52) ? ? ? ?at hudson.plugins.cobertura.CoberturaPublisher.perform(CoberturaPublisher.java:265) ? ? ? ?at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36) ? ? ? ?at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705) ? ? ? ?at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680) ? ? ? ?at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:658) ? ? ? ?at hudson.model.Build$RunnerImpl.post2(Build.java:162) ? ? ? ?at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627) ? ? ? ?at hudson.model.Run.run(Run.java:1459) ? ? ? ?at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) ? ? ? ?at hudson.model.ResourceController.execute(ResourceController.java:88) ? ? ? ?at hudson.model.Executor.run(Executor.java:239) Caused by: org.xml.sax.SAXParseException: Premature end of file. ? ? ? ?at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195) ? ? ? ?at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174) ? ? ? ?at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388) ? ? ? ?at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414) ? ? ? ?at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:1059) ? ? ? ?at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648) ? ? ? ?at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511) ? ? ? ?at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808) ? ? ? ?at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) ? ? ? ?at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) ? ? ? ?at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205) ? ? ? ?at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522) ? ? ? ?at javax.xml.parsers.SAXParser.parse(SAXParser.java:395) ? ? ? ?at javax.xml.parsers.SAXParser.parse(SAXParser.java:198) ? ? ? ?at hudson.plugins.cobertura.CoberturaCoverageParser.parse(CoberturaCoverageParser.java:81) ? ? ? ?... 12 more Build step 'Publish Cobertura Coverage Report' changed build result to FAILURE Recording test results _______________________________________________ Ipython-build mailing list Ipython-build at lists.ipython.org http://lists.ipython.org/listinfo.cgi/ipython-build-ipython.org From bussonniermatthias at gmail.com Tue May 15 11:53:45 2012 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Tue, 15 May 2012 17:53:45 +0200 Subject: [IPython-dev] [Ipython-build] Automatic IPython build reports Build failed in Jenkins: ipython-mac #107 In-Reply-To: References: <443275147.9.1337065346866.JavaMail.uipyth01@master3> Message-ID: <289FBF37-2F74-4C56-935E-49E6F01369B2@gmail.com> Le 15 mai 2012 ? 14:13, Thomas Kluyver a ?crit : > We periodically get failure reports like this from the mac build bot. > It looks like one or more of the xml files is an empty file ("Test > report file /home/slave/jenkins/workspace/ipython-mac/results/IPython.parallel.xunit.xml > was length 0"). When I re-run it (i.e. redownload the zip file), it > works. > > Matthias, any ideas what might be causing it? Shiningpanda currently > downloads your results at 9am French time - should I shift that time? It should be ok. I had a strange problem this morning, computer stuck with hundreds of python process running and not able to kill them so I had to hard reboot, I wasn't able to look at the exact command line in ps, but I guess it is linked to the failure... i'll try to investigate. -- Matthias From takowl at gmail.com Tue May 15 13:09:17 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 15 May 2012 18:09:17 +0100 Subject: [IPython-dev] Website now built with Sphinx 1.1.3 Message-ID: I've updated the website, and it's now built with Sphinx 1.1.3. If everyone updating the website can check that their Sphinx version is up to date, it will keep things neat in the pages repository. Thanks, Thomas From fperez.net at gmail.com Tue May 15 14:42:37 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 May 2012 11:42:37 -0700 Subject: [IPython-dev] Website now built with Sphinx 1.1.3 In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 10:09 AM, Thomas Kluyver wrote: > I've updated the website, and it's now built with Sphinx 1.1.3. If > everyone updating the website can check that their Sphinx version is > up to date, it will keep things neat in the pages repository. Great, thanks! f From fperez.net at gmail.com Tue May 15 15:07:49 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 May 2012 12:07:49 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? Message-ID: Hi folks, in trying to field a question about the notebook from a colleague, I realized that I don't know anymore what are the earliest supported browser versions. When the notebook was just getting off the ground, it used to work with Firefox 4 (after manual configuration) but I know that the websockets protocol has evolved and I don't really know if FF4 even works anymore. Does anyone have a machine with FF4 they can test with, or know out right (perhaps Min or Brian actually know) the answer to this? Ditto with info on Chrome and Safari? We do know that IE will only work when IE 10 comes out, since IE9 has zero websockets, and Opera still seems to have some glitches. But it would be great to have an unambiguous answer for FFox, Chrome and Safari... Cheers, f From benjaminrk at gmail.com Tue May 15 15:26:05 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 15 May 2012 12:26:05 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 12:07 PM, Fernando Perez wrote: > Hi folks, > > in trying to field a question about the notebook from a colleague, I > realized that I don't know anymore what are the earliest supported > browser versions. When the notebook was just getting off the ground, > it used to work with Firefox 4 (after manual configuration) but I know > that the websockets protocol has evolved and I don't really know if > FF4 even works anymore. Even if FF4 works, I don't think we should support it. > > Does anyone have a machine with FF4 they can test with, or know out > right (perhaps Min or Brian actually know) the answer to this? Ditto > with info on Chrome and Safari? We do know that IE will only work > when IE 10 comes out, since IE9 has zero websockets, and Opera still > seems to have some glitches. But it would be great to have an > unambiguous answer for FFox, Chrome and Safari... With the current silent-update model for FF and Chrome, does minimum browser version matter any more? Can't we just say 'current'? Even if it technically works, I don't see any point in our supporting abandoned versions. If we really need a number, I would go with stable as of today (12 and 19, respectively), but I think 'current' is a better descriptor. Based on this table, the minimum numbers for websocket support without manual configuration should be: FF 6, Chrome 6, and Safari 5.0.1, but if anyone with a browser older than FF 12, Chrome 19, or Safari 5.1 has an issue, our answer should be to update your browser first, then come back and see us. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue May 15 16:25:16 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 May 2012 13:25:16 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 12:26 PM, MinRK wrote: > With the current silent-update model for FF and Chrome, does minimum browser > version matter any more? ?Can't we just say 'current'? ?Even if it > technically works, I don't see any point in our supporting abandoned > versions. ?If we really need a number, I would go with stable as of today > (12 and 19, respectively), but I think 'current' is a better descriptor. > > Based on?this table, the minimum numbers for websocket support without > manual configuration should be: FF 6, Chrome 6, and Safari 5.0.1, but if > anyone with a browser older than FF 12, Chrome 19, or Safari 5.1 has an > issue, our answer should be to update your browser first, then come back and > see us. Well, the question arose in the context of environments such as the DoD where all automatic upgrades are disabled. In such places, telling users to upgrade first doesn't work. Though we can obviously say "FF 4 is not supported, only FF10 or newer", we just need to be able to give them a clear answer on that one. For most 'normal' users I agree with you, but this is a legitimate question for certain environments (and given how these guys are currently funding some of our work, I do want to make sure we give them a clear answer). Cheers, f From benjaminrk at gmail.com Tue May 15 17:07:39 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 15 May 2012 14:07:39 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 1:25 PM, Fernando Perez wrote: > On Tue, May 15, 2012 at 12:26 PM, MinRK wrote: > > With the current silent-update model for FF and Chrome, does minimum > browser > > version matter any more? Can't we just say 'current'? Even if it > > technically works, I don't see any point in our supporting abandoned > > versions. If we really need a number, I would go with stable as of today > > (12 and 19, respectively), but I think 'current' is a better descriptor. > > > > Based on this table, the minimum numbers for websocket support without > > manual configuration should be: FF 6, Chrome 6, and Safari 5.0.1, but if > > anyone with a browser older than FF 12, Chrome 19, or Safari 5.1 has an > > issue, our answer should be to update your browser first, then come back > and > > see us. > > Well, the question arose in the context of environments such as the > DoD where all automatic upgrades are disabled. In such places, > telling users to upgrade first doesn't work. Though we can obviously > say "FF 4 is not supported, only FF10 or newer", we just need to be > able to give them a clear answer on that one. > > For most 'normal' users I agree with you, but this is a legitimate > question for certain environments (and given how these guys are > currently funding some of our work, I do want to make sure we give > them a clear answer). > Okay, that's fair. It's going to be untenable in the long run from a support perspective, though, as the rapid-update model makes it completely unreasonable for us to have test environments with the old browsers. -MinRK > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue May 15 17:17:22 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 15 May 2012 14:17:22 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 2:07 PM, MinRK wrote: > > > On Tue, May 15, 2012 at 1:25 PM, Fernando Perez > wrote: >> >> On Tue, May 15, 2012 at 12:26 PM, MinRK wrote: >> > With the current silent-update model for FF and Chrome, does minimum >> > browser >> > version matter any more? ?Can't we just say 'current'? ?Even if it >> > technically works, I don't see any point in our supporting abandoned >> > versions. ?If we really need a number, I would go with stable as of >> > today >> > (12 and 19, respectively), but I think 'current' is a better descriptor. >> > >> > Based on?this table, the minimum numbers for websocket support without >> > manual configuration should be: FF 6, Chrome 6, and Safari 5.0.1, but if >> > anyone with a browser older than FF 12, Chrome 19, or Safari 5.1 has an >> > issue, our answer should be to update your browser first, then come back >> > and >> > see us. >> >> Well, the question arose in the context of environments such as the >> DoD where all automatic upgrades are disabled. ?In such places, >> telling users to upgrade first doesn't work. ?Though we can obviously >> say "FF 4 is not supported, only FF10 or newer", we just need to be >> able to give them a clear answer on that one. >> >> For most 'normal' users I agree with you, but this is a legitimate >> question for certain environments (and given how these guys are >> currently funding some of our work, I do want to make sure we give >> them a clear answer). > > > Okay, that's fair. ?It's going to be untenable in the long run from a > support perspective, though, as the rapid-update model makes it completely > unreasonable for us to have test environments with the old browsers. I think we should distinguish between "support" and "might work". I agree with Min that we "support" the latest versions of FF, Chrome and Safari. Other browser versions might likely work, but if you have problems, we will first ask you to update your browser... Cheers, Brian > -MinRK > >> >> >> Cheers, >> >> f >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Tue May 15 18:52:33 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 May 2012 15:52:33 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 2:17 PM, Brian Granger wrote: > I think we should distinguish between "support" and "might work". ?I > agree with Min that we "support" the latest versions of FF, Chrome and > Safari. ?Other browser versions might likely work, but if you have > problems, we will first ask you to update your browser... Yes, and I think here I'm not asking for 'support' but rather for information. I think we basically want to be able to say "to the best of our knowledge, FF X and Chrome Y work well with IPython version 0.A.b". That doesn't mean no glitches will ever be found, but at least it's a reasonable starting point. Cheers, f From ellisonbg at gmail.com Tue May 15 19:17:52 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 15 May 2012 16:17:52 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 3:52 PM, Fernando Perez wrote: > On Tue, May 15, 2012 at 2:17 PM, Brian Granger wrote: >> I think we should distinguish between "support" and "might work". ?I >> agree with Min that we "support" the latest versions of FF, Chrome and >> Safari. ?Other browser versions might likely work, but if you have >> problems, we will first ask you to update your browser... > > Yes, and I think here I'm not asking for 'support' but rather for > information. ?I think we basically want to be able to say "to the best > of our knowledge, FF X and Chrome Y work well with IPython version > 0.A.b". ?That doesn't mean no glitches will ever be found, but at > least it's a reasonable starting point. Then I think the answer to that is what Min gave above: FF 6, Chrome 6, and Safari 5.0.1, > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Wed May 16 00:59:16 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 May 2012 21:59:16 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 4:17 PM, Brian Granger wrote: > Then I think the answer to that is what Min gave above: > > > FF 6, Chrome 6, and Safari 5.0.1, That's good enough for now, I think. Thanks! f From jenshnielsen at gmail.com Wed May 16 02:42:40 2012 From: jenshnielsen at gmail.com (Jens Nielsen) Date: Wed, 16 May 2012 08:42:40 +0200 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: Let me just add one point about Firefox support. Since Firefox before 10.0 is completely unsuported by mozilla and contains several security issues one should not recommend to use this. If an IT department wants to deploy a specific Firefox version they should go for the ESR (Extended support release) which is right now based on Firefox 10 and will be supported until Firefox 18 with security updates and then replaced by a new ESR based on Firefox 17. http://www.mozilla.org/en-US/firefox/organizations/faq/ Thus I think that the recommended Firefox version for the notebook should be either the last ESR or the latest release version and the notebook should ideally be tested against both of these. Cheers Jens On Wed, May 16, 2012 at 6:59 AM, Fernando Perez wrote: > On Tue, May 15, 2012 at 4:17 PM, Brian Granger > wrote: > > Then I think the answer to that is what Min gave above: > > > > > > FF 6, Chrome 6, and Safari 5.0.1, > > That's good enough for now, I think. Thanks! > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed May 16 02:46:54 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 May 2012 23:46:54 -0700 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: On Tue, May 15, 2012 at 11:42 PM, Jens Nielsen wrote: > Let me just add one point about Firefox support. Thanks for that datapoint. I'll relay it along, though we're not *recommending* those versions be used, rather stating minimal bars on what we know works. What they deploy on their systems is completely beyond our control in this case. Cheers, f From bussonniermatthias at gmail.com Wed May 16 05:00:43 2012 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Wed, 16 May 2012 11:00:43 +0200 Subject: [IPython-dev] [Ipython-build] Automatic IPython build reports Build failed in Jenkins: ipython-mac #107 In-Reply-To: References: <443275147.9.1337065346866.JavaMail.uipyth01@master3> Message-ID: <33D78ADA-55B6-4C36-91A6-96756C6E5EB3@gmail.com> Le 15 mai 2012 ? 14:13, Thomas Kluyver a ?crit : > We periodically get failure reports like this from the mac build bot. > It looks like one or more of the xml files is an empty file ("Test > report file /home/slave/jenkins/workspace/ipython-mac/results/IPython.parallel.xunit.xml > was length 0"). When I re-run it (i.e. redownload the zip file), it > works. > > Matthias, any ideas what might be causing it? Shiningpanda currently > downloads your results at 9am French time - should I shift that time? > It works when i'm running everything myself in the venv, so it might be a problem with the cron? the other issues I have is that wen iptest is run by the cron I've a python process that never ends : /.../venv/jenkins/bin/python -c from IPython import embed_kernel^Jdef go():^J a=5^J b="hi there"^J embed_kernel()^Jgo()^J As for the time, it is ok, and you can choose almost at anytime, the venv is totally cleaned and reinstall every day at 2, and a test of last IPython every 4 hours without reinstalling the all venv. I'll try to investigate, but i'll be away for a few days. -- Matthias > Thanks, > Thomas > > ---------- Forwarded message ---------- > From: > Date: 15 May 2012 08:02 > Subject: [Ipython-build] Automatic IPython build reports Build failed > in Jenkins: ipython-mac #107 > To: ipython-build at lists.ipython.org > > > See > > ------------------------------------------ > Started by timer > [EnvInject] - Loading node environment variables. > Building remotely on Default Debian 6 - Squeeze .1 in workspace > > [ipython-mac] $ /bin/sh -xe /tmp/hudson2257515646098530057.sh > + mkdir -p results > + cd results > + rm -f IPython.config.coverage.xml IPython.config.xunit.xml > IPython.core.coverage.xml IPython.core.xunit.xml > IPython.extensions.coverage.xml IPython.extensions.xunit.xml > IPython.frontend.coverage.xml IPython.frontend.xunit.xml > IPython.lib.coverage.xml IPython.lib.xunit.xml > IPython.nbformat.coverage.xml IPython.nbformat.xunit.xml > IPython.parallel.coverage.xml IPython.parallel.xunit.xml > IPython.scripts.coverage.xml IPython.scripts.xunit.xml > IPython.testing.coverage.xml IPython.testing.xunit.xml > IPython.utils.coverage.xml IPython.utils.xunit.xml > IPython.zmq.coverage.xml IPython.zmq.xunit.xml stderrandout.log > test-latest.zip > + wget -O test-latest.zip http://elacave.lmdb.eu/~carreau/ipd/test-latest.zip > --2012-05-15 09:02:23-- http://elacave.lmdb.eu/~carreau/ipd/test-latest.zip > Resolving elacave.lmdb.eu... 91.121.208.77, 2001:41d0:1:fd4d::1 > Connecting to elacave.lmdb.eu|91.121.208.77|:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: 95865 (94K) [application/zip] > Saving to: ?test-latest.zip? > > 0K .......... .......... .......... .......... .......... 53% 2.35M 0s > 50K .......... .......... .......... .......... ... 100% 5.22M=0.03s > > 2012-05-15 09:02:23 (3.16 MB/s) - ?test-latest.zip? saved [95865/95865] > > + unzip test-latest.zip > Archive: test-latest.zip > inflating: IPython.config.coverage.xml > inflating: IPython.config.xunit.xml > inflating: IPython.core.coverage.xml > inflating: IPython.core.xunit.xml > inflating: IPython.extensions.coverage.xml > inflating: IPython.extensions.xunit.xml > inflating: IPython.frontend.coverage.xml > inflating: IPython.frontend.xunit.xml > inflating: IPython.lib.coverage.xml > inflating: IPython.lib.xunit.xml > inflating: IPython.nbformat.coverage.xml > inflating: IPython.nbformat.xunit.xml > extracting: IPython.parallel.coverage.xml > extracting: IPython.parallel.xunit.xml > inflating: IPython.scripts.coverage.xml > inflating: IPython.scripts.xunit.xml > inflating: IPython.testing.coverage.xml > inflating: IPython.testing.xunit.xml > inflating: IPython.utils.coverage.xml > inflating: IPython.utils.xunit.xml > inflating: IPython.zmq.coverage.xml > inflating: IPython.zmq.xunit.xml > inflating: stderrandout.log > + touch IPython.config.coverage.xml IPython.core.coverage.xml > IPython.extensions.coverage.xml IPython.frontend.coverage.xml > IPython.lib.coverage.xml IPython.nbformat.coverage.xml > IPython.parallel.coverage.xml IPython.scripts.coverage.xml > IPython.testing.coverage.xml IPython.utils.coverage.xml > IPython.zmq.coverage.xml > + touch IPython.config.xunit.xml IPython.core.xunit.xml > IPython.extensions.xunit.xml IPython.frontend.xunit.xml > IPython.lib.xunit.xml IPython.nbformat.xunit.xml > IPython.parallel.xunit.xml IPython.scripts.xunit.xml > IPython.testing.xunit.xml IPython.utils.xunit.xml > IPython.zmq.xunit.xml > Publishing Cobertura coverage report... > Publishing Cobertura coverage results... > FATAL: Unable to parse > /sp/mnt/jenkins/ipython/jobs/ipython-mac/builds/2012-05-15_09-02-19/coverage6.xml > hudson.util.IOException2: Cannot parse coverage results > at hudson.plugins.cobertura.CoberturaCoverageParser.parse(CoberturaCoverageParser.java:89) > at hudson.plugins.cobertura.CoberturaCoverageParser.parse(CoberturaCoverageParser.java:52) > at hudson.plugins.cobertura.CoberturaPublisher.perform(CoberturaPublisher.java:265) > at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36) > at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705) > at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680) > at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:658) > at hudson.model.Build$RunnerImpl.post2(Build.java:162) > at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627) > at hudson.model.Run.run(Run.java:1459) > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) > at hudson.model.ResourceController.execute(ResourceController.java:88) > at hudson.model.Executor.run(Executor.java:239) > Caused by: org.xml.sax.SAXParseException: Premature end of file. > at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195) > at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174) > at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388) > at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414) > at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:1059) > at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648) > at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511) > at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808) > at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) > at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) > at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205) > at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522) > at javax.xml.parsers.SAXParser.parse(SAXParser.java:395) > at javax.xml.parsers.SAXParser.parse(SAXParser.java:198) > at hudson.plugins.cobertura.CoberturaCoverageParser.parse(CoberturaCoverageParser.java:81) > ... 12 more > Build step 'Publish Cobertura Coverage Report' changed build result to FAILURE > Recording test results > _______________________________________________ > Ipython-build mailing list > Ipython-build at lists.ipython.org > http://lists.ipython.org/listinfo.cgi/ipython-build-ipython.org From takowl at gmail.com Wed May 16 05:33:32 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 16 May 2012 10:33:32 +0100 Subject: [IPython-dev] [Ipython-build] Automatic IPython build reports Build failed in Jenkins: ipython-mac #107 In-Reply-To: <33D78ADA-55B6-4C36-91A6-96756C6E5EB3@gmail.com> References: <443275147.9.1337065346866.JavaMail.uipyth01@master3> <33D78ADA-55B6-4C36-91A6-96756C6E5EB3@gmail.com> Message-ID: On 16 May 2012 10:00, Matthias BUSSONNIER wrote: > It works when i'm running everything myself in the venv, so it might be a problem with the cron? > the other issues I have is that wen iptest is run by the cron I've a python process that never ends : > /.../venv/jenkins/bin/python -c from IPython import embed_kernel^Jdef go():^J ? ?a=5^J ? ?b="hi there"^J ? ?embed_kernel()^Jgo()^J It looks like that's a test in zmq/tests/test_embed_kernel.py - it starts a process to embed the kernel in, but never terminates it. It looks like it should be as simple as adding km.shutdown_kernel() to the context manager. > As for the time, it is ok, and you can choose almost at anytime, the venv is totally cleaned and reinstall every day at 2, > and a test of last IPython every 4 hours without reinstalling the all venv. > > I'll try to investigate, but i'll be away for a few days. Thanks. I'll leave it at 9 am for now, then. Thomas From damianavila at gmail.com Wed May 16 05:50:14 2012 From: damianavila at gmail.com (=?ISO-8859-1?Q?Dami=E1n_Avila?=) Date: Wed, 16 May 2012 06:50:14 -0300 Subject: [IPython-dev] Minimal versions of browsers for the notebook? In-Reply-To: References: Message-ID: <4FB37856.1020402@gmail.com> El 16/05/12 03:46, Fernando Perez escribi?: > On Tue, May 15, 2012 at 11:42 PM, Jens Nielsen wrote: >> Let me just add one point about Firefox support. > Thanks for that datapoint. I'll relay it along, though we're not > *recommending* those versions be used, rather stating minimal bars on > what we know works. What they deploy on their systems is completely > beyond our control in this case. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev FYI, the browsers based in webkit as a rendering engine (I am developing one specifically suited for IPython, but you have Epiphany, for example...) needs at least the 2.2 version of webkit (at least qt4.81 or PyQt4.91) to support the websocket protocol needed for run IPython Notebook. From takowl at gmail.com Wed May 16 08:10:29 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 16 May 2012 13:10:29 +0100 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: On 14 May 2012 10:25, Erik Tollerud wrote: > Further investigation has > revealed that it only happens if I try to raise an exception of a > class exactly the same as what's given in the first argument to > set_custom_exc (Exception, in this case), while if I raise > sub-exceptions, everything works just fine. ?Is this a bug, or am I > mis-using it somehow? Having worked out what's going on, I would consider it a bug, although fixing it changes the API slightly. I've made a pull request for it. https://github.com/ipython/ipython/pull/1742 Thomas From benjaminrk at gmail.com Wed May 16 11:54:07 2012 From: benjaminrk at gmail.com (Min RK) Date: Wed, 16 May 2012 08:54:07 -0700 Subject: [IPython-dev] [Ipython-build] Automatic IPython build reports Build failed in Jenkins: ipython-mac #107 In-Reply-To: References: <443275147.9.1337065346866.JavaMail.uipyth01@master3> <33D78ADA-55B6-4C36-91A6-96756C6E5EB3@gmail.com> Message-ID: <7B8D0F2B-5620-4C89-904F-EF1E587CB81D@gmail.com> On May 16, 2012, at 2:33 AM, Thomas Kluyver wrote: > On 16 May 2012 10:00, Matthias BUSSONNIER wrote: >> It works when i'm running everything myself in the venv, so it might be a problem with the cron? >> the other issues I have is that wen iptest is run by the cron I've a python process that never ends : >> /.../venv/jenkins/bin/python -c from IPython import embed_kernel^Jdef go():^J a=5^J b="hi there"^J embed_kernel()^Jgo()^J > > It looks like that's a test in zmq/tests/test_embed_kernel.py - it > starts a process to embed the kernel in, but never terminates it. It > looks like it should be as simple as adding km.shutdown_kernel() to > the context manager. I merged PR 1727 yesterday, which does exactly that. > >> As for the time, it is ok, and you can choose almost at anytime, the venv is totally cleaned and reinstall every day at 2, >> and a test of last IPython every 4 hours without reinstalling the all venv. >> >> I'll try to investigate, but i'll be away for a few days. > > Thanks. I'll leave it at 9 am for now, then. > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From takowl at gmail.com Wed May 16 15:14:16 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 16 May 2012 20:14:16 +0100 Subject: [IPython-dev] TryNext exceptions usage Message-ID: Hi all, Following on from discussion on pull request #1741, we're proposing to remove the ability for a TryNext exception to carry args and kwargs. This is part of the CommandChainDispatcher system, whereby a series of functions can be called until one *doesn't* raise TryNext. In principle, a function can raise TryNext with args/kwargs, which will replace the original arguments when the next function in the chain is called. In practice, there's nothing in the IPython codebase that does that, and it's hard to think of a use case. The principle of loose coupling suggests that a function shouldn't know what's next in the chain. Does anyone object to us removing that functionality (i.e. when a CommandChainDispatcher is called, each function would get the original args/kwargs)? In particular, do we know of any extension code that uses TryNext exceptions to change args/kwargs? Thanks, Thomas From carl.input at gmail.com Wed May 16 22:04:29 2012 From: carl.input at gmail.com (Carl Smith) Date: Thu, 17 May 2012 03:04:29 +0100 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: Hi I remember hearing that stdin in Notebook was something that you knew how to implement, but needed time to get done. I was just wondering what's going on with that. Recently, I've been playing around on EC2, where having Notebook as a fully functional, remote shell would be enormously useful. The current need to have a bunch of stuff installed on your local machine before you can actually use The Cloud is ridiculous. On 16 May 2012 13:10, Thomas Kluyver wrote: > On 14 May 2012 10:25, Erik Tollerud wrote: >> Further investigation has >> revealed that it only happens if I try to raise an exception of a >> class exactly the same as what's given in the first argument to >> set_custom_exc (Exception, in this case), while if I raise >> sub-exceptions, everything works just fine. ?Is this a bug, or am I >> mis-using it somehow? > > Having worked out what's going on, I would consider it a bug, although > fixing it changes the API slightly. I've made a pull request for it. > > https://github.com/ipython/ipython/pull/1742 > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Wed May 16 22:22:28 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 16 May 2012 19:22:28 -0700 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: On Wed, May 16, 2012 at 7:04 PM, Carl Smith wrote: > I remember hearing that stdin in Notebook was something that you knew > how to implement, but needed time to get done. I was just wondering > what's going on with that. Haven't had any time to work on that, I'm afraid. It's not super difficult, but it does require some knowledge of the networking bits. > Recently, I've been playing around on EC2, where having Notebook as a > fully functional, remote shell would be enormously useful. The current > need to have a bunch of stuff installed on your local machine before > you can actually use The Cloud is ridiculous. You can *sort of* use it already with !cmd the only thing that doesn't work is truly interactive stuff that requires stdin reading. For now, in a cloud scenario the only reasonable solution is to ssh into your notebook node and use ipython console --existing [json info] after getting the connection info with %connect_info. Fortunately starcluster makes the process of ssh-ing into your notebook node completely trivial, so it's not a completely absurd solution. Obviously in the long run we want real stdin in the notebook, I'm just offering a temporary workaround. And if you're not using starcluster for your EC2 setup, switch to it right away: it will save you a LOT of time and effort. Cheers, f From carl.input at gmail.com Sat May 19 13:05:10 2012 From: carl.input at gmail.com (Carl Smith) Date: Sat, 19 May 2012 18:05:10 +0100 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: Hi Fernando Thanks for your input. I've used StarCluster, its really good, but for what I'm trying to do, I found it easier to just use boto and do things by hand. I really want lots of small VMs, one per user, controlled by a single, larger server that exists to manage users and their VMs. Each user would have complete control over their own instance, but only via IPython. They wouldn't have/need anything on their local machine besides a decent browser. This is meant to be used to learn programming collaboratively. It's not very difficult to do what I'm looking into doing, but it'd not work well without stdin. I'm not familiar with IPython internals, so I guess I'll just have to be patient. It's not especially urgent anyway. Again, thanks for you reply and all the hard work. Carl On 17 May 2012 03:22, Fernando Perez wrote: > On Wed, May 16, 2012 at 7:04 PM, Carl Smith wrote: >> I remember hearing that stdin in Notebook was something that you knew >> how to implement, but needed time to get done. I was just wondering >> what's going on with that. > > Haven't had any time to work on that, I'm afraid. ?It's not super > difficult, but it does require some knowledge of the networking bits. > >> Recently, I've been playing around on EC2, where having Notebook as a >> fully functional, remote shell would be enormously useful. The current >> need to have a bunch of stuff installed on your local machine before >> you can actually use The Cloud is ridiculous. > > You can *sort of* use it already with > > !cmd > > the only thing that doesn't work is truly interactive stuff that > requires stdin reading. ?For now, in a cloud scenario the only > reasonable solution is to ssh into your notebook node and use > > ipython console --existing [json info] > > after getting the connection info with %connect_info. ?Fortunately > starcluster makes the process of ssh-ing into your notebook node > completely trivial, so it's not a completely absurd solution. > > Obviously in the long run we want real stdin in the notebook, I'm just > offering a temporary workaround. ?And if you're not using starcluster > for your EC2 setup, switch to it right away: it will save you a LOT of > time and effort. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Sat May 19 15:49:55 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 19 May 2012 12:49:55 -0700 Subject: [IPython-dev] excepthook-like behavior in ipython In-Reply-To: References: Message-ID: On Sat, May 19, 2012 at 10:05 AM, Carl Smith wrote: > It's not very difficult to do what I'm looking into doing, but it'd > not work well without stdin. I'm not familiar with IPython internals, > so I guess I'll just have to be patient. It's not especially urgent > anyway. Understood (and thanks for the boto pointer, worth keeping in mind). Hopefully it won't take us too long to get this done... Best f From takowl at gmail.com Tue May 22 14:23:24 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 22 May 2012 19:23:24 +0100 Subject: [IPython-dev] In-process kernel Message-ID: I was just thinking about something we've been discussing for a while, but not (as far as I know) got round to implementing. We'd like to implement an in-process kernel. This would be particularly valuable for GUI applications which want to embed the Qt console and expose objects within the application - at present, they're in the odd situation that they can have either the kernel or the frontend embedded in the application, but not both. Ideally this would be structured so that these applications wouldn't have an unnecessary dependency on zeromq, but that's not too important. I think the critical design decision is whether we run kernel functions synchronously - and block the application if some operation is taking a long time - or have a kernel thread, and risk the dangers of multi-threading (e.g. if the user tries to run commands that affect the GUI directly, Qt will likely segfault). We might end up with both to support different use cases, but which one do you think we should put together first? Thanks, Thomas From benjaminrk at gmail.com Tue May 22 14:46:00 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 22 May 2012 11:46:00 -0700 Subject: [IPython-dev] In-process kernel In-Reply-To: References: Message-ID: On Tue, May 22, 2012 at 11:23 AM, Thomas Kluyver wrote: > I was just thinking about something we've been discussing for a while, > but not (as far as I know) got round to implementing. We'd like to > implement an in-process kernel. This would be particularly valuable > for GUI applications which want to embed the Qt console and expose > objects within the application - at present, they're in the odd > situation that they can have either the kernel or the frontend > embedded in the application, but not both. Ideally this would be > structured so that these applications wouldn't have an unnecessary > dependency on zeromq, but that's not too important. > > I think the critical design decision is whether we run kernel > functions synchronously - and block the application if some operation > is taking a long time - or have a kernel thread, and risk the dangers > of multi-threading (e.g. if the user tries to run commands that affect > the GUI directly, Qt will likely segfault). We might end up with both > to support different use cases, but which one do you think we should > put together first? > The kernel has the ability to integrate into the Qt eventloop with %gui qt. Why shouldn't this enable both happily residing in the Qt loop without resorting to threads? -MinRK > > Thanks, > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue May 22 17:10:03 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 22 May 2012 22:10:03 +0100 Subject: [IPython-dev] In-process kernel In-Reply-To: References: Message-ID: On 22 May 2012 19:46, MinRK wrote: > The kernel has the ability to integrate into the Qt eventloop with %gui qt. > ?Why shouldn't this enable both happily residing in the Qt loop without > resorting to threads? As I understand it, this integration will still block the event loop if you run a command that takes a long time in the kernel. Thomas From benjaminrk at gmail.com Tue May 22 17:21:26 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 22 May 2012 14:21:26 -0700 Subject: [IPython-dev] In-process kernel In-Reply-To: References: Message-ID: On Tue, May 22, 2012 at 2:10 PM, Thomas Kluyver wrote: > On 22 May 2012 19:46, MinRK wrote: > > The kernel has the ability to integrate into the Qt eventloop with %gui > qt. > > Why shouldn't this enable both happily residing in the Qt loop without > > resorting to threads? > > As I understand it, this integration will still block the event loop > if you run a command that takes a long time in the kernel. > Ah, right. I would say that this is a normal, expected behavior, and the first stab should just run everything in a single Qt loop. -MinRK > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ischnell at enthought.com Wed May 23 19:13:28 2012 From: ischnell at enthought.com (Ilan Schnell) Date: Wed, 23 May 2012 18:13:28 -0500 Subject: [IPython-dev] ANN: EPD 7.3 (and 8 preview beta) released Message-ID: Hello, I am pleased to announce the release of Enthought Python Distribution, EPD version 7.3, along with its "EPD Free" counterpart. The highlights of this release are: the addition of enaml, Shapely and several other packages, as well as updates to over 30 packages, including SciPy and IPython. To see which libraries are included in the free vs. full version, please see: http://www.enthought.com/products/epdlibraries.php The complete list of additions, updates and fixes is in the change log: http://www.enthought.com/products/changelog.php EPD 8 preview beta ------------------ EPD 8.0 beta takes all that we know and love in EPD 7.x and adds an all-new graphical development and analysis environment. The new GUI is focused on providing a fast, lightweight interface designed for scientists and engineers. Some of the key features are: * A Python-centric text editor including tab-completion plus on-the-fly code analysis. * An interactive Python (IPython) prompt integrated with the code editor to enable rapid prototyping and exploration. * A Python package manager to make is easier to discover, install, and update packages in the Enthought Python Distribution. * Integrated documentation, both on the GUI itself and standard online documentation. EPD 8 beta can be downloaded from: https://beta.enthought.com/EPD_8/download/ About EPD --------- The Enthought Python Distribution (EPD) is a "kitchen-sink-included" distribution of the Python programming language, including over 90 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization tools, and many other tools. EPD is currently available as a single-click installer for Windows XP, Vista and 7, MacOS (10.5 and 10.6), RedHat 3, 4, 5 and 6, as well as Solaris 10 (x86 and x86_64/amd64 on all platforms). All versions of EPD (32 and 64-bit) are free for academic use. An annual subscription including installation support is available for individual and commercial use. Additional support options, including customization, bug fixes and training classes are also available: http://www.enthought.com/products/epd_sublevels.php - The EPD Team From takowl at gmail.com Wed May 23 19:34:10 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 24 May 2012 00:34:10 +0100 Subject: [IPython-dev] Python 3.3 compatibility - ready for review Message-ID: I've got all the tests passing on a local installation of Python 3.3, if anyone has time to review the pull request: https://github.com/ipython/ipython/pull/1748 The bulk of changes are just for tests which relied on the order of dictionaries (which was never guaranteed, and is now random). There are also some functional changes relating to the new import system, and the dict subclass we used for the namespace during testing is gone - doctests now need to be run with globals that are an actual dict. Thanks, Thomas From fperez.net at gmail.com Fri May 25 22:47:09 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 25 May 2012 19:47:09 -0700 Subject: [IPython-dev] Quick call: please do not merge ANY PRs until #1732 is finished Message-ID: Hi all, #1732 (https://github.com/ipython/ipython/pull/1732) is absolutely enormous and very complex, and right now the rebase is proving to be exceptionally painful. So I want to ask that no more PRs are merged until this is cleared... Thanks! f From fperez.net at gmail.com Sat May 26 02:24:36 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 25 May 2012 23:24:36 -0700 Subject: [IPython-dev] Question about test_pr and python3 Message-ID: Hey Thomas, I can't seem to get test_pr to successfully run the test suite for me with py3, I keep getting these: ====================================================================== ERROR: test_startup_ipy (IPython.core.tests.test_profile.ProfileStartupTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/fperez/.ipy_pr_tests/venv-python3.2/lib/python3.2/site-packages/ipython-0.13.dev-py3.2.egg/IPython/testing/decorators.py", line 229, in skipper_func return f(*args, **kwargs) File "/home/fperez/.ipy_pr_tests/venv-python3.2/lib/python3.2/site-packages/ipython-0.13.dev-py3.2.egg/IPython/core/tests/test_profile.py", line 118, in test_startup_ipy self.validate('test') File "/home/fperez/.ipy_pr_tests/venv-python3.2/lib/python3.2/site-packages/ipython-0.13.dev-py3.2.egg/IPython/core/tests/test_profile.py", line 107, in validate tt.ipexec_validate(self.fname, output, '', options=self.options) File "/home/fperez/.ipy_pr_tests/venv-python3.2/lib/python3.2/site-packages/ipython-0.13.dev-py3.2.egg/IPython/testing/tools.py", line 252, in ipexec_validate out, err = ipexec(fname, options) File "/home/fperez/.ipy_pr_tests/venv-python3.2/lib/python3.2/site-packages/ipython-0.13.dev-py3.2.egg/IPython/testing/tools.py", line 210, in ipexec ipython_cmd = find_cmd('ipython3' if py3compat.PY3 else 'ipython') File "/home/fperez/.ipy_pr_tests/venv-python3.2/lib/python3.2/site-packages/ipython-0.13.dev-py3.2.egg/IPython/utils/process.py", line 71, in find_cmd raise FindCmdError('command could not be found: %s' % cmd) IPython.utils.process.FindCmdError: command could not be found: ipython3 Am I supposed to install manually the branch? thanks f From takowl at gmail.com Sat May 26 05:26:58 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 26 May 2012 10:26:58 +0100 Subject: [IPython-dev] Question about test_pr and python3 In-Reply-To: References: Message-ID: On 26 May 2012 07:24, Fernando Perez wrote: > Am I supposed to install manually the branch? Nope, sorry, that was an oversight. I always had ipython3 installed, so I forgot that I needed to set $PATH for the tests. It should be fixed now. Thanks, Thomas From fperez.net at gmail.com Sun May 27 00:08:25 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 May 2012 21:08:25 -0700 Subject: [IPython-dev] Cell magics ready for merging, review eyes welcome Message-ID: Hi folks, I think we're more or less ready to merge cell magics! These work not only in the notebook, but also at the terminal and the qt console, so you can type %%cython # cython code here even at the IPython terminal :) If anyone is willing to test the PR and have a look, I'd be most grateful: https://github.com/ipython/ipython/pull/1732 Since this PR is massive (almost 100 commits and a diff almost 11,000 lines long), we really need to get this merged before long. So even a quick check that things work for you (or that they don't!) will be appreciated. Cheers, f ps - the %%cython magic itself will be in a separate PR, this one is just the actual infrastructure to enable cell magics. Then we'll add a few we'll ship by default. For now I think the only one will be %%timeit, if I have time to implement it quickly (it should be easy). From fperez.net at gmail.com Sun May 27 03:07:09 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 00:07:09 -0700 Subject: [IPython-dev] Cell magics are in! Message-ID: Hi folks, I'm excited to report that we now have cell magics in IPython... PR 1732 [1] has just been merged [2], which implements the design discussed in IPEP 1 [3]. This is probably one of the largest PRs we've had so far, with over 100 commits, over 100 comments and a diff that's almost 11000 lines long (a lot of it moving code around, obviously it's not all new code). But it brings two very important thigns: 1) a refactor of the magic system to finally remove the old mixin class we'd had since the very first days of IPython in 2001. This is a cleanup I've been wanting to do for over 10 years! The new setup makes the magic system have a very clean api, that is easy to use both for the implementation of core features and for users to create their own magics. 2) the new concept of cell magics: these are magics that get not only the line they're on, but the entire cell body as well. And while these are most naturally used in the notebook, as you would expect we've built them at the core of IPython, so you can use them with all the clients (terminal, qt console, notebook). For example, this is a Cython magic that Brian just prototyped out (we'll have a production version of it soon included). Note that this was copied *from a regular text terminal*, not from the notebook: In [3]: from IPython.core.magic import register_line_cell_magic In [4]: @register_line_cell_magic ...: def cython(line, cell): ...: """Compile and import a cell as a .pyx file.""" ...: import sys ...: from importlib import import_module ...: module = line.strip() ...: fname = module + '.pyx' ...: with open(fname,'w') as f: ...: f.write(cell) ...: if 'pyximport' not in sys.modules: ...: import pyximport ...: pyximport.install(reload_support=True) ...: globals()[module] = import_module(module) ...: In [5]: %%cython bam ...: def f(x): ...: return 2.0*x ...: In [6]: bam.f(10) Out[6]: 20.0 In a similar spirit, Jonathan Taylor recently created one to call R transparently in the notebook: https://github.com/jonathan-taylor/Rmagic This one hasn't been fully updated to the final API, but the core code is there and now it should be a trivial matter to update it. I want to thank everyone who pitched in with ideas during the discussion and review period, and I hope you'll all enjoy this and come up with great ways to use the system. For now, you can see how the system works by playing with %%timeit and %%prun, the only two builtins that I extended to work also as cell magics. For more details, see the documentation where we've added also a long new section with details and examples of how to create your own [4]. Cheers, f [1] https://github.com/ipython/ipython/pull/1732 [2] https://github.com/ipython/ipython/commit/61eb2ffeebb91a94fe9befe2c30e7839781ddc52 [2] https://github.com/ipython/ipython/issues/1611 [3] http://ipython.org/ipython-doc/dev/interactive/reference.html#magic-command-system From fperez.net at gmail.com Sun May 27 04:24:18 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 01:24:18 -0700 Subject: [IPython-dev] Shining panda failure that I don't understand Message-ID: Hi folks this build is reporting a failure: https://jenkins.shiningpanda.com/ipython/job/ipython-multi/PLATFORM=debian6,PYTHON=CPython-2.7/109/changes and the list of changes are all the commits I just merged. But the actual traceback is very odd: ERROR: Failure: ImportError (cannot import name asbytes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/slave/jenkins/shiningpanda/jobs/f786c3c0/virtualenvs/ff035a1d/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName addr.filename, addr.module) File "/home/slave/jenkins/shiningpanda/jobs/f786c3c0/virtualenvs/ff035a1d/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/home/slave/jenkins/shiningpanda/jobs/f786c3c0/virtualenvs/ff035a1d/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/home/slave/jenkins/shiningpanda/jobs/f786c3c0/virtualenvs/ff035a1d/lib/python2.7/site-packages/IPython/parallel/engine/streamkernel.py", line 42, in from IPython.parallel.util import serialize_object, unpack_apply_message, asbytes ImportError: cannot import name asbytes Now, the weird thing is that *there is no file streamkernel.py in IPython.parallel.engine, at all*. Thomas, do you think this could be a stale file from an old branch left by accident in the build dir? Cheers, f From takowl at gmail.com Sun May 27 05:36:34 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 27 May 2012 10:36:34 +0100 Subject: [IPython-dev] Shining panda failure that I don't understand In-Reply-To: References: Message-ID: On 27 May 2012 09:24, Fernando Perez wrote: > Now, the weird thing is that *there is no file streamkernel.py in > IPython.parallel.engine, at all*. > > Thomas, do you think this could be a stale file from an old branch > left by accident in the build dir? Yep, I reported this as #1766, and Min pointed out the problem. I've just done the necessary cleanup, so hopefully that error will be gone tonight. Next up, the Python 3.3 fixes ;-) Thanks, Thomas From ellisonbg at gmail.com Sun May 27 13:53:32 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 27 May 2012 10:53:32 -0700 Subject: [IPython-dev] [IPython-User] Cell magics are in! In-Reply-To: References: Message-ID: Kudos to Fernando for pushing this through! On Sun, May 27, 2012 at 12:07 AM, Fernando Perez wrote: > Hi folks, > > > I'm excited to report that we now have cell magics in IPython... PR > 1732 [1] has just been merged [2], which implements the design > discussed in IPEP 1 [3]. This is probably one of the largest PRs we've > had so far, with over 100 commits, over 100 comments and a diff that's > almost 11000 lines long (a lot of it moving code around, obviously > it's not all new code). ?But it brings two very important thigns: > > 1) a refactor of the magic system to finally remove the old mixin > class we'd had since the very first days of IPython in 2001. ?This is > a cleanup I've been wanting to do for over 10 years! ?The new setup > makes the magic system have ?a very clean api, that is easy to use > both for the implementation of core features and for users to create > their own magics. This is a huge improvement and is one of the last areas of the core that hadn't been cleaned up. This puts us in a great position. > 2) the new concept of cell magics: these are magics that get not only > the line they're on, but the entire cell body as well. ?And while > these are most naturally used in the notebook, as you would expect > we've built them at the core of IPython, so you can use them with all > the clients (terminal, qt console, notebook). ?For example, this is a > Cython magic that Brian just prototyped out (we'll have a production > version of it soon included). ?Note that this was copied *from a > regular text terminal*, not from the notebook: > > In [3]: from IPython.core.magic import register_line_cell_magic > > In [4]: @register_line_cell_magic > ? ...: def cython(line, cell): > ? ...: ? ? """Compile and import a cell as a .pyx file.""" > ? ...: ? ? import sys > ? ...: ? ? from importlib import import_module > ? ...: ? ? module = line.strip() > ? ...: ? ? fname = module + '.pyx' > ? ...: ? ? with open(fname,'w') as f: > ? ...: ? ? ? ? f.write(cell) > ? ...: ? ? if 'pyximport' not in sys.modules: > ? ...: ? ? ? ? import pyximport > ? ...: ? ? ? ? pyximport.install(reload_support=True) > ? ...: ? ? globals()[module] = import_module(module) > ? ...: > > In [5]: %%cython bam > ? ...: def f(x): > ? ...: ? ? return 2.0*x > ? ...: > > In [6]: bam.f(10) > Out[6]: 20.0 I have a much improved version of the cython magic that I will submit a PR for soon (hopefully today). > In a similar spirit, Jonathan Taylor recently created one to call R > transparently in the notebook: > > https://github.com/jonathan-taylor/Rmagic > > This one hasn't been fully updated to the final API, but the core code > is there and now it should be a trivial matter to update it. > > > I want to thank everyone who pitched in with ideas during the > discussion and review period, and I hope you'll all enjoy this and > come up with great ways to use the system. ?For now, you can see how > the system works by playing with %%timeit and %%prun, the only two > builtins that I extended to work also as cell magics. Even these two are a great step. > For more details, see the documentation where we've added also a long > new section with details and examples of how to create your own [4]. Again, great work! Cheers, Brian > Cheers, > > f > > [1] https://github.com/ipython/ipython/pull/1732 > [2] https://github.com/ipython/ipython/commit/61eb2ffeebb91a94fe9befe2c30e7839781ddc52 > [2] https://github.com/ipython/ipython/issues/1611 > [3] http://ipython.org/ipython-doc/dev/interactive/reference.html#magic-command-system > _______________________________________________ > IPython-User mailing list > IPython-User at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-user -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Sun May 27 17:45:27 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 14:45:27 -0700 Subject: [IPython-dev] [IPython-User] Cell magics are in! In-Reply-To: References: Message-ID: On Sun, May 27, 2012 at 10:53 AM, Brian Granger wrote: > > Again, great work! Thanks for the kind words, much appreciated after this marathon effort :) It was really great to have so much solid feedback at all stages. I feel like we're hitting a good stride overall with the project, in terms of a very productive iterative cycle of discussion and implementation. Cheers, f From fperez.net at gmail.com Sun May 27 17:46:22 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 14:46:22 -0700 Subject: [IPython-dev] Shining panda failure that I don't understand In-Reply-To: References: Message-ID: On Sun, May 27, 2012 at 2:36 AM, Thomas Kluyver wrote: > Yep, I reported this as #1766, and Min pointed out the problem. I've > just done the necessary cleanup, so hopefully that error will be gone > tonight. Next up, the Python 3.3 fixes ;-) Embarrassingly, I see I even commented on it in the morning, but by evening I didn't recognize the traceback :) I was totally cross-eyed yesterday by the end, sorry. Glad to see it fixed, thanks! f From fperez.net at gmail.com Sun May 27 17:48:14 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 14:48:14 -0700 Subject: [IPython-dev] Question about test_pr and python3 In-Reply-To: References: Message-ID: On Sat, May 26, 2012 at 2:26 AM, Thomas Kluyver wrote: > > Nope, sorry, that was an oversight. I always had ipython3 installed, > so I forgot that I needed to set $PATH for the tests. It should be > fixed now. Worked perfectly for me yesterday, thanks! This is a *really* great addition to the worfklow, thanks again for implementing it. It makes the review process so much smoother, when we can be confident from direct evidence that a given PR passes multiple test suites beyond what the reviewer may have access to. Great job. Cheers, f From takowl at gmail.com Sun May 27 19:26:31 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 28 May 2012 00:26:31 +0100 Subject: [IPython-dev] Question about test_pr and python3 In-Reply-To: References: Message-ID: On 27 May 2012 22:48, Fernando Perez wrote: > This is a *really* great addition to the worfklow, thanks again for > implementing it. ?It makes the review process so much smoother, when > we can be confident from direct evidence that a given PR passes > multiple test suites beyond what the reviewer may have access to. > Great job. Thanks, and I'm glad it's turning out to be so useful. I have in the back of my mind that once it has incubated in our tools directory, we could spin this out as a separately installable tool that other projects could use. The trickiest bit would be how to standardise running the test suite, but I'm sure we could find ways to do that. Thomas From tsyu80 at gmail.com Sun May 27 19:39:16 2012 From: tsyu80 at gmail.com (Tony Yu) Date: Sun, 27 May 2012 19:39:16 -0400 Subject: [IPython-dev] timeit cell-mode bug? Message-ID: Congratulations on merging the cell magics branch! This looks like it'll allow some really amazing stuff, but just having the timeit, prun, and cython magics seem worth the effort. Speaking of `timeit` I noticed that it returns without execution if no setup code is given. For example, %%timeit 2**2 will return without execution. A simple change to the magics def seems to work on my system: https://github.com/tonysyu/ipython/commit/4f6b8aa607f6fa35fcf17387a8d248c56aedcc6f Cheers! -Tony -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sun May 27 19:45:26 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 28 May 2012 00:45:26 +0100 Subject: [IPython-dev] timeit cell-mode bug? In-Reply-To: References: Message-ID: On 28 May 2012 00:39, Tony Yu wrote: > Speaking of `timeit` I noticed that it returns without execution if no setup > code is given. For example, > > %%timeit > 2**2 > > > will return without execution. A simple change to the magics def seems to > work on my > system:?https://github.com/tonysyu/ipython/commit/4f6b8aa607f6fa35fcf17387a8d248c56aedcc6f Well caught - can you make a pull request with that change? Thanks, Thomas From tsyu80 at gmail.com Sun May 27 19:50:40 2012 From: tsyu80 at gmail.com (Tony Yu) Date: Sun, 27 May 2012 19:50:40 -0400 Subject: [IPython-dev] timeit cell-mode bug? In-Reply-To: References: Message-ID: On Sun, May 27, 2012 at 7:45 PM, Thomas Kluyver wrote: > On 28 May 2012 00:39, Tony Yu wrote: > > Speaking of `timeit` I noticed that it returns without execution if no > setup > > code is given. For example, > > > > %%timeit > > 2**2 > > > > > > will return without execution. A simple change to the magics def seems to > > work on my > > system: > https://github.com/tonysyu/ipython/commit/4f6b8aa607f6fa35fcf17387a8d248c56aedcc6f > > Well caught - can you make a pull request with that change? > > Thanks, > Thomas > > That was fast! Submitted as PR 1769 . -Tony -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sun May 27 22:26:39 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 19:26:39 -0700 Subject: [IPython-dev] Question about test_pr and python3 In-Reply-To: References: Message-ID: On Sun, May 27, 2012 at 4:26 PM, Thomas Kluyver wrote: > I have in the > back of my mind that once it has incubated in our tools directory, we > could spin this out as a separately installable tool that other > projects could use. The trickiest bit would be how to standardise > running the test suite, but I'm sure we could find ways to do that. Certainly. We can set things up so that for package X, we support the stanza python -c 'import X;X.test()' as the default way of running the test suite. That already is the convention used by many projects, so it's not too much to ask, I think. Cheers, f From fperez.net at gmail.com Sun May 27 22:29:28 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 27 May 2012 19:29:28 -0700 Subject: [IPython-dev] timeit cell-mode bug? In-Reply-To: References: Message-ID: On Sun, May 27, 2012 at 4:50 PM, Tony Yu wrote: > > That was fast! Submitted as PR 1769. Great, thanks! Just merged it, sorry I didn't catch that case yesterday, I was pretty out of it by the end when I implemented timeit and prun cell mode... Good catch. Cheers, f From jason-sage at creativetrax.com Mon May 28 11:14:03 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Mon, 28 May 2012 10:14:03 -0500 Subject: [IPython-dev] live coding Message-ID: <4FC3963B.8080900@creativetrax.com> This is an amazing vision of (among other things) live interactive coding: http://vimeo.com/36579366 Bret Victor (who gave the above talk) "has designed experimental UI concepts at Apple, interactive data graphics for Al Gore, and musical instruments at Alesis." He shows several amazing demos of what coding could be like if it was really truly interactive. Some real-life implementations of some of these ideas are here: http://mainstem.org/ (click on an example, and then click any number or color in the code) http://gabrielflor.it/water (click on any number) I challenge us to think of ways to implement some of these principles in our tools. I don't usually use the word transformative, but I think this really would be transformative in how we approach coding and the ipython or sage tools. Thanks, Jason P.S. Actually, Bret Victor's website has a number of other writeups of visions and demos that are very intriguing: http://worrydream.com/. Read some of the papers in the "Research" section to see some really cool concepts. From fperez.net at gmail.com Tue May 29 03:19:19 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 29 May 2012 00:19:19 -0700 Subject: [IPython-dev] Fwd: [ipython] Colons in notebook name causes notebook deletion without warning (#1781) In-Reply-To: References: Message-ID: Hi all, I'm going to bed now but wanted to bump this up on the list in case anyone has a chance to tackle it. As far as I'm concerned, bugs that cause complete data loss for users are top priority so I'll do something about it if needed, but it would be great if someone else beats me to it :) Cheers, f ---------- Forwarded message ---------- From: kinverarity1 Date: Tue, May 29, 2012 at 12:03 AM Subject: [ipython] Colons in notebook name causes notebook deletion without warning (#1781) To: Fernando Perez If you change the name of a notebook by clicking on the notebook title in the browser, and the new name includes a colon, the notebook is "saved" without objection, while in the background the ipynb file is wiped -- the filename is truncated at the colon and the file is empty. This is not apparent in the browser until you close the notebook. This is on Windows. I discovered this by successively changing the names of a set of 5 notebooks to include colons, and continued to make changes to the notebooks, saving, etc. Then I closed them and later went to re-open them to discover everything was gone. There was never any warning at all while in the browser. Given that there is no warning that the notebook title == filename and colons are a fairly common character, this strikes me as a pretty serious issue! --- Reply to this email directly or view it on GitHub: https://github.com/ipython/ipython/issues/1781 From ellisonbg at gmail.com Tue May 29 12:51:04 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 29 May 2012 09:51:04 -0700 Subject: [IPython-dev] Fwd: [ipython] Colons in notebook name causes notebook deletion without warning (#1781) In-Reply-To: References: Message-ID: Ouch, this is painful. I know how to fix this and will try to get to it today. Cheers, Brian On Tue, May 29, 2012 at 12:19 AM, Fernando Perez wrote: > Hi all, > > I'm going to bed now but wanted to bump this up on the list in case > anyone has a chance to tackle it. ?As far as I'm concerned, bugs that > cause complete data loss for users are top priority so I'll do > something about it if needed, but it would be great if someone else > beats me to it :) > > Cheers, > > f > > > ---------- Forwarded message ---------- > From: kinverarity1 > > Date: Tue, May 29, 2012 at 12:03 AM > Subject: [ipython] Colons in notebook name causes notebook deletion > without warning (#1781) > To: Fernando Perez > > > If you change the name of a notebook by clicking on the notebook title > in the browser, and the new name includes a colon, the notebook is > "saved" without objection, while in the background the ipynb file is > wiped -- the filename is truncated at the colon and the file is empty. > This is not apparent in the browser until you close the notebook. This > is on Windows. > > I discovered this by successively changing the names of a set of 5 > notebooks to include colons, and continued to make changes to the > notebooks, saving, etc. Then I closed them and later went to re-open > them to discover everything was gone. There was never any warning at > all while in the browser. Given that there is no warning that the > notebook title == filename and colons are a fairly common character, > this strikes me as a pretty serious issue! > > --- > Reply to this email directly or view it on GitHub: > https://github.com/ipython/ipython/issues/1781 > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jason-sage at creativetrax.com Tue May 29 16:08:57 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Tue, 29 May 2012 15:08:57 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager Message-ID: <4FC52CD9.3060609@creativetrax.com> I'm experimenting with making a KernelManager that starts kernels by forking instead of launching a new fresh process. I'm getting a "ValueError: Unsigned Message" error that I'm not sure what to do about, though. I've put up my short experimental code at https://gist.github.com/2830375. If you do: python -i testfork.py you'll see the Unsigned Message error from the example computation done at the end of that file. Is there a key or something that I should be doing to "sign" my messages? If someone could give this a quick look-over and see if there are any obvious problems, that would be great. Thanks, Jason From fperez.net at gmail.com Tue May 29 16:19:03 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 29 May 2012 13:19:03 -0700 Subject: [IPython-dev] Fwd: [ipython] Colons in notebook name causes notebook deletion without warning (#1781) In-Reply-To: References: Message-ID: On Tue, May 29, 2012 at 9:51 AM, Brian Granger wrote: > Ouch, this is painful. ?I know how to fix this and will try to get to it today. Awesome, thanks!! Cheers, f From fperez.net at gmail.com Tue May 29 16:36:42 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 29 May 2012 13:36:42 -0700 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: <4FC52CD9.3060609@creativetrax.com> References: <4FC52CD9.3060609@creativetrax.com> Message-ID: Hi Jason, On Tue, May 29, 2012 at 1:08 PM, Jason Grout wrote: > Is there a key or something that I should be > doing to "sign" my messages? ?If someone could give this a quick > look-over and see if there are any obvious problems, that would be great. Yes, kernels sign their messages, to make it harder for other processes in the same machine to send them arbitrary commands (remember, our protocol is openly explained, so without this any ipython process would basically be a user shell listening on a port for the first clown to say 'rm -rf $HOME'). See the code in zmq/session.py around line 423 for the signing, and the key is a member of the Session class which you can change/reassign (which will trigger a recomputation of the auth field used to sign the messages, see line 298). HTH, f From jason-sage at creativetrax.com Tue May 29 19:04:14 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Tue, 29 May 2012 18:04:14 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: References: <4FC52CD9.3060609@creativetrax.com> Message-ID: <4FC555EE.4040402@creativetrax.com> On 5/29/12 3:36 PM, Fernando Perez wrote: > Hi Jason, > > On Tue, May 29, 2012 at 1:08 PM, Jason Grout > wrote: >> Is there a key or something that I should be >> doing to "sign" my messages? If someone could give this a quick >> look-over and see if there are any obvious problems, that would be great. > > Yes, kernels sign their messages, to make it harder for other > processes in the same machine to send them arbitrary commands > (remember, our protocol is openly explained, so without this any > ipython process would basically be a user shell listening on a port > for the first clown to say 'rm -rf $HOME'). See the code in > zmq/session.py around line 423 for the signing, and the key is a > member of the Session class which you can change/reassign (which will > trigger a recomputation of the auth field used to sign the messages, > see line 298). Thanks. I remember being a part of the original authentication conversation last year (we implemented the same thing for the sage cell server). The question is: how do I get the key the kernel expects messages to be signed with? Should I be initializing the kernel with that? If so, how should I initialize a kernel to use a specific key for its sessions? It seems that the initializing code is in the KernelApp class in kernelapp.py. If I understand traits correctly, the various *_port traits can be initialized by passing in keyword arguments into the constructor, and that is how the ports are being correctly initialized in my gist. However, the key for the session seems to be only initialized if I read a connection file (in load_connection_file). So a more specific question is: can I initialize a session key using embed_kernel using just the arguments to embed_kernel? Or equivalently, using just initialization arguments to KernelApp, can I set the kernel's session key? Thanks, Jason From jason-sage at creativetrax.com Tue May 29 19:36:38 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Tue, 29 May 2012 18:36:38 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: <4FC555EE.4040402@creativetrax.com> References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> Message-ID: <4FC55D86.5060800@creativetrax.com> On 5/29/12 6:04 PM, Jason Grout wrote: > On 5/29/12 3:36 PM, Fernando Perez wrote: >> Hi Jason, >> >> On Tue, May 29, 2012 at 1:08 PM, Jason Grout >> wrote: >>> Is there a key or something that I should be >>> doing to "sign" my messages? If someone could give this a quick >>> look-over and see if there are any obvious problems, that would be great. >> >> Yes, kernels sign their messages, to make it harder for other >> processes in the same machine to send them arbitrary commands >> (remember, our protocol is openly explained, so without this any >> ipython process would basically be a user shell listening on a port >> for the first clown to say 'rm -rf $HOME'). See the code in >> zmq/session.py around line 423 for the signing, and the key is a >> member of the Session class which you can change/reassign (which will >> trigger a recomputation of the auth field used to sign the messages, >> see line 298). > > Thanks. I remember being a part of the original authentication > conversation last year (we implemented the same thing for the sage cell > server). The question is: how do I get the key the kernel expects > messages to be signed with? Should I be initializing the kernel with > that? If so, how should I initialize a kernel to use a specific key for > its sessions? It seems that the initializing code is in the KernelApp > class in kernelapp.py. If I understand traits correctly, the various > *_port traits can be initialized by passing in keyword arguments into > the constructor, and that is how the ports are being correctly > initialized in my gist. However, the key for the session seems to be > only initialized if I read a connection file (in load_connection_file). > So a more specific question is: can I initialize a session key using > embed_kernel using just the arguments to embed_kernel? Or equivalently, > using just initialization arguments to KernelApp, can I set the kernel's > session key? Just a follow-up: changing the first few lines of embed_kernel to: if IPKernelApp.initialized(): app = IPKernelApp.instance() else: app = IPKernelApp.instance(**kwargs) if key: app.config.Session.key=key app.initialize([]) and adding a `key` keyword argument to embed_kernel seems to work. Should I submit a pull request, or is there a better way to do this? Thanks, Jason From benjaminrk at gmail.com Tue May 29 21:19:36 2012 From: benjaminrk at gmail.com (MinRK) Date: Tue, 29 May 2012 18:19:36 -0700 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: <4FC55D86.5060800@creativetrax.com> References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> Message-ID: On Tue, May 29, 2012 at 4:36 PM, Jason Grout wrote: > On 5/29/12 6:04 PM, Jason Grout wrote: > > On 5/29/12 3:36 PM, Fernando Perez wrote: > >> Hi Jason, > >> > >> On Tue, May 29, 2012 at 1:08 PM, Jason Grout > >> wrote: > >>> Is there a key or something that I should be > >>> doing to "sign" my messages? If someone could give this a quick > >>> look-over and see if there are any obvious problems, that would be > great. > >> > >> Yes, kernels sign their messages, to make it harder for other > >> processes in the same machine to send them arbitrary commands > >> (remember, our protocol is openly explained, so without this any > >> ipython process would basically be a user shell listening on a port > >> for the first clown to say 'rm -rf $HOME'). See the code in > >> zmq/session.py around line 423 for the signing, and the key is a > >> member of the Session class which you can change/reassign (which will > >> trigger a recomputation of the auth field used to sign the messages, > >> see line 298). > > > > Thanks. I remember being a part of the original authentication > > conversation last year (we implemented the same thing for the sage cell > > server). The question is: how do I get the key the kernel expects > > messages to be signed with? Should I be initializing the kernel with > > that? If so, how should I initialize a kernel to use a specific key for > > its sessions? It seems that the initializing code is in the KernelApp > > class in kernelapp.py. If I understand traits correctly, the various > > *_port traits can be initialized by passing in keyword arguments into > > the constructor, and that is how the ports are being correctly > > initialized in my gist. However, the key for the session seems to be > > only initialized if I read a connection file (in load_connection_file). > > So a more specific question is: can I initialize a session key using > > embed_kernel using just the arguments to embed_kernel? Or equivalently, > > using just initialization arguments to KernelApp, can I set the kernel's > > session key? > > Just a follow-up: changing the first few lines of embed_kernel to: > > if IPKernelApp.initialized(): > app = IPKernelApp.instance() > else: > app = IPKernelApp.instance(**kwargs) > if key: > app.config.Session.key=key > app.initialize([]) > > > and adding a `key` keyword argument to embed_kernel seems to work. > Should I submit a pull request, or is there a better way to do this? > That should be unnecessary, because you can just pass a config object: cfg = Config() cfg.Session.key = foo embed_kernel(config=cfg) There should be nothing at all that you cannot specify with the current call as it is. But that's all beside the point, because the normal way to communicate connections to the kernel is through a *file* (App.connection_file configurable). I know it may seem odd to use files when you are using fork, but this will still be required if you want to start other frontends against that kernel. Here is a fork of your script that only passes the connection file, and lets the existing machinery do the work: https://gist.github.com/2831998 The main change that had to happen was specifying KM.session.key *before* launching the kernel, since it's used when writing the connection file. I would also note that I think the entry point should probably *not* be embed_kernel, rather a variation of the standard ipkernel.launch_new_instance entry point, which doesn't skip all of the normal namespace initialization like embed_kernel does, for the sake of embedding. -MinRK > Thanks, > > Jason > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Wed May 30 01:14:06 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Wed, 30 May 2012 00:14:06 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> Message-ID: <4FC5AC9E.2090209@creativetrax.com> On 5/29/12 8:19 PM, MinRK wrote: > That should be unnecessary, because you can just pass a config object: Great! Thank you! I feel like I spend a lot of time poking around in the code because I'm not familiar with the mindset of the code (for example, thanks for the advice about KernelManager vs. BlockingKernelManager). One thing that I'm still not understanding deeply enough is how Traits and the ipython config system really works, so I appreciate your comments and modifications there too. > > cfg = Config() > cfg.Session.key = foo > > embed_kernel(config=cfg) > > There should be nothing at all that you cannot specify with the current > call as it is. Thanks. I'll be thinking of how I can do things with Config objects in the future. > > But that's all beside the point, because the normal way to communicate > connections to the kernel is through a *file* (App.connection_file > configurable). I know it may seem odd to use files when you are using > fork, but this will still be required if you want to start other > frontends against that kernel. In our specific case, we plan to fork hundreds of these instances as very short-lived computation engines (it's a rewrite of the sage cell server). We don't see a use-case for other clients connecting to the engine (indeed, the engine is on a separate server that has no connection to the user except through our designated channel, and we want to keep it that way for security). Another reason to not write a file to disk containing the security key is to prevent two different forking engines (but from the same user) from accessing the same secret key. We'll have two completely different web users launching two different kernels (same user account) to do computations. It would be bad if one user can trivially read a file to see how to mess up the other kernel. If we generate a random key and pass it on to the forked kernel in memory, it makes it much harder for the second user to send messages to the first kernel. > > Here is a fork of your script that only passes the connection file, and > lets the existing machinery do the work: https://gist.github.com/2831998 > > The main change that had to happen was specifying KM.session.key > *before* launching the kernel, since it's used when writing the > connection file. Okay, of course, if you're writing a file, that makes sense. Thanks! > I would also note that I think the entry point should probably *not* be > embed_kernel, rather a variation of the standard > ipkernel.launch_new_instance entry point, which doesn't skip all of the > normal namespace initialization like embed_kernel does, for the sake of > embedding. Thanks for the pointer. I'll look at this and see the difference between it and embed_kernel. I was just trying to somehow get a kernel launched without using Popen, and embed_kernel was an obvious choice from the name. Thanks again for your help! It really helps us to understand the philosophy behind the code, as well as the code itself. Thanks, Jason From jason-sage at creativetrax.com Wed May 30 01:42:01 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Wed, 30 May 2012 00:42:01 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: <4FC5AC9E.2090209@creativetrax.com> References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> <4FC5AC9E.2090209@creativetrax.com> Message-ID: <4FC5B329.8050200@creativetrax.com> On 5/30/12 12:14 AM, Jason Grout wrote: > But that's all beside the point, because the normal way to communicate >> connections to the kernel is through a*file* (App.connection_file >> configurable). How about if the "file name" is a basestring/string, then it be considered a filename, but if not, it be treated as just a file object? That allows for configuration information to be stored much more flexibly. For example, we might have a central server that stores all configuration information in a database, and is accessed using a custom API. Why the insistence that we always pass config information in a file in a directory on disk, rather than a more generic file-like object, with a convenience shortcut of a string meaning a file on disk? Thanks, Jason From benjaminrk at gmail.com Wed May 30 03:42:36 2012 From: benjaminrk at gmail.com (Min RK) Date: Wed, 30 May 2012 00:42:36 -0700 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: <4FC5B329.8050200@creativetrax.com> References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> <4FC5AC9E.2090209@creativetrax.com> <4FC5B329.8050200@creativetrax.com> Message-ID: -MinRK On May 29, 2012, at 10:42 PM, Jason Grout wrote: > On 5/30/12 12:14 AM, Jason Grout wrote: >> But that's all beside the point, because the normal way to communicate >>> connections to the kernel is through a*file* (App.connection_file >>> configurable). > > How about if the "file name" is a basestring/string, then it be > considered a filename, but if not, it be treated as just a file object? > That allows for configuration information to be stored much more > flexibly. For example, we might have a central server that stores all > configuration information in a database, and is accessed using a custom > API. Why the insistence that we always pass config information in a > file in a directory on disk, rather than a more generic file-like > object, with a convenience shortcut of a string meaning a file on disk? > We don't insist, and it's not required. It's just the cleanest and best model we have found for every existing use case thus far. If you don't want to use a connection file, then simply specify the values with a config object, similar to what you were doing in the first go. I only changed your script to use the connection file because it re-uses more existing code, and is thus simpler, clearer, and less prone to mistakes. If there is a *reason* to not use the connection file, then simply don't. I don't see any need for a file-like object. -MinRK > Thanks, > > Jason > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From jason-sage at creativetrax.com Wed May 30 03:46:05 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Wed, 30 May 2012 02:46:05 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> <4FC5AC9E.2090209@creativetrax.com> <4FC5B329.8050200@creativetrax.com> Message-ID: <4FC5D03D.5020505@creativetrax.com> > We don't insist, and it's not required. It's just the cleanest and > best model we have found for every existing use case thus far. If > you don't want to use a connection file, then simply specify the > values with a config object, similar to what you were doing in the > first go. I only changed your script to use the connection file > because it re-uses more existing code, and is thus simpler, clearer, > and less prone to mistakes. If there is a *reason* to not use the > connection file, then simply don't. I don't see any need for a > file-like object. Ah, perfect. Thanks for the clarification. Thanks again for your help! Jason From benjaminrk at gmail.com Wed May 30 04:01:59 2012 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 May 2012 01:01:59 -0700 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: <4FC5D03D.5020505@creativetrax.com> References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> <4FC5AC9E.2090209@creativetrax.com> <4FC5B329.8050200@creativetrax.com> <4FC5D03D.5020505@creativetrax.com> Message-ID: On Wed, May 30, 2012 at 12:46 AM, Jason Grout wrote: > > > We don't insist, and it's not required. It's just the cleanest and > > best model we have found for every existing use case thus far. If > > you don't want to use a connection file, then simply specify the > > values with a config object, similar to what you were doing in the > > first go. I only changed your script to use the connection file > > because it re-uses more existing code, and is thus simpler, clearer, > > and less prone to mistakes. If there is a *reason* to not use the > > connection file, then simply don't. I don't see any need for a > > file-like object. > > Ah, perfect. Thanks for the clarification. Thanks again for your help! > In fact, if you check out https://gist.github.com/2831998 now, it will do exactly that: no connection files involved. -MinRK > > Jason > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Wed May 30 04:22:14 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Wed, 30 May 2012 03:22:14 -0500 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> <4FC5AC9E.2090209@creativetrax.com> <4FC5B329.8050200@creativetrax.com> <4FC5D03D.5020505@creativetrax.com> Message-ID: <4FC5D8B6.60603@creativetrax.com> On 5/30/12 3:01 AM, MinRK wrote: > > > On Wed, May 30, 2012 at 12:46 AM, Jason Grout > > wrote: > > > > We don't insist, and it's not required. It's just the cleanest and > > best model we have found for every existing use case thus far. If > > you don't want to use a connection file, then simply specify the > > values with a config object, similar to what you were doing in the > > first go. I only changed your script to use the connection file > > because it re-uses more existing code, and is thus simpler, clearer, > > and less prone to mistakes. If there is a *reason* to not use the > > connection file, then simply don't. I don't see any need for a > > file-like object. > > Ah, perfect. Thanks for the clarification. Thanks again for your help! > > > In fact, if you check out https://gist.github.com/2831998 now, it will > do exactly that: no connection files involved. > I was just in the middle of overriding start_kernel; your modifications are exactly what I was aiming for. Thanks! Jason From fperez.net at gmail.com Wed May 30 16:35:06 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 May 2012 13:35:06 -0700 Subject: [IPython-dev] "Unsigned Message" error in a forking kernel manager In-Reply-To: References: <4FC52CD9.3060609@creativetrax.com> <4FC555EE.4040402@creativetrax.com> <4FC55D86.5060800@creativetrax.com> <4FC5AC9E.2090209@creativetrax.com> <4FC5B329.8050200@creativetrax.com> <4FC5D03D.5020505@creativetrax.com> Message-ID: On Wed, May 30, 2012 at 1:01 AM, MinRK wrote: > In fact, if you check out https://gist.github.com/2831998 now, it will do > exactly that: no connection files involved. This looks like it would make for a great example to ship, just with a couple of comments on certain values (such as what can be a valid key). I'm sure others than Jason would benefit from it, and we've found it typically very useful to be able to point people to these kinds of examples. Cheers, f From fperez.net at gmail.com Thu May 31 01:22:14 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 May 2012 22:22:14 -0700 Subject: [IPython-dev] Cython magics have been merged! Message-ID: Hi folks, we just concluded review and merge of the cython magics: https://github.com/ipython/ipython/pull/1770! Kudos to Brian for a terrific job, and please let us know if you spot any problems. There's an example notebook provided (docs/examples/notebooks/cython_extension.ipynb) so you can see it in action. We think this will be very useful, and the system also serves as a reference on how to implement other extensions with useful magics. The R one is also in review and very close to ready: https://github.com/ipython/ipython/pull/1780, so pitch in with your feedback if there's anything there you think can be improved. Cheers, f From fperez.net at gmail.com Thu May 31 03:13:02 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 00:13:02 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present Message-ID: Brian, I'd mentioned it in review but we forgot about it at the end... We missed a bit of isolation for the test suite with the new cython magics... The fix is pretty easy, it's a simple matter of adding an exclusion in iptest.py like the ones for curses or the other optional libraries Just pinging here in case someone can quickly do it before tomorrow (crashing to bed now). Cheers f ---------- Forwarded message ---------- From: Date: Thu, May 31, 2012 at 12:01 AM Subject: [Ipython-build] Automatic IPython build reports Jenkins build became unstable: ipython-mac #124 To: ipython-build at lists.ipython.org See _______________________________________________ Ipython-build mailing list Ipython-build at lists.ipython.org http://lists.ipython.org/listinfo.cgi/ipython-build-ipython.org From ellisonbg at gmail.com Thu May 31 03:17:48 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 00:17:48 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 12:13 AM, Fernando Perez wrote: > Brian, I'd mentioned it in review but we forgot about it at the end... > ?We missed a bit of isolation for the test suite with the new cython > magics... > The fix is pretty easy, it's a simple matter of adding an exclusion in > iptest.py like the ones for curses or the other optional libraries I added that already. Not sure why this is failing as that module has the right logic to skip the tests if Cython is not present. I will look more tomorrow though. > Just pinging here in case someone can quickly do it before tomorrow > (crashing to bed now). > > Cheers > > f > ---------- Forwarded message ---------- > From: ? > Date: Thu, May 31, 2012 at 12:01 AM > Subject: [Ipython-build] Automatic IPython build reports Jenkins build > became unstable: ipython-mac #124 > To: ipython-build at lists.ipython.org > > > See > > _______________________________________________ > Ipython-build mailing list > Ipython-build at lists.ipython.org > http://lists.ipython.org/listinfo.cgi/ipython-build-ipython.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From hemmecke at gmail.com Thu May 31 05:52:13 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Thu, 31 May 2012 11:52:13 +0200 Subject: [IPython-dev] SVN or GIT Message-ID: <4FC73F4D.80000@gmail.com> Hello This text speaks of SVN. http://ipython.org/faq.html#what-is-the-best-way-to-install-ipython Is this still up-to-date? Ralf From hemmecke at gmail.com Thu May 31 05:56:35 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Thu, 31 May 2012 11:56:35 +0200 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: Message-ID: <4FC74053.5060509@gmail.com> Hello Fernando, > The R one is also in review and very close to ready: > https://github.com/ipython/ipython/pull/1780, so pitch in with your > feedback if there's anything there you think can be improved. This sounds as if I should look more closely into the ipython notebook again in order to use it as a front-end to FriCAS [1]. When I tried to install ipython from git, I realized that I have to install zeromq. That seems to be no problem on the latest ubuntu. I haven't actually tried, but at least I can find packages like libzmq-dev which I could apt-get install. However, on Debian 6 there is no such package. Unfortunately, [2] is not completely clear about zeromq on systems that do not have it. In any case, are there some instructions to install the ipython notebook on debian 6 (including all the prerequisites)? In fact, I'd like to have some instructions that don't need 'root' access to this debian 6 system. Can I install it under virtualenv? In fact, I would be interested to use the fricas input language inside the notebook, not python. Would this be possible with the ipython notebook? Ralf [1] http://fricas.sourceforge.net/ [2] http://ipython.org/ipython-doc/stable/install/install.html From takowl at gmail.com Thu May 31 06:14:24 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 31 May 2012 11:14:24 +0100 Subject: [IPython-dev] Cython magics have been merged! In-Reply-To: References: Message-ID: On 31 May 2012 06:22, Fernando Perez wrote: > Kudos to Brian for a terrific job, and please let us know if you spot > any problems. It seems the code that should skip the cythonmagic tests if Cython isn't present isn't working as expected. I've filed an issue for it: https://github.com/ipython/ipython/issues/1802 Thomas From takowl at gmail.com Thu May 31 07:06:19 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 31 May 2012 12:06:19 +0100 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: Oops, I should have spotted this thread before I posted the same thing. Sorry for the noise. Thomas From takowl at gmail.com Thu May 31 07:09:36 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 31 May 2012 12:09:36 +0100 Subject: [IPython-dev] SVN or GIT In-Reply-To: <4FC73F4D.80000@gmail.com> References: <4FC73F4D.80000@gmail.com> Message-ID: On 31 May 2012 10:52, Ralf Hemmecke wrote: > Is this still up-to-date? In a word, no. It seems specifying ipython==dev still tries to check it out from svn. Does anyone know how we point it to git? If not, I'll just remove that line. Thanks for letting us know about that. Thomas From takowl at gmail.com Thu May 31 07:15:12 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 31 May 2012 12:15:12 +0100 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: <4FC74053.5060509@gmail.com> References: <4FC74053.5060509@gmail.com> Message-ID: On 31 May 2012 10:56, Ralf Hemmecke wrote: > In any case, are there some instructions to install the ipython notebook > on debian 6 (including all the prerequisites)? In fact, I'd like to have > some instructions that don't need 'root' access to this debian 6 system. > Can I install it under virtualenv? Yes, but it's a bit fiddly to get zeromq if you don't have root access. The Shiningpanda test machines run Debian 6 without root access. I built zeromq manually and installed it in /home/slave/local, then the test run does this inside a virtualenv: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/slave/local/lib export ZMQ_DIR=/home/slave/local pip install nose coverage pip install pyzmq pip install readline rm -rf build/ python setup.py install # Install IPython Best wishes, Thomas From fperez.net at gmail.com Thu May 31 11:12:25 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 08:12:25 -0700 Subject: [IPython-dev] SVN or GIT In-Reply-To: References: <4FC73F4D.80000@gmail.com> Message-ID: On Thu, May 31, 2012 at 4:09 AM, Thomas Kluyver wrote: > Does anyone know how we point it to git? No idea, I'm the setuptools ignoramus in the room... f From satra at mit.edu Thu May 31 11:17:39 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Thu, 31 May 2012 11:17:39 -0400 Subject: [IPython-dev] SVN or GIT In-Reply-To: References: <4FC73F4D.80000@gmail.com> Message-ID: i can do this with pip ;) pip install git+git://github.com/ipython/ipython.git#egg=ipython cheers, satra On Thu, May 31, 2012 at 11:12 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 4:09 AM, Thomas Kluyver wrote: > > Does anyone know how we point it to git? > > No idea, I'm the setuptools ignoramus in the room... > > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu May 31 11:38:20 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 08:38:20 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: <4FC74053.5060509@gmail.com> References: <4FC74053.5060509@gmail.com> Message-ID: On Thu, May 31, 2012 at 2:56 AM, Ralf Hemmecke wrote: > In fact, I would be interested to use the fricas input language inside > the notebook, not python. Would this be possible with the ipython notebook? Yes, it's possible but it will require some work on your part. Basically you'll have to implement something that speaks the IPython messaging protocol (http://ipython.org/ipython-doc/dev/development/messaging.html). There are basically two choices: 1) you can reuse the IPython kernel machinery and gut out the innermost execution methods, so that instead of using Python they call out your fricas process/library. 2) You can implement directly in fricas the zeromq protocol itself. The second option is more work but would produce something a bit cleaner and more standalone. Cheers, f From fperez.net at gmail.com Thu May 31 12:29:55 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 09:29:55 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 12:17 AM, Brian Granger wrote: > I added that already. ?Not sure why this is failing as that module has > the right logic to skip the tests if Cython is not present. ?I will > look more tomorrow though. I've been trying things here, and I'm *quite* puzzled as to why this isn't getting skipped. Identical-looking code does work for excluding a lot of other optional things, but the cython one is getting picked up despite all attempts at excluding it. I'm kind of stumped right now. From fperez.net at gmail.com Thu May 31 12:40:13 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 09:40:13 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 9:29 AM, Fernando Perez wrote: > > I'm kind of stumped right now. I've put a full-gore nose log here: https://gist.github.com/2844633 The weirdness is the following: a modified exclusions list indeed is detected by nose, which early on prints: nose.selector: DEBUG: plugin setting want /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.py to False nose.selector: DEBUG: wantFile /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.py? False nose.selector: DEBUG: plugin setting want /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.pyc to False nose.selector: DEBUG: wantFile /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.pyc? False but further down, something else in the nose machinery backtracks and now says it does want this file: nose.selector: DEBUG: plugin setting want /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py to True nose.selector: DEBUG: wantFile /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py? True nose.loader: DEBUG: load from /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py (None) nose.selector: DEBUG: Test name /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py resolved to file /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py, module None, call None nose.selector: DEBUG: Final resolution of test name /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py: file /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py module IPython.extensions.tests.test_cythonmagic call None nose.importer: DEBUG: Import IPython.extensions.tests.test_cythonmagic from /home/fperez/ipython/ipython Which then proceeds to blow up (this was done in a bare virtualenv that didn't find my system-installed Cython). I have no clue what is making nose ignore the earlier exclusion decision... Cheers, f From benjaminrk at gmail.com Thu May 31 12:44:46 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 31 May 2012 09:44:46 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 9:40 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 9:29 AM, Fernando Perez > wrote: > > > > I'm kind of stumped right now. > > I've put a full-gore nose log here: https://gist.github.com/2844633 > > The weirdness is the following: a modified exclusions list indeed is > detected by nose, which early on prints: > > nose.selector: DEBUG: plugin setting want > /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.py to > False > nose.selector: DEBUG: wantFile > /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.py? False > nose.selector: DEBUG: plugin setting want > /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.pyc to > False > nose.selector: DEBUG: wantFile > /home/fperez/ipython/ipython/IPython/extensions/cythonmagic.pyc? False > > but further down, something else in the nose machinery backtracks and > now says it does want this file: > > nose.selector: DEBUG: plugin setting want > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py > to True > nose.selector: DEBUG: wantFile > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py? > True > nose.loader: DEBUG: load from > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py > (None) > nose.selector: DEBUG: Test name > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py > resolved to file > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py, > module None, call None > nose.selector: DEBUG: Final resolution of test name > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py: > file > /home/fperez/ipython/ipython/IPython/extensions/tests/test_cythonmagic.py > module IPython.extensions.tests.test_cythonmagic call None > nose.importer: DEBUG: Import IPython.extensions.tests.test_cythonmagic > from /home/fperez/ipython/ipython > > Which then proceeds to blow up (this was done in a bare virtualenv > that didn't find my system-installed Cython). > > I have no clue what is making nose ignore the earlier exclusion decision... > It might be related to the fact that it's in the tests dir. I don't think we've excluded a *test* file before, always real code / dirs. -MinRK > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu May 31 12:51:39 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 09:51:39 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 9:44 AM, MinRK wrote: > It might be related to the fact that it's in the tests dir. ?I don't think > we've excluded a *test* file before, always real code / dirs. Well, we do have these that exclude things like test_mongodb.py and test_pylabtools.py: if not have['pymongo']: exclusions.append(ipjoin('parallel', 'controller', 'mongodb')) exclusions.append(ipjoin('parallel', 'tests', 'test_mongodb')) if not have['matplotlib']: exclusions.extend([ipjoin('core', 'pylabtools'), ipjoin('core', 'tests', 'test_pylabtools'), ipjoin('zmq', 'pylab'), and those have worked so far. I just confirmed that they work indeed by running the test suite for IPython.core with a '1/0' in the test_pylabtools.py file, and I get what I expect: - in a venv that's missing matplotlib, the test_pylabtools is skipped and thus the whole test suite passes cleanly. - in my normal environment that finds matplotlib, that 1/0 makes it blow up with an error as expected. So the exclusion for a single test file does seem to work OK from this check... Puzzled... f From benjaminrk at gmail.com Thu May 31 13:04:29 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 31 May 2012 10:04:29 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 9:51 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 9:44 AM, MinRK wrote: > > > It might be related to the fact that it's in the tests dir. I don't > think > > we've excluded a *test* file before, always real code / dirs. > > Well, we do have these that exclude things like test_mongodb.py and > test_pylabtools.py: > > if not have['pymongo']: > exclusions.append(ipjoin('parallel', 'controller', 'mongodb')) > exclusions.append(ipjoin('parallel', 'tests', 'test_mongodb')) > > if not have['matplotlib']: > exclusions.extend([ipjoin('core', 'pylabtools'), > ipjoin('core', 'tests', 'test_pylabtools'), > ipjoin('zmq', 'pylab'), > > > and those have worked so far. I just confirmed that they work indeed > by running the test suite for IPython.core with a '1/0' in the > test_pylabtools.py file, and I get what I expect: > > - in a venv that's missing matplotlib, the test_pylabtools is skipped > and thus the whole test suite passes cleanly. > > - in my normal environment that finds matplotlib, that 1/0 makes it > blow up with an error as expected. > > So the exclusion for a single test file does seem to work OK from this > check... > I apologize if this is obvious, but are we aware that the test_cythonmagic file is *not* excluded in master? if not have['cython']: exclusions.extend([ipjoin('extensions', 'cythonmagic')]) should be: if not have['cython']: exclusions.extend([ipjoin('extensions', 'cythonmagic')]) exclusions.extend([ipjoin('extensions', 'tests', 'test_cythonmagic')]) With this addition, and a manually disabled Cython, testing works fine for me. And Fernando, if you look carefully at your log, the excluded and included files are not actually the same. -MinRK > > Puzzled... > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu May 31 13:21:44 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 10:21:44 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 10:04 AM, MinRK wrote: > I apologize if this is obvious, but are we aware that the test_cythonmagic > file is *not* excluded in master? > > ? ? if not have['cython']: > ? ? ? ? exclusions.extend([ipjoin('extensions', 'cythonmagic')]) > > should be: > > ? ? if not have['cython']: > ? ? ? ? exclusions.extend([ipjoin('extensions', 'cythonmagic')]) > ? ? ? ? exclusions.extend([ipjoin('extensions', 'tests', > 'test_cythonmagic')]) > > With this addition, and a manually disabled Cython, testing works fine for > me. Yes, I'd noticed that, but in my local testing I still get the problem even with that fix. In fact, this is what I have: if not have['cython']: exclusions.extend([ipjoin('extensions', 'cythonmagic')]) exclusions.extend([ipjoin('extensions', 'cythonmagic.py')]) exclusions.extend([ipjoin('extensions', 'test', 'test_cythonmagic')]) exclusions.extend([ipjoin('extensions', 'test', 'test_cythonmagic.py')]) ARGGHHH!!! Nevermind, I fixed the exclusions list with 'test' not 'tests' as the subdir. I'm a moron. f From ellisonbg at gmail.com Thu May 31 13:24:46 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 10:24:46 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> Message-ID: Or you could do this as a cell magic... On Thu, May 31, 2012 at 8:38 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 2:56 AM, Ralf Hemmecke wrote: >> In fact, I would be interested to use the fricas input language inside >> the notebook, not python. Would this be possible with the ipython notebook? > > Yes, it's possible but it will require some work on your part. > Basically you'll have to implement something that speaks the IPython > messaging protocol > (http://ipython.org/ipython-doc/dev/development/messaging.html). > There are basically two choices: > > 1) you can reuse the IPython kernel machinery and gut out the > innermost execution methods, so that instead of using Python they call > out your fricas process/library. > > 2) You can implement directly in fricas the zeromq protocol itself. > > The second option is more work but would produce something a bit > cleaner and more standalone. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jason-sage at creativetrax.com Thu May 31 13:29:38 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 31 May 2012 12:29:38 -0500 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> Message-ID: <4FC7AA82.4030103@creativetrax.com> On 5/31/12 12:24 PM, Brian Granger wrote: > Or you could do this as a cell magic... I'm curious: is there a way to set a default cell magic that is automatically applied to each and every cell? I can imagine us having a %%sage cell magic, and it being nice to do: %set_default_cell_magic sage or something, and then from then on, it's practically like I'm running Sage. Or R. Or fricas. Or whatever. Thanks, Jason From ellisonbg at gmail.com Thu May 31 13:33:38 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 10:33:38 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: <4FC7AA82.4030103@creativetrax.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: We have started tho think about things like that, but haven't quite gotten there. I think the main question I have is whether or not this transformation should be applied by the frontends or the kernel. The problem with the kernel is that there could be multiple frontends connected to the kernel and it would be very confusing for those users to all of a sudden to switch modes. Also, I don't like the idea of having stateful magics like this. We have tried that with the %autopx magic is it tends to be very confusing. I guess as I write, I am leaning towards this being a frontend transformation. On Thu, May 31, 2012 at 10:29 AM, Jason Grout wrote: > On 5/31/12 12:24 PM, Brian Granger wrote: >> Or you could do this as a cell magic... > > I'm curious: is there a way to set a default cell magic that is > automatically applied to each and every cell? ?I can imagine us having a > %%sage cell magic, and it being nice to do: > > %set_default_cell_magic sage > > or something, and then from then on, it's practically like I'm running > Sage. ?Or R. ?Or fricas. ?Or whatever. > > Thanks, > > Jason > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From bussonniermatthias at gmail.com Thu May 31 13:46:28 2012 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Thu, 31 May 2012 19:46:28 +0200 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: <2ADEF2DD-08B0-476A-8221-75D00590D914@gmail.com> Le 31 mai 2012 ? 19:33, Brian Granger a ?crit : > On Thu, May 31, 2012 at 10:29 AM, Jason Grout > wrote: >> On 5/31/12 12:24 PM, Brian Granger wrote: >>> Or you could do this as a cell magic... >> >> I'm curious: is there a way to set a default cell magic that is >> automatically applied to each and every cell? I can imagine us having a >> %%sage cell magic, and it being nice to do: >> >> %set_default_cell_magic sage >> >> or something, and then from then on, it's practically like I'm running >> Sage. Or R. Or fricas. Or whatever. This make me think, how do the cell magics works if one export a ipython session to a .ipy file ? How is handled the end of the 'cell' or block ? Can you write a script, smith like ``` some python %%cython some cython %%(end) some python ``` -- Matthias From ellisonbg at gmail.com Thu May 31 13:47:37 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 10:47:37 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: <2ADEF2DD-08B0-476A-8221-75D00590D914@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <2ADEF2DD-08B0-476A-8221-75D00590D914@gmail.com> Message-ID: I am not sure, but I think it will work in an .ipy file. On Thu, May 31, 2012 at 10:46 AM, Matthias BUSSONNIER wrote: > > Le 31 mai 2012 ? 19:33, Brian Granger a ?crit : >> On Thu, May 31, 2012 at 10:29 AM, Jason Grout >> wrote: >>> On 5/31/12 12:24 PM, Brian Granger wrote: >>>> Or you could do this as a cell magic... >>> >>> I'm curious: is there a way to set a default cell magic that is >>> automatically applied to each and every cell? ?I can imagine us having a >>> %%sage cell magic, and it being nice to do: >>> >>> %set_default_cell_magic sage >>> >>> or something, and then from then on, it's practically like I'm running >>> Sage. ?Or R. ?Or fricas. ?Or whatever. > > This make me think, how do the cell magics works if one export a ipython session to a .ipy file ? > How is handled the end of the 'cell' or block ? > > Can you write a script, smith like > > ``` > some python > > %%cython > some cython > > %%(end) > > some python > ``` > > -- > Matthias > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From bussonniermatthias at gmail.com Thu May 31 14:03:31 2012 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Thu, 31 May 2012 20:03:31 +0200 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <2ADEF2DD-08B0-476A-8221-75D00590D914@gmail.com> Message-ID: <43725CAC-FC3C-4574-BB8B-2D9325F97065@gmail.com> Le 31 mai 2012 ? 19:47, Brian Granger a ?crit : > I am not sure, but I think it will work in an .ipy file. Yes, it works, but the cell magic execute until the end of the file. How difficult would it be to execute the cell magics until next line beginning with `%%`then carry on normal execution ? -- Matthias > > On Thu, May 31, 2012 at 10:46 AM, Matthias BUSSONNIER > wrote: >> >> Le 31 mai 2012 ? 19:33, Brian Granger a ?crit : >>> On Thu, May 31, 2012 at 10:29 AM, Jason Grout >>> wrote: >>>> On 5/31/12 12:24 PM, Brian Granger wrote: >>>>> Or you could do this as a cell magic... >>>> >>>> I'm curious: is there a way to set a default cell magic that is >>>> automatically applied to each and every cell? I can imagine us having a >>>> %%sage cell magic, and it being nice to do: >>>> >>>> %set_default_cell_magic sage >>>> >>>> or something, and then from then on, it's practically like I'm running >>>> Sage. Or R. Or fricas. Or whatever. >> >> This make me think, how do the cell magics works if one export a ipython session to a .ipy file ? >> How is handled the end of the 'cell' or block ? >> >> Can you write a script, smith like >> >> ``` >> some python >> >> %%cython >> some cython >> >> %%(end) >> >> some python >> ``` >> >> -- >> Matthias >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From ellisonbg at gmail.com Thu May 31 14:04:27 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 11:04:27 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: <43725CAC-FC3C-4574-BB8B-2D9325F97065@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <2ADEF2DD-08B0-476A-8221-75D00590D914@gmail.com> <43725CAC-FC3C-4574-BB8B-2D9325F97065@gmail.com> Message-ID: I think you have to insert 2 empty lines to indicate the end of the cell magic. On Thu, May 31, 2012 at 11:03 AM, Matthias BUSSONNIER wrote: > > Le 31 mai 2012 ? 19:47, Brian Granger a ?crit : > >> I am not sure, but I think it will work in an .ipy file. > > Yes, it works, but the cell magic execute until the end of the file. > How difficult would it be to execute the cell magics until next line beginning with `%%`then carry on normal execution ? > > -- > Matthias > >> >> On Thu, May 31, 2012 at 10:46 AM, Matthias BUSSONNIER >> wrote: >>> >>> Le 31 mai 2012 ? 19:33, Brian Granger a ?crit : >>>> On Thu, May 31, 2012 at 10:29 AM, Jason Grout >>>> wrote: >>>>> On 5/31/12 12:24 PM, Brian Granger wrote: >>>>>> Or you could do this as a cell magic... >>>>> >>>>> I'm curious: is there a way to set a default cell magic that is >>>>> automatically applied to each and every cell? ?I can imagine us having a >>>>> %%sage cell magic, and it being nice to do: >>>>> >>>>> %set_default_cell_magic sage >>>>> >>>>> or something, and then from then on, it's practically like I'm running >>>>> Sage. ?Or R. ?Or fricas. ?Or whatever. >>> >>> This make me think, how do the cell magics works if one export a ipython session to a .ipy file ? >>> How is handled the end of the 'cell' or block ? >>> >>> Can you write a script, smith like >>> >>> ``` >>> some python >>> >>> %%cython >>> some cython >>> >>> %%(end) >>> >>> some python >>> ``` >>> >>> -- >>> Matthias >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From hemmecke at gmail.com Thu May 31 14:13:19 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Thu, 31 May 2012 20:13:19 +0200 Subject: [IPython-dev] notebook %magic In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: <4FC7B4BF.2000701@gmail.com> I learnt about this "magic" first time from the sage-notebook. Sage-notebook has/had a way to allow fricas to be running natively. It's now some time ago that I modified the underlying code in such a way that all cells would be of fricas type. In fact, the %magicword got into the way. In fricas %pi is an way to input a symbolic \pi. And that should not be considered something magic. Since at that time the magic was unimportant for me, I changed the magic % into %%% and that allowed me to switch to a sage type cell by putting %%%sage at the beginning of a cell. What I want to say by this is that sticking to % as the magic prefix is totally bad for my use case. 1) Is there a way to change the magic prefix from % to something user defined? It would be OK for me if this is done at startup of the notebook. I'm not sure for the ipython notebook, but I remember that in the sage-notebook the % prefix was actually stored inside the notebook itself without having a global setting at the beginning of the notebook to set this prefix character. 2) What the default cell type is could maybe set at start time of the notebook server and then have (similar to the sage-notebook (as I knew it)) a menu that allows to select the default cell type. Ralf From ellisonbg at gmail.com Thu May 31 14:15:37 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 11:15:37 -0700 Subject: [IPython-dev] notebook %magic In-Reply-To: <4FC7B4BF.2000701@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> Message-ID: On Thu, May 31, 2012 at 11:13 AM, Ralf Hemmecke wrote: > I learnt about this "magic" first time from the sage-notebook. > Sage-notebook has/had a way to allow fricas to be running natively. > > It's now some time ago that I modified the underlying code in such a way > that all cells would be of fricas type. In fact, the %magicword got into > the way. In fricas %pi is an way to input a symbolic \pi. And that > should not be considered something magic. > > Since at that time the magic was unimportant for me, I changed the magic > % into %%% and that allowed me to switch to a sage type cell by putting > %%%sage at the beginning of a cell. > > What I want to say by this is that sticking to % as the magic prefix is > totally bad for my use case. > > 1) Is there a way to change the magic prefix from % to something user > defined? It would be OK for me if this is done at startup of the > notebook. I'm not sure for the ipython notebook, but I remember that in > the sage-notebook the % prefix was actually stored inside the notebook > itself without having a global setting at the beginning of the notebook > to set this prefix character. Because of how we implement magics in the kernel, this would not be possible. > 2) What the default cell type is could maybe set at start time of the > notebook server and then have (similar to the sage-notebook (as I knew > it)) a menu that allows to select the default cell type. I think these things will have to be set in the frontend, not the kernel/server. Cheers, Brian > Ralf > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From bussonniermatthias at gmail.com Thu May 31 14:19:07 2012 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Thu, 31 May 2012 20:19:07 +0200 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <2ADEF2DD-08B0-476A-8221-75D00590D914@gmail.com> <43725CAC-FC3C-4574-BB8B-2D9325F97065@gmail.com> Message-ID: Le 31 mai 2012 ? 20:04, Brian Granger a ?crit : > I think you have to insert 2 empty lines to indicate the end of the cell magic. Well, no, it does not work and cell magics only works if they are on the first line of a file... it gives strange things if they are not... So right now now cell magics in scripts. we'll do that for a later IPep. -- Matthias > > On Thu, May 31, 2012 at 11:03 AM, Matthias BUSSONNIER > wrote: >> >> Le 31 mai 2012 ? 19:47, Brian Granger a ?crit : >> >>> I am not sure, but I think it will work in an .ipy file. >> >> Yes, it works, but the cell magic execute until the end of the file. >> How difficult would it be to execute the cell magics until next line beginning with `%%`then carry on normal execution ? >> >> -- >> Matthias >> >>> >>> On Thu, May 31, 2012 at 10:46 AM, Matthias BUSSONNIER >>> wrote: >>>> >>>> Le 31 mai 2012 ? 19:33, Brian Granger a ?crit : >>>>> On Thu, May 31, 2012 at 10:29 AM, Jason Grout >>>>> wrote: >>>>>> On 5/31/12 12:24 PM, Brian Granger wrote: >>>>>>> Or you could do this as a cell magic... >>>>>> >>>>>> I'm curious: is there a way to set a default cell magic that is >>>>>> automatically applied to each and every cell? I can imagine us having a >>>>>> %%sage cell magic, and it being nice to do: >>>>>> >>>>>> %set_default_cell_magic sage >>>>>> >>>>>> or something, and then from then on, it's practically like I'm running >>>>>> Sage. Or R. Or fricas. Or whatever. >>>> >>>> This make me think, how do the cell magics works if one export a ipython session to a .ipy file ? >>>> How is handled the end of the 'cell' or block ? >>>> >>>> Can you write a script, smith like >>>> >>>> ``` >>>> some python >>>> >>>> %%cython >>>> some cython >>>> >>>> %%(end) >>>> >>>> some python >>>> ``` >>>> >>>> -- >>>> Matthias >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >>> >>> -- >>> Brian E. Granger >>> Cal Poly State University, San Luis Obispo >>> bgranger at calpoly.edu and ellisonbg at gmail.com >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Thu May 31 14:39:01 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 11:39:01 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: On Thu, May 31, 2012 at 10:33 AM, Brian Granger wrote: > We have started tho think about things like that, but haven't quite > gotten there. ?I think the main question I have is whether or not this > transformation should be applied by the frontends or the kernel. ?The > problem with the kernel is that there could be multiple frontends > connected to the kernel and it would be very confusing for those users > to all of a sudden to switch modes. ?Also, I don't like the idea of > having stateful magics like this. ?We have tried that with the %autopx > magic is it tends to be very confusing. ?I guess as I write, I am > leaning towards this being a frontend transformation. My thinking is actually looking a bit different right now, let me try to explain. For a long time we had the notion of a user-configurable prefilter function, that would be given every line of input to transform. That's becaue IPython didn't have a concept of cells or much else beyond the current line. Sage (and others) plugged their syntactic transformations via this mechanism, as was intended. It worked, but it's not ideal. Now that we have a concept of cells, I'd like to see all users of the idea of custom IPython-based environments to move to a cell-based approach. The transformations pipeline is now reasonably well encapsulated, so we can come up with a clean API for user-defined transformations that work at the cell level. Users should be able to define in a profile a function to be applied in the pipeline, that gets a cell and returns a cell. But that means that this would happen at the kernel level, not the client level, and I think that's OK: these would be highly customized kernels, and such that you'd want them to behave identically no matter what client you're using to control them. The above doesn't prevent someone from writing a client that does extra stuff to its input before sending it to an IPython kernel. Since the kernel is agnostic to the clients, people can already do that and we'd never know. Someone could for example subclass our Qtconsole and write one that has additional transformations on all input typed before it sends it out. But I think that we should think up an official, clean way of updating the old mishmash of prefiltering transforms for future use. Sage isn't the only project doing this kind of thing, so having an official API for it is, I think, a good idea. Cheers, f From benjaminrk at gmail.com Thu May 31 14:40:31 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 31 May 2012 11:40:31 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: Anyone else seeing this test failure: ImportError: Building module _test_cython_pyximport failed: ["DistutilsSetupError: each element of 'ext_modules' option must be an Extension instance or 2-tuple\n"] ? The reason being that Extension has changed due to autoreload, and an isinstance check in distutils is failing as a result. I can confirm that it is autoreload-related, because the tests pass in isolation, but if I run an autoreload test *in between* cython tests, I see the above failure: $> iptest IPython.extensions.cythonmagic IPython.extensions.tests.test_autoreload IPython.extensions.tests.test_cythonmagic I don't know enough to be sure whether this is a bug in autoreload, or a conflict with pyximport, or what. The autoreload tests should probably not be allowed to run in the same process as the rest of the suite. -MinRK On Thu, May 31, 2012 at 10:21 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 10:04 AM, MinRK wrote: > > I apologize if this is obvious, but are we aware that the > test_cythonmagic > > file is *not* excluded in master? > > > > if not have['cython']: > > exclusions.extend([ipjoin('extensions', 'cythonmagic')]) > > > > should be: > > > > if not have['cython']: > > exclusions.extend([ipjoin('extensions', 'cythonmagic')]) > > exclusions.extend([ipjoin('extensions', 'tests', > > 'test_cythonmagic')]) > > > > With this addition, and a manually disabled Cython, testing works fine > for > > me. > > Yes, I'd noticed that, but in my local testing I still get the problem > even with that fix. > > In fact, this is what I have: > > if not have['cython']: > exclusions.extend([ipjoin('extensions', 'cythonmagic')]) > exclusions.extend([ipjoin('extensions', 'cythonmagic.py')]) > exclusions.extend([ipjoin('extensions', 'test', > 'test_cythonmagic')]) > exclusions.extend([ipjoin('extensions', 'test', > 'test_cythonmagic.py')]) > > ARGGHHH!!! Nevermind, I fixed the exclusions list with 'test' not > 'tests' as the subdir. I'm a moron. > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu May 31 14:43:24 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 11:43:24 -0700 Subject: [IPython-dev] notebook %magic In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> Message-ID: On Thu, May 31, 2012 at 11:15 AM, Brian Granger wrote: >> 1) Is there a way to change the magic prefix from % to something user >> defined? It would be OK for me if this is done at startup of the >> notebook. I'm not sure for the ipython notebook, but I remember that in >> the sage-notebook the % prefix was actually stored inside the notebook >> itself without having a global setting at the beginning of the notebook >> to set this prefix character. > > Because of how we implement magics in the kernel, this would not be possible. A bit more detail: that logic is delicate enough that in the recent big magics refactoring, I made the explicit decision to punt and hardcode '%%' as the cell magic prefix. While one *could* in principle make the magic escapes configurable, there's a LOT of code in IPython by now where that is hardcoded, including multiple delicate regular expressions that are used on every single line of input. I simply don't think it's realistic right now to make that configurable. Cheers, f From ellisonbg at gmail.com Thu May 31 14:49:13 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 11:49:13 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: On Thu, May 31, 2012 at 11:39 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 10:33 AM, Brian Granger wrote: >> We have started tho think about things like that, but haven't quite >> gotten there. ?I think the main question I have is whether or not this >> transformation should be applied by the frontends or the kernel. ?The >> problem with the kernel is that there could be multiple frontends >> connected to the kernel and it would be very confusing for those users >> to all of a sudden to switch modes. ?Also, I don't like the idea of >> having stateful magics like this. ?We have tried that with the %autopx >> magic is it tends to be very confusing. ?I guess as I write, I am >> leaning towards this being a frontend transformation. > > My thinking is actually looking a bit different right now, let me try > to explain. ?For a long time we had the notion of a user-configurable > prefilter function, that would be given every line of input to > transform. ?That's becaue IPython didn't have a concept of cells or > much else beyond the current line. ?Sage (and others) plugged their > syntactic transformations via this mechanism, as was intended. ?It > worked, but it's not ideal. > > Now that we have a concept of cells, I'd like to see all users of the > idea of custom IPython-based environments to move to a cell-based > approach. ?The transformations pipeline is now reasonably well > encapsulated, so we can come up with a clean API for user-defined > transformations that work at the cell level. ?Users should be able to > define in a profile a function to be applied in the pipeline, that > gets a cell and returns a cell. > > But that means that this would happen at the kernel level, not the > client level, and I think that's OK: these would be highly customized > kernels, and such that you'd want them to behave identically no matter > what client you're using to control them. Doing the transformation always for all kernels solves the nasty stateful issues we ran into with %autopx. > The above doesn't prevent someone from writing a client that does > extra stuff to its input before sending it to an IPython kernel. > Since the kernel is agnostic to the clients, people can already do > that and we'd never know. ?Someone could for example subclass our > Qtconsole and write one that has additional transformations on all > input typed before it sends it out. > > But I think that we should think up an official, clean way of updating > the old mishmash of prefiltering transforms for future use. ?Sage > isn't the only project doing this kind of thing, so having an official > API for it is, I think, a good idea. I think there are some other issues we have to be careful about: * Implementing it purely in the kernel would possibly break the terminal and qtconsole as they rely on input splitter to make decisions for the frontend. Our transformation pipeline is actually quite complex in that code goes through multiple passes of the input splitter for static transformations and then the prefilter for dynamic ones. * This further opens the door for the notebook to support other languages. People will want things like tab completion, syntax highlighting, tooltips, etc to work. This will require some very careful thinking because I am not sure we want to make IPython fully multi-language aware. This is really important though and we need to think about these things. Cheers, Brian > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Thu May 31 14:49:36 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 11:49:36 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: Hey Min, On Thu, May 31, 2012 at 11:40 AM, MinRK wrote: > Anyone else seeing this test failure: > > ImportError: Building module _test_cython_pyximport failed: > ["DistutilsSetupError: each element of 'ext_modules' option must be an > Extension instance or 2-tuple\n"] > > ? > > The reason being that Extension has changed due to autoreload, and an > isinstance check in distutils is failing as a result. > > I can confirm that it is autoreload-related, because the tests pass in > isolation, but if I run an autoreload test *in between* cython tests, I see > the above failure: > >>gt; iptest IPython.extensions.cythonmagic >> IPython.extensions.tests.test_autoreload >> IPython.extensions.tests.test_cythonmagic > > I don't know enough to be sure whether this is a bug in autoreload, or a > conflict with pyximport, or what. > > The autoreload tests should probably not be allowed to run in the same > process as the rest of the suite. No, I haven't seen the error, but I agree with you that we probably want autoreload testing all by itself. Right now unfortunately our test grouping is done by package, so this isolation may require a bit of custom code, I'm afraid. From benjaminrk at gmail.com Thu May 31 14:53:22 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 31 May 2012 11:53:22 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 11:49 AM, Fernando Perez wrote: > Hey Min, > > On Thu, May 31, 2012 at 11:40 AM, MinRK wrote: > > Anyone else seeing this test failure: > > > > ImportError: Building module _test_cython_pyximport failed: > > ["DistutilsSetupError: each element of 'ext_modules' option must be an > > Extension instance or 2-tuple\n"] > > > > ? > > > > The reason being that Extension has changed due to autoreload, and an > > isinstance check in distutils is failing as a result. > > > > I can confirm that it is autoreload-related, because the tests pass in > > isolation, but if I run an autoreload test *in between* cython tests, I > see > > the above failure: > > > >>gt; iptest IPython.extensions.cythonmagic > >> IPython.extensions.tests.test_autoreload > >> IPython.extensions.tests.test_cythonmagic > > > > I don't know enough to be sure whether this is a bug in autoreload, or a > > conflict with pyximport, or what. > > > > The autoreload tests should probably not be allowed to run in the same > > process as the rest of the suite. > > No, I haven't seen the error, but I agree with you that we probably > want autoreload testing all by itself. Right now unfortunately our > test grouping is done by package, so this isolation may require a bit > of custom code, I'm afraid. > Two simple options: 1. move test_autoreload somewhere else 2. always exclude test_autoreload, and give it its own group without that exclusion > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu May 31 14:56:51 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 11:56:51 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: On Thu, May 31, 2012 at 11:49 AM, Brian Granger wrote: > I think there are some other issues we have to be careful about: > > * Implementing it purely in the kernel would possibly break the > terminal and qtconsole as they rely on input splitter to make > decisions for the frontend. ?Our transformation pipeline is actually > quite complex in that code goes through multiple passes of the input > splitter for static transformations and then the prefilter for dynamic > ones. Not quite: these clients actually instantiate their own inputsplitter object, so they are using the same *library* as the kernel, but not the same *object*. What this does mean is that if somebody implements a kernel with highly customized input logic, they can't expect the default IPython clients to play nicely with it, they will need to also write custom clients. But that seems fair to me: if you deviate a lot from the default semantics and you're working with a system that has both clients and servers, you need to ensure your new semantics are handled consistently everywhere. > * This further opens the door for the notebook to support other > languages. ?People will want things like tab completion, syntax > highlighting, tooltips, etc to work. ?This will require some very > careful thinking because I am not sure we want to make IPython fully > multi-language aware. > > This is really important though and we need to think about these things. Yes, the multi-language support does open some pretty delicate issues from the client side, if we want to do it system-wide. With custom cells like we have right now with Cython and R the problems aren't too severe (though already tab completion and highlighting will be wrong in those cells), but if we want to do it system-wide, it will indeed require some very careful thought. Cheers, f From fperez.net at gmail.com Thu May 31 14:57:33 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 11:57:33 -0700 Subject: [IPython-dev] Cython magics crashes test suite when cython not present In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 11:53 AM, MinRK wrote: > Two simple options: > > 1. move test_autoreload somewhere else > 2. always exclude test_autoreload, and give it its own group without that > exclusion I think I prefer #2, so we don't create a bizarre directory layout just for this... From ellisonbg at gmail.com Thu May 31 15:12:44 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 12:12:44 -0700 Subject: [IPython-dev] ipython notebook installation instructions In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> Message-ID: On Thu, May 31, 2012 at 11:56 AM, Fernando Perez wrote: > On Thu, May 31, 2012 at 11:49 AM, Brian Granger wrote: >> I think there are some other issues we have to be careful about: >> >> * Implementing it purely in the kernel would possibly break the >> terminal and qtconsole as they rely on input splitter to make >> decisions for the frontend. ?Our transformation pipeline is actually >> quite complex in that code goes through multiple passes of the input >> splitter for static transformations and then the prefilter for dynamic >> ones. > > Not quite: ?these clients actually instantiate their own inputsplitter > object, so they are using the same *library* as the kernel, but not > the same *object*. ?What this does mean is that if somebody implements > a kernel with highly customized input logic, they can't expect the > default IPython clients to play nicely with it, they will need to also > write custom clients. Yes, the input splitter on the frontend side will have to be configured in the same way as the one on the kernel. If the frontend and kernel are running different hosts, that is problematic. There is another aspect of this. The ability of the input splitter to work properly depends on a very fragile set of regular expressions and line by line state. I am not sure if it is possible to get all of that to work while simultaneously allowing 1) arbitrary text in cells and 2) arbitrary transformations of that text. It is possible we could get this to work, but I am not even sure it can be done in a consistent way. I think the idea of putting these things in the kernel might make sense, but to play devils advocate for a second... Another reason to have this in the frontends. If someone writes a notebook, and that notebook has R syntax in all of its cells, that notebook won't run on anything but R-transforming kernels. When an Python notebook runs on an R kernel, or vice-versa, you will get confusing SyntaxErrors. This requires a separate notebook server or kernel manager for each such type of notebook. What happens when I have users that have R notebooks, Matlab notebooks, etc., etc. You end up with an incredibly difficult situation to manage, both for users, the person running the notebook server and the developers of the notebook. I think the reality is that most users will want to switch back and forth between different syntaxes. We need to make that simple, easy and lightweight and I don't see how to do that with a kernel-side approach. > But that seems fair to me: if you deviate a lot from the default > semantics and you're working with a system that has both clients and > servers, you need to ensure your new semantics are handled > consistently everywhere. > >> * This further opens the door for the notebook to support other >> languages. ?People will want things like tab completion, syntax >> highlighting, tooltips, etc to work. ?This will require some very >> careful thinking because I am not sure we want to make IPython fully >> multi-language aware. >> >> This is really important though and we need to think about these things. > > Yes, the multi-language support does open some pretty delicate issues > from the client side, if we want to do it system-wide. ?With custom > cells like we have right now with Cython and R the problems aren't too > severe (though already tab completion and highlighting will be wrong > in those cells), but if we want to do it system-wide, it will indeed > require some very careful thought. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jason-sage at creativetrax.com Thu May 31 15:15:16 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 31 May 2012 14:15:16 -0500 Subject: [IPython-dev] notebook %magic In-Reply-To: <4FC7B4BF.2000701@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> Message-ID: <4FC7C344.4070301@creativetrax.com> On 5/31/12 1:13 PM, Ralf Hemmecke wrote: > 1) Is there a way to change the magic prefix from % to something user > defined? It would be OK for me if this is done at startup of the > notebook. I'm not sure for the ipython notebook, but I remember that in > the sage-notebook the % prefix was actually stored inside the notebook > itself without having a global setting at the beginning of the notebook > to set this prefix character. That's right; it's hardcoded into the Sage notebook. Thanks, Jason From ellisonbg at gmail.com Thu May 31 15:20:34 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 12:20:34 -0700 Subject: [IPython-dev] Issue with AsyncResult.display_output Message-ID: Min, I am seeing some weird behavior with AsyncResult when targets is only a single engine. Basically, an extra \n is being added to stdout and stderr. This only happens when there is a single engine's results being re-published. Any thoughts on this one? Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Thu May 31 15:45:41 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 31 May 2012 12:45:41 -0700 Subject: [IPython-dev] Issue with AsyncResult.display_output In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 12:20 PM, Brian Granger wrote: > Min, > > I am seeing some weird behavior with AsyncResult when targets is only > a single engine. Basically, an extra \n is being added to stdout and > stderr. This only happens when there is a single engine's results > being re-published. Any thoughts on this one? > Simple condition was missing: https://github.com/ipython/ipython/pull/1812 > > Cheers, > > Brian > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu May 31 16:01:31 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 13:01:31 -0700 Subject: [IPython-dev] Issue with AsyncResult.display_output In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 12:45 PM, MinRK wrote: > > > On Thu, May 31, 2012 at 12:20 PM, Brian Granger wrote: >> >> Min, >> >> I am seeing some weird behavior with AsyncResult when targets is only >> a single engine. ?Basically, an extra \n is being added to stdout and >> stderr. ?This only happens when there is a single engine's results >> being re-published. ?Any thoughts on this one? > > > Simple condition was missing:?https://github.com/ipython/ipython/pull/1812 OK great, I will review there. Thanks, Brian >> >> >> Cheers, >> >> Brian >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From hemmecke at gmail.com Thu May 31 16:32:22 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Thu, 31 May 2012 22:32:22 +0200 Subject: [IPython-dev] notebook %magic In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> Message-ID: <4FC7D556.6040304@gmail.com> > A bit more detail: that logic is delicate enough that in the recent > big magics refactoring, I made the explicit decision to punt and > hardcode '%%' as the cell magic prefix. :-( Although... %% is not as bad as % for me. But maybe I don't yet understand enough this magic business. I'll probably get back after reading some more code. If you were telling me that typing %pi at the beginning of an input (or output) line would be interpreted correctly as %pi (from fricas) and nothing else, then I'd be satisfied. Anyway, if the ipython notebook should some day be able to be used as a frontend for maple or mathematica then %, %% and %%% are bad, because of their meaning inside the respective system. > While one *could* in principle make the magic escapes configurable, > there's a LOT of code in IPython by now where that is hardcoded, > including multiple delicate regular expressions that are used on > every single line of input. Of course, I don't yet know the source code, but why wouldn't it be a feasible way to start the notebook code with something like escapesequence='%$' escaperegex='%\$' and replace the 'delicate regular expressions' appropriately? > I simply don't think it's realistic right now to make that > configurable. Yes, of course. It's open source and different people have different goals. No problem. Although... of course, I would be happier with a configurable escape mechanism. Ralf From takowl at gmail.com Thu May 31 16:49:02 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 31 May 2012 21:49:02 +0100 Subject: [IPython-dev] notebook %magic In-Reply-To: <4FC7D556.6040304@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> Message-ID: On 31 May 2012 21:32, Ralf Hemmecke wrote: > Anyway, if the ipython notebook should some day be able to be used as a > frontend for maple or mathematica then %, %% and %%% are bad, because of > their meaning inside the respective system. As far as I know, the notebook itself doesn't know about the escape sequences - that's all processed in the kernel. A hypothetical maple or mathematica kernel wouldn't process ipython escape sequences, so it shouldn't be an issue. > Of course, I don't yet know the source code, but why wouldn't it be a > feasible way to start the notebook code with something like > > escapesequence='% > > escaperegex='%\ It's *feasible*, but not practical, because we've written the code with the assumption that the escape character is not configurable, and we've got enough to do at the moment without introducing the extra complexity to make it easy to change. If I remember the history right, magic functions were originally @foo, but then Python gained decorators using @, so magic functions were changed to %foo. Best wishes, Thomas From hemmecke at gmail.com Thu May 31 17:02:40 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Thu, 31 May 2012 23:02:40 +0200 Subject: [IPython-dev] notebook %magic In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> Message-ID: <4FC7DC70.5030809@gmail.com> On 05/31/2012 10:49 PM, Thomas Kluyver wrote: > On 31 May 2012 21:32, Ralf Hemmecke wrote: >> Anyway, if the ipython notebook should some day be able to be used as a >> frontend for maple or mathematica then %, %% and %%% are bad, because of >> their meaning inside the respective system. > > As far as I know, the notebook itself doesn't know about the escape > sequences - that's all processed in the kernel. A hypothetical maple > or mathematica kernel wouldn't process ipython escape sequences, so it > shouldn't be an issue. That maple/mathematica wouldn't process or even see magic sequences is clear. I am more afraid of the fact that input/output lines of mathematica/maple/fricas could confuse the program part that deals with magic lines. Just suppose my notebook simply contains a cell that only has the line %pi I don't yet know what ipython would be doing with that line, but it looks like magic so I guess that cell would be empty if it is seen by the fricas engine. Am I wrong? But I'd like fricas to see '%pi'. Ralf From bussonniermatthias at gmail.com Thu May 31 17:09:22 2012 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Thu, 31 May 2012 23:09:22 +0200 Subject: [IPython-dev] notebook %magic In-Reply-To: <4FC7DC70.5030809@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> <4FC7DC70.5030809@gmail.com> Message-ID: wouldn't a %%fricas cell magic get the whole text of the cell and be able to process %pi in a different manner than what IPython does now ? On Thu, May 31, 2012 at 11:02 PM, Ralf Hemmecke wrote: > On 05/31/2012 10:49 PM, Thomas Kluyver wrote: > > On 31 May 2012 21:32, Ralf Hemmecke wrote: > >> Anyway, if the ipython notebook should some day be able to be used as a > >> frontend for maple or mathematica then %, %% and %%% are bad, because of > >> their meaning inside the respective system. > > > > As far as I know, the notebook itself doesn't know about the escape > > sequences - that's all processed in the kernel. A hypothetical maple > > or mathematica kernel wouldn't process ipython escape sequences, so it > > shouldn't be an issue. > > That maple/mathematica wouldn't process or even see magic sequences is > clear. I am more afraid of the fact that input/output lines of > mathematica/maple/fricas could confuse the program part that deals with > magic lines. Just suppose my notebook simply contains a cell that only > has the line > > %pi > > I don't yet know what ipython would be doing with that line, but it > looks like magic so I guess that cell would be empty if it is seen by > the fricas engine. Am I wrong? But I'd like fricas to see '%pi'. > > Ralf > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Thu May 31 17:15:45 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 31 May 2012 16:15:45 -0500 Subject: [IPython-dev] notebook %magic In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> <4FC7DC70.5030809@gmail.com> Message-ID: <4FC7DF81.6000803@creativetrax.com> On 5/31/12 4:09 PM, Matthias Bussonnier wrote: > wouldn't a %%fricas cell magic get the whole text of the cell and be > able to process %pi in a different manner than what IPython does now ? > Does IPython look for line-level magics inside of cell-level magics? Would IPython consider %pi a line-level magic invocation, even if it was in a %%fricas cell? Jason From ellisonbg at gmail.com Thu May 31 17:16:40 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 14:16:40 -0700 Subject: [IPython-dev] notebook %magic In-Reply-To: <4FC7DF81.6000803@creativetrax.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> <4FC7DC70.5030809@gmail.com> <4FC7DF81.6000803@creativetrax.com> Message-ID: No, when a cell begins with `%%`, all further preprocessing of the cell is halted, so `%pi` in the middle of a fricas cell would work just fine. On Thu, May 31, 2012 at 2:15 PM, Jason Grout wrote: > On 5/31/12 4:09 PM, Matthias Bussonnier wrote: >> wouldn't a %%fricas cell magic get the whole text of the cell and be >> able to process %pi in a different manner than what IPython does now ? >> > > Does IPython look for line-level magics inside of cell-level magics? > Would IPython consider %pi a line-level magic invocation, even if it was > in a %%fricas cell? > > Jason > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From hemmecke at gmail.com Thu May 31 17:59:02 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Thu, 31 May 2012 23:59:02 +0200 Subject: [IPython-dev] notebook %magic In-Reply-To: References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> <4FC7DC70.5030809@gmail.com> <4FC7DF81.6000803@creativetrax.com> Message-ID: <4FC7E9A6.3010404@gmail.com> Oh, there is line-level and cell-level... I didn't know. Anyway, eventually, I'd like to have fricas the default type of all cells without explicitly specifying it in each cell. Sorry that I don't yet understand, whether that is possible at all. I'm currently trying to install ipython and the connection to R. Ralf On 05/31/2012 11:16 PM, Brian Granger wrote: > No, when a cell begins with `%%`, all further preprocessing of the > cell is halted, so `%pi` in the middle of a fricas cell would work > just fine. > > On Thu, May 31, 2012 at 2:15 PM, Jason Grout > wrote: >> On 5/31/12 4:09 PM, Matthias Bussonnier wrote: >>> wouldn't a %%fricas cell magic get the whole text of the cell and be >>> able to process %pi in a different manner than what IPython does now ? >>> >> >> Does IPython look for line-level magics inside of cell-level magics? >> Would IPython consider %pi a line-level magic invocation, even if it was >> in a %%fricas cell? >> >> Jason >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > From ellisonbg at gmail.com Thu May 31 18:00:40 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 15:00:40 -0700 Subject: [IPython-dev] notebook %magic In-Reply-To: <4FC7E9A6.3010404@gmail.com> References: <4FC74053.5060509@gmail.com> <4FC7AA82.4030103@creativetrax.com> <4FC7B4BF.2000701@gmail.com> <4FC7D556.6040304@gmail.com> <4FC7DC70.5030809@gmail.com> <4FC7DF81.6000803@creativetrax.com> <4FC7E9A6.3010404@gmail.com> Message-ID: On Thu, May 31, 2012 at 2:59 PM, Ralf Hemmecke wrote: > Oh, there is line-level and cell-level... I didn't know. > > Anyway, eventually, I'd like to have fricas the default type of all > cells without explicitly specifying it in each cell. That is not possible, but it is something we are (can you tell) actively thinking about. > Sorry that I don't yet understand, whether that is possible at all. I'm > currently trying to install ipython and the connection to R. > > Ralf > > On 05/31/2012 11:16 PM, Brian Granger wrote: >> No, when a cell begins with `%%`, all further preprocessing of the >> cell is halted, so `%pi` in the middle of a fricas cell would work >> just fine. >> >> On Thu, May 31, 2012 at 2:15 PM, Jason Grout >> ?wrote: >>> On 5/31/12 4:09 PM, Matthias Bussonnier wrote: >>>> wouldn't a %%fricas cell magic get the whole text of the cell and be >>>> able to process %pi in a different manner than what IPython does now ? >>>> >>> >>> Does IPython look for line-level magics inside of cell-level magics? >>> Would IPython consider %pi a line-level magic invocation, even if it was >>> in a %%fricas cell? >>> >>> Jason >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From asmeurer at gmail.com Thu May 31 18:03:53 2012 From: asmeurer at gmail.com (Aaron Meurer) Date: Thu, 31 May 2012 16:03:53 -0600 Subject: [IPython-dev] Cython magics have been merged! In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 11:22 PM, Fernando Perez wrote: > Hi folks, > > we just concluded review and merge of the cython magics: > https://github.com/ipython/ipython/pull/1770! > > Kudos to Brian for a terrific job, and please let us know if you spot > any problems. ?There's an example notebook provided > (docs/examples/notebooks/cython_extension.ipynb) so you can see it in > action. This is really cool. One suggestion: when you save your notebooks, restart the kernel and run all cells in order, so that the numbering is in order. It's a little confusing to see random numberings throughout the example notebooks. Maybe there could be an option to always save notebooks with the cell numbers in order. Aaron Meurer > > We think this will be very useful, and the system also serves as a > reference on how to implement other extensions with useful magics. > > The R one is also in review and very close to ready: > https://github.com/ipython/ipython/pull/1780, so pitch in with your > feedback if there's anything there you think can be improved. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From hemmecke at gmail.com Thu May 31 19:25:23 2012 From: hemmecke at gmail.com (Ralf Hemmecke) Date: Fri, 01 Jun 2012 01:25:23 +0200 Subject: [IPython-dev] local ipython install from github Message-ID: <4FC7FDE3.7010509@gmail.com> Maybe this is interesting for someone or for the ipython documentation... https://sites.google.com/site/hemmecke/ipython-nb Ralf From fperez.net at gmail.com Thu May 31 19:48:13 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 16:48:13 -0700 Subject: [IPython-dev] Cython magics have been merged! In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 3:03 PM, Aaron Meurer wrote: > This is really cool. We're very excited, and the R one is nearly ready, should be in tomorrow! > One suggestion: when you save your notebooks, restart the kernel and > run all cells in order, so that the numbering is in order. ?It's a > little confusing to see random numberings throughout the example > notebooks. ?Maybe there could be an option to always save notebooks > with the cell numbers in order. Very true, we need to keep that in mind. Cheers, f From fperez.net at gmail.com Thu May 31 20:30:05 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 17:30:05 -0700 Subject: [IPython-dev] Another monster landed: refactored JavaScript, new tooltips and interactive widgets in the notebook Message-ID: Hi folks, After the merge of the previous big two branches (the merging of the interactive and parallel kernels into one object and the magics refactoring/cell support), the third monster we had in flight has now landed: https://github.com/ipython/ipython/pull/1711/ For those of you not following the github hooks obsessively, here's the high-level view of the goodies: - New graphical tooltip with keyboard control: successive presses of the TAB key will expand it, pin it for 10 s (a clock icon is shown), and send its content to the bottom help pager (which is now resizable). - Completer has been refactored into a new class and now can analyze the current cell to support completions on objects that don't exist in the kernel yet (because the cell hasn't been executed). - All the client-side JavaScript has been decoupled to ease reuse of parts of the machinery without having to build a full-blown notebook. This will make it much easier to communicate with an IPython kernel from existing web pages and to integrate single cells into other sites, without loading the full notebook document-like UI. - This refactoring also enables the possibility of writing dynamic javascript widgets that are returned from Python code and that present an interactive view to the user, with callbacks in Javascript executing calls to the Kernel. This will enable many interactive elements to be added by users in notebooks. An example of this capability has been provided as a proof of concept in `docs/examples/widgets` that lets you directly communicate with one or more parallel engines, acting as a mini-console for parallel debugging and introspection. Kudos to Brian and Matthias for the excellent work they put into this, it was over 84 commits and involved the merging of several intermediate pull requests. But the goodies shipped therein should be *very* useful. We've seen recently attempts at interactive plots with d3/flot and other similar tools; now the machinery is ready to do that cleanly, so by all means have fun! Cheers, f From jason-sage at creativetrax.com Thu May 31 21:59:22 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 31 May 2012 20:59:22 -0500 Subject: [IPython-dev] Another monster landed: refactored JavaScript, new tooltips and interactive widgets in the notebook In-Reply-To: References: Message-ID: <4FC821FA.6080001@creativetrax.com> On 5/31/12 7:30 PM, Fernando Perez wrote: > - All the client-side JavaScript has been decoupled to ease reuse of > parts of the machinery without having to build a full-blown notebook. > This will make it much easier to communicate with an IPython kernel > from existing web pages and to integrate single cells into other > sites, without loading the full notebook document-like UI. Congrats on the major developments. We're right now using this one (above) to implement a new version of the Sage cell server. We have a simple demo running, and are working on that forking kernel manager from the other thread, as well as an ipcluster-based solution for managing kernels, to provide a backend. Thanks again! Jason From fperez.net at gmail.com Thu May 31 22:06:13 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 19:06:13 -0700 Subject: [IPython-dev] Another monster landed: refactored JavaScript, new tooltips and interactive widgets in the notebook In-Reply-To: <4FC821FA.6080001@creativetrax.com> References: <4FC821FA.6080001@creativetrax.com> Message-ID: On Thu, May 31, 2012 at 6:59 PM, Jason Grout wrote: > Congrats on the major developments. ?We're right now using this one > (above) to implement a new version of the Sage cell server. ?We have a > simple demo running, and are working on that forking kernel manager from > the other thread, as well as an ipcluster-based solution for managing > kernels, to provide a backend. Great, glad to hear that! We're starting to be very happy with the internal machinery, so we're excited to see it in use 'in the wild' already. Let us know when there's something up we can play with! Cheers, f From jason-sage at creativetrax.com Thu May 31 22:26:05 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 31 May 2012 21:26:05 -0500 Subject: [IPython-dev] Another monster landed: refactored JavaScript, new tooltips and interactive widgets in the notebook In-Reply-To: References: <4FC821FA.6080001@creativetrax.com> Message-ID: <4FC8283D.3000704@creativetrax.com> On 5/31/12 9:06 PM, Fernando Perez wrote: > On Thu, May 31, 2012 at 6:59 PM, Jason Grout > wrote: >> Congrats on the major developments. We're right now using this one >> (above) to implement a new version of the Sage cell server. We have a >> simple demo running, and are working on that forking kernel manager from >> the other thread, as well as an ipcluster-based solution for managing >> kernels, to provide a backend. > > Great, glad to hear that! We're starting to be very happy with the > internal machinery, so we're excited to see it in use 'in the wild' > already. Let us know when there's something up we can play with! Actually, we are reaching a point where we'd love some comments from you guys. The issue we have is that we need many short-lived, instant-startup kernels that are relatively secure (to execute arbitrary code). We've been working in parallel to explore using an ipcluster-based solution (where the engines are started by forking a master engine), as well as the ForkingKernelManager we talked about the other day (implemented as part of a multikernel manager, like in the present IPython notebook). We need to settle on an approach in the next few days and start implementing. Tomorrow our two groups are getting together to discuss the advantages/disadvantages of each approach, and if the approaches might be combined in some way. So we plan to post more details tomorrow and ask for advice and comments on our analysis. We hope to have a short demo to show a week after at the Sage Days in Seattle. Thanks, Jason From fperez.net at gmail.com Thu May 31 22:55:38 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 19:55:38 -0700 Subject: [IPython-dev] Bumping up priority of thinking about output scrolling/control Message-ID: Hey, Running code with %%px on 16 engines that had been wired to spit stdout by default is quickly making me think about a discussion we started earlier: https://github.com/ipython/ipython/issues/1553 Now that the 'big three' have landed, I think we can begin to scope what a 0.13 release would involve, and I'd *really* like to have some kind of solution for this. As Titus indicated in Boulder, without this some of his use cases were really unmanageable in practice, and I'm getting pretty close to that right now too... Bright ideas? f From ellisonbg at gmail.com Thu May 31 23:22:35 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 31 May 2012 20:22:35 -0700 Subject: [IPython-dev] Bumping up priority of thinking about output scrolling/control In-Reply-To: References: Message-ID: i can look into this. But the other option for the output of multiple engines is to build a simple widget that has a slider and allows you the slide through the output of the different engines. But I agree that we need a more general solution to this issue. On Thu, May 31, 2012 at 7:55 PM, Fernando Perez wrote: > Hey, > > Running code with %%px on 16 engines that had been wired to spit > stdout by default is quickly making me think about a discussion we > started earlier: > > https://github.com/ipython/ipython/issues/1553 > > Now that the 'big three' have landed, I think we can begin to scope > what a 0.13 release would involve, and I'd *really* like to have some > kind of solution for this. ?As Titus indicated in Boulder, without > this some of his use cases were really unmanageable in practice, and > I'm getting pretty close to that right now too... > > Bright ideas? > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Thu May 31 23:30:32 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 31 May 2012 20:30:32 -0700 Subject: [IPython-dev] Bumping up priority of thinking about output scrolling/control In-Reply-To: References: Message-ID: On Thu, May 31, 2012 at 8:22 PM, Brian Granger wrote: > i can look into this. ?But the other option for the output of multiple > engines is to build a simple widget that has a slider and allows you > the slide through the output of the different engines. ?But I agree > that we need a more general solution to this issue. I think it's worth thining about the clean solution for the long run: Titus had examples that were unmanageable even without any parallelism, simply because of very long running code that had super verbose stdout. Without something to manage output size, his demos would have been simply impossible. He actually did the public demos off Min's quick branch, but that's not a long-term solution :) So let's see what we can do for a clean solution we're happy with that applies even in the absence of multiple engines. Obviously your idea could also be quite useful for that case, but let's first think about what to do with stdout of absurd lengths... Cheers, f